blob: 9d58ffcd5c625beaa3cdb8bd9fa877c5ea0441a1 [file] [log] [blame] [view]
# PIP-259: Make the config httpMaxRequestHeaderSize of the pulsar web server to configurable
Discuss: https://lists.apache.org/thread/f11cld5cbc8sodhgvs5s28lw8nxsr9dc
### Motivation
We have two ways to manage pulsar's resources:
- By client API (Can manage some resources, such as `create topic`, `create subscriber`, and so on)
- By admin API (Can manage all the resources)
The `client API` has no limit on the request length. And the `admin API` has a limit on the request length(such as HTTP request line and HTTP request headers), this limitation is implemented by the built-in web container Jetty.
Almost resources can be created by two APIs, but can only be modified and deleted by `admin API`. This causes us to be unable to modify or delete resources created by `client API` with too long a name because it exceeds Jetty's default HTTP request URI length limit.
### Goal
#### 1. For web servers
Provide a way to modify Jetty's `httpMaxRequestHeaderSize` configuration (involves two servers: the web server in pulsar and the web server in pulsar-proxy), then users can edit or delete the resources whose name is too long.
#### 2.For the internal HTTP client in pulsar-proxy
Set the config `httpClientRequestBufferSize` for the internal Jetty-client's of Pulsar-proxy.
Since the pulsar-proxy handles HTTP requests like this `pulsar-admin.sh` -> `proxy web server` -> `(highlight) internal client in proxy` -> `pulsar web server`.
When the internal client forwards a request, it forwards the request header and the request body, and all the data passes through a buffer( we call it Buf ), like this:
- Receive a request
- Put the request line and request headers input to the Buf.
- <strong>(highlight)</strong>Flush the Buf ( If the data in the request line and request header exceeds the length of the buf, an error is reported )
- Put the request body input to the Buf.
- Flush the Buf if it is full.
So we should set the `buff size` of the Buf: `pulsar-proxy.conf.httpClientRequestBufferSize` -> `buf size of the internal client`.
### API Changes
#### ServiceConfiguration.java
```java
@FieldContext(
category = CATEGORY_HTTP,
doc = """
The maximum size in bytes of the request header.
Larger headers will allow for more and/or larger cookies plus larger form content encoded in a URL.
However, larger headers consume more memory and can make a server more vulnerable to denial of service
attacks.
"""
)
private int httpMaxRequestHeaderSize = 8 * 1024;
```
#### ProxyConfiguration.java
```java
@FieldContext(
minValue = 1,
category = CATEGORY_HTTP,
doc = """
The maximum size in bytes of the request header.
Larger headers will allow for more and/or larger cookies plus larger form content encoded in a URL.
However, larger headers consume more memory and can make a server more vulnerable to denial of service
attacks.
"""
)
private int httpMaxRequestHeaderSize = 8 * 1024;
```
### Anything else
This change should cherry-pick into the previous branches ( includes `2.9~2.11` )
If the user uses the features `RETRY Topic` or `DLQ`, it is possible that pulsar will automatically create some topics with names that are too long and cannot be managed, the [scenario has been discussed in the email](https://lists.apache.org/thread/q1m23ckyy10wvtzy65v8bwqwnh7r0gc8) before