| |
| # PIP-358: let resource weight work for OverloadShedder, LeastLongTermMessageRate, ModularLoadManagerImpl. |
| |
| # Background knowledge |
| |
| Initially, we introduce `loadBalancerCPUResourceWeight`, `loadBalancerBandwidthInResourceWeight`, `loadBalancerBandwidthOutResourceWeight`, |
| `loadBalancerMemoryResourceWeight`, `loadBalancerDirectMemoryResourceWeight` in `ThresholdShedder` to control the resource weight for |
| different resources when calculating the load of the broker. |
| Then we let it work for `LeastResourceUsageWithWeight` for better bundle placement policy. |
| |
| But https://github.com/apache/pulsar/pull/19559 and https://github.com/apache/pulsar/pull/21168 have pointed out that the actual load |
| of the broker is not related to the memory usage and direct memory usage, thus we have changed the default value of |
| `loadBalancerMemoryResourceWeight`, `loadBalancerDirectMemoryResourceWeight` to 0.0. |
| |
| There are still some places where memory usage and direct memory usage are used to calculate the load of the broker, such as |
| `OverloadShedder`, `LeastLongTermMessageRate`, `ModularLoadManagerImpl`. We should let the resource weight work for these places |
| so that we can set the resource weight to 0.0 to avoid the impact of memory usage and direct memory usage on the load of the broker. |
| |
| # Motivation |
| |
| The actual load of the broker is not related to the memory usage and direct memory usage, thus we should let the resource weight work for |
| `OverloadShedder`, `LeastLongTermMessageRate`, `ModularLoadManagerImpl` so that we can set the resource weight to 0.0 to avoid the impact of |
| memory usage and direct memory usage on the load of the broker. |
| |
| |
| # Detailed Design |
| |
| Let resource weight work for `OverloadShedder`, `LeastLongTermMessageRate`, `ModularLoadManagerImpl`. |
| - For `OverloadShedder`, `LeastLongTermMessageRate`, we replace `getMaxResourceUsage()` with `getMaxResourceUsageWithWeight()` in the calculation of the load of the broker. |
| - For `ModularLoadManagerImpl`, we replace `getMaxResourceUsage()` with `getMaxResourceUsageWithWeight()` when checking if the broker is overloaded and decide whether to update the broker data to metadata store. |
| |
| # Backward & Forward Compatibility |
| |
| |
| # Links |
| |
| * Mailing List discussion thread: https://lists.apache.org/thread/lj34s3vmjbzlwmy8d66d0bsb25vnq9ky |
| * Mailing List voting thread: https://lists.apache.org/thread/b7dzm0yz6l40pkxmxhto5mro7brmz57r |