apache /
kudu /
66a79dddf2faf31dbb49cd8e08c0e322c4bb57ce KUDU-3407 Avoid unchecked scheduling of flush operations.
In some clusters, the memory usages of tservers might be 60% ~ 80%
for a long time. During this time the maintenance manager will not
run any operation other than wal gc and MRS/DRS flushes, which will
make the performance of tservers worse and worse and eventually break
due to OOM.
This patch add an argument to give a chance to do other operations
while server is under memory pressure.
This mechanism works when the memory usage is between
memory_pressure_percentage and memory_limit_soft_percentage.
Higher the memory usage is, higher the probability to flush
MRS/DMS.
e.g.
memory_pressure_percentage = 60%
memory_limit_soft_percentage = 80%
The probability of not flushing MRS/DMS is the value of
run_non_memory_ops_prob. As the memory increases, it gradually
decreases to 0, when thememory usage is 80%.
Change-Id: Idc2fd3a850cf99d54ef2980211b712468440ed80
Reviewed-on: http://gerrit.cloudera.org:8080/20166
Tested-by: Kudu Jenkins
Reviewed-by: Alexey Serbin <alexey@apache.org>
3 files changed
