apache /
kudu /
740e9150d7072d95599951ee998df58d8ffe55b6 KUDU-2612 add perf scenario to TxnWriteOpsITest
This patch adds WriteOpPerf scenario to TxnWriteOpsITest. The new
scenario is to evaluate --tablet_max_pending_txn_write_ops setting for
tablet servers: it runs for a short time to count number of completed
Write RPCs in context of a transactional session. The scenario focuses
on single-row write operations to pinpoint the latency of processing
txn write operations when performing registration of transaction
participants. Probably, the current locking approach for the
TxnOpDispatcher's queue while submitting the accumulated operations
isn't optimal. Apparently, the exponential back-off timing built into
the client's RPC retry logic is also an important factor: I saw
significant deviation in the number of completed RPCs from run to run.
Below are results averaged for 100 runs of the benchmark scenario with
varying --max_pending_txn_write_ops accordingly and the following
settings fixed:
--prepare_connections_to_tservers=true
--clients=8
--sessions_per_client=1
--benchmark_run_time_ms=50
I used the following script to get the accumulated results for
writes in a transactional context:
for i in {0..99}; do
./bin/txn_write_ops-itest --gtest_filter='*WriteOpPerf' \
--max_pending_txn_write_ops=<X> 2>&1 | grep 'write RPCs' | \
awk '{print $9}'; done | \
awk 'BEGIN {sum=0} {sum += $0} END {print sum}'
RELEASE build:
--max_pending_txn_write_ops=0 : 442.13 RPCs
--max_pending_txn_write_ops=2 : 494.33 RPCs
--max_pending_txn_write_ops=5 : 471.90 RPCs
--max_pending_txn_write_ops=10 : 490.22 RPCs
--max_pending_txn_write_ops=20 : 469.21 RPCs
DEBUG build:
--max_pending_txn_write_ops=0 : 83.74 RPCs
--max_pending_txn_write_ops=2 : 98.18 RPCs
--max_pending_txn_write_ops=5 : 95.23 RPCs
--max_pending_txn_write_ops=10 : 98.12 RPCs
--max_pending_txn_write_ops=20 : 94.40 RPCs
I also measured the performance of transactional vs non-transactional
for various time intervals in RELEASE build, where for transactional
writes I used --max_pending_txn_write_ops=2 setting:
50ms:
non-transactional: 588.33 RPCs (11767 req/sec)
transactional: 487.82 RPCs ( 9756 req/sec)
3000ms:
non-transactional: 40041.63 RPCs (13347 req/sec)
transactional: 39759.07 RPCs (13253 req/sec)
8000ms:
non-transactional: 106832.37 RPCs (13354 req/sec)
transactional: 105922.65 RPCs (13240 req/sec)
Change-Id: I0370dbb289a4e1cfc154205ae92e13da510682b4
Reviewed-on: http://gerrit.cloudera.org:8080/17105
Tested-by: Alexey Serbin <aserbin@cloudera.com>
Reviewed-by: Alexey Serbin <aserbin@cloudera.com>
1 file changed
