tree 0028138292492e9862a36b86eff28d463e697780
parent 9f2e3253319d7d9733cf764a63e308b91787538a
author hahao <hao.hao@cloudera.com> 1504426479 -0700
committer Adar Dembo <adar@cloudera.com> 1504588367 +0000

KUDU-2131: switch to LIFO log container retrieval

Currently, LBM selects available log container from available container
queue using FIFO, given the intention to spread the load across all
available containers. However, this may cause inefficiency especially
in the case of tablet copy if there is a long available containers
queue. One failed test case is copying tablet of size 26.81GB on a
tablet server with 14 disks and 19199 containers (only 97 of them are
full). In this case, there are too many containers involved, so that
DownloadBlocks took longer that 3m, the default idle timeout for a
tablet copy session, to commit the new blocks such that the session is
terminated before DownloadWALs.

This patch switches to LIFO for log container selection, so that
finalized blocks will exhaust one container at a time. Thus, when
batching disk synchronization for a groups of created blocks, the
number of fsync() operations will be significant lower for tablet
servers with plenty of available log containers.

It also marks '--tablet_copy_idle_timeout_ms', the flag that controls
the amount of time without activity before a tablet copy session
expires, as a user facing flag. And renames it to
'--tablet_copy_idle_timeout_sec'.

Change-Id: I6eb0bc99422c293417b64e0b6e86c87a3d55e687
Reviewed-on: http://gerrit.cloudera.org:8080/7956
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo <adar@cloudera.com>