apache /
kudu /
ceca6abc506fc4c1012ced6e8a6564897b3af4ad KUDU-2612 tablet servers automatically register txn participants
With this patch, tablet servers automatically register tablets as
transaction participants and issue appropriate BEGIN_TXN operations
upon receiving write operations targeting tablet replicas which they
host.
Internally, the newly introduced logic is mostly embedded into the
TxnOpDispatcher class. Every TxnOpDispatcher object is allowed to
accumulate up to a certain number of pending write requests in its queue
while awaiting for the completion of the preliminary tasks mentioned
above. Once the TxnOpDispatcher's queue is at capacity, a tablet server
starts responding with the ErrorStatusPB::ERROR_SERVER_TOO_BUSY error
code to incoming write requests in the context of the corresponding
transaction, and Kudu clients automatically retry requests, as expected.
The capacity of the TxnOpDispatcher's queue is controlled by a newly
introduced --tablet_max_pending_txn_write_ops flag. By default, the
flag is set to 2. Buffering a few write requests should help to avoid
needless retries in case if a client packs many operations into one
write request: that's exactly the behavior for Kudu client sessions
using the AUTO_FLUSH_BACKGROUND mode. If such buffering isn't desired
for some reason, set --tablet_max_pending_txn_write_ops=0: in that case
a client will retry the very first operation sent to a tablet server
in the context of a transaction until the tablet server completes the
preliminary tasks mentioned above. The flag has runtime semantics,
so no restart of a tablet server is required upon modification of the
flag's setting.
Each tablet replica maintains a txn_id --> TxnOpDispatcher map,
with an entry's lifecycle as below:
* An entry is added upon receiving write request in the context
of a multi-row transaction.
* An entry is removed upon applying either ParticipantOpPB::ABORT_TXN
or ParticipantOpPB::FINALIZE_COMMIT operation.
* If a write request is received after transaction has been committed
or aborted, the entry is automatically removed once receiving
corresponding error response from any of the following components:
** from TxnStatusManager in at attempts to register a participant
in the context of committed/aborted transaction
** from the replica itself in an attempt to add
ParticipantOpPB::BEGIN_COMMIT operation
In other words, the system automatically gets rid of no-longer-needed
TxnOpDispatcher entries.
This patch also contains several test scenarios to cover the newly
introduced functionality. I also updated other related tests to remove
the artificial registration of corresponding transaction participants,
so those tests now rely on the newly introduced functionality.
Change-Id: Ia383f7afd208c44695c57aab82e3818fa1712ce6
Reviewed-on: http://gerrit.cloudera.org:8080/17037
Tested-by: Kudu Jenkins
Reviewed-by: Andrew Wong <awong@cloudera.com>
12 files changed
