tree 47e492cef49edcb4dc54cf3185dff1c14a9fce8b
parent 8b2bd18c6624301ad45ce361c90bfd29691afea7
author Alexey Serbin <alexey@apache.org> 1611423455 -0800
committer Alexey Serbin <aserbin@cloudera.com> 1614726454 +0000

KUDU-2612 tablet servers automatically register txn participants

With this patch, tablet servers automatically register tablets as
transaction participants and issue appropriate BEGIN_TXN operations
upon receiving write operations targeting tablet replicas which they
host.

Internally, the newly introduced logic is mostly embedded into the
TxnOpDispatcher class.  Every TxnOpDispatcher object is allowed to
accumulate up to a certain number of pending write requests in its queue
while awaiting for the completion of the preliminary tasks mentioned
above.  Once the TxnOpDispatcher's queue is at capacity, a tablet server
starts responding with the ErrorStatusPB::ERROR_SERVER_TOO_BUSY error
code to incoming write requests in the context of the corresponding
transaction, and Kudu clients automatically retry requests, as expected.
The capacity of the TxnOpDispatcher's queue is controlled by a newly
introduced --tablet_max_pending_txn_write_ops flag.  By default, the
flag is set to 2.  Buffering a few write requests should help to avoid
needless retries in case if a client packs many operations into one
write request: that's exactly the behavior for Kudu client sessions
using the AUTO_FLUSH_BACKGROUND mode.  If such buffering isn't desired
for some reason, set --tablet_max_pending_txn_write_ops=0: in that case
a client will retry the very first operation sent to a tablet server
in the context of a transaction until the tablet server completes the
preliminary tasks mentioned above.  The flag has runtime semantics,
so no restart of a tablet server is required upon modification of the
flag's setting.

Each tablet replica maintains a txn_id --> TxnOpDispatcher map,
with an entry's lifecycle as below:
  * An entry is added upon receiving write request in the context
    of a multi-row transaction.
  * An entry is removed upon applying either ParticipantOpPB::ABORT_TXN
    or ParticipantOpPB::FINALIZE_COMMIT operation.
  * If a write request is received after transaction has been committed
    or aborted, the entry is automatically removed once receiving
    corresponding error response from any of the following components:
      ** from TxnStatusManager in at attempts to register a participant
         in the context of committed/aborted transaction
      ** from the replica itself in an attempt to add
         ParticipantOpPB::BEGIN_COMMIT operation
In other words, the system automatically gets rid of no-longer-needed
TxnOpDispatcher entries.

This patch also contains several test scenarios to cover the newly
introduced functionality.  I also updated other related tests to remove
the artificial registration of corresponding transaction participants,
so those tests now rely on the newly introduced functionality.

Change-Id: Ia383f7afd208c44695c57aab82e3818fa1712ce6
Reviewed-on: http://gerrit.cloudera.org:8080/17037
Tested-by: Kudu Jenkins
Reviewed-by: Andrew Wong <awong@cloudera.com>