commit 407cb35e594e2334ae687f7c884651c7642349fa
author Charan Reddy Guttapalem <reddycharan18@gmail.com> Fri Apr 19 10:11:08 2019 -0700
committer GitHub <noreply@github.com> Fri Apr 19 10:11:08 2019 -0700
tree c3cedc73460eb2bce1300b1c4d4278bfb5ebb830
parent eaa6014049e6bc8253f5b0282e4ace8a3aff92e2

ISSUE #1967: make ledger creation and removal robust to zk connectionloss


Descriptions of the changes in this PR:

The bookkeeper project ZooKeeperClient wrapper for the ZooKeeper client
will resend zk node creations and removals upon reconnect after a
ConnectionLoss event. In the event that the original succeeded, the
resent operation will erroneously return LedgerExistException or
NoSuchLedgerExistsException for creation and removal respectively.

For removal, this patch limits the operation by allowing it to always
succeed if the ledger does not exist in order to make it idempotent.
This is appears to be the simplest solution as exclusive removal isn't
important.

**Note, the above is an actual change to the bk client semantics**

For creation, exclusive creation is cleary important for correctness,
so this patch adds a creator token field to the LedgerMetdata to
disambiguate the above race from a real race. For
AbstractZkLedgerManager, this is simply a random long value.

There's an oportunity for optimization with the above if exclusive
ledger creation failures are expected to be common.  You only actually
need to perform this check if the operation was really resent.  I chose
not to go this route yet because it would require messing with the
ZooKeeperClient interface to surface that information without burdening
other callers.

If the client is set to version 2 or older, this field will be ignored
and the old behavior will be retained.  If the client is version 3 or
newer but creation races with an older client, the new client will
interpret the nonce to be BLANK and thereby detect the race correctly.



Reviewers: Enrico Olivelli <eolivelli@gmail.com>, Sijie Guo <sijie@apache.org>

This closes #2006 from reddycharan/zkretrialrobust, closes #1967