fixed a bug at WriteLock caused by read-delete race on a znode.
Bug description:
T1 currently owns a zk lock as signified by znode n1, T2 creates a znode n2
and realizes n1 is saller. T2 is going to register a watcher on n1 but at the
same moment T1 released n1. T2 register fails, breaks from while loop, and wait().
Nobody will ever wake up T2 again. Consequently all subsequent callers for the
same lock are also blocked.
Test:
Repeated our loadtest and the bug doesn't reappear.
For detailed bug report see this post:
http://mail-archives.apache.org/mod_mbox/helix-dev/201605.mbox/%3CCAB-bdySG8Uf6c1fyVHpSu-5pD99VHE=mrL=j3QNkaTWaEtKQ+w@mail.gmail.com%3E
diff --git a/helix-core/src/main/java/org/apache/helix/lock/zk/WriteLock.java b/helix-core/src/main/java/org/apache/helix/lock/zk/WriteLock.java
index aef7618..b842ff8 100644
--- a/helix-core/src/main/java/org/apache/helix/lock/zk/WriteLock.java
+++ b/helix-core/src/main/java/org/apache/helix/lock/zk/WriteLock.java
@@ -179,7 +179,7 @@
List<String> names = zookeeper.getChildren(dir, false);
for (String name : names) {
if (name.startsWith(prefix)) {
- id = name;
+ id = dir + "/" + name;
if (LOG.isDebugEnabled()) {
LOG.debug("Found id created last time: " + id);
}
@@ -230,14 +230,15 @@
ZNodeName lastChildName = lessThanMe.last();
lastChildId = lastChildName.getName();
if (LOG.isDebugEnabled()) {
- LOG.debug("watching less than me node: " + lastChildId);
+ LOG.debug("watching less than me node: " + lastChildId + ", my id: " + idName.getName());
}
Stat stat = zookeeper.exists(lastChildId, new LockWatcher());
if (stat != null) {
return Boolean.FALSE;
} else {
LOG.warn("Could not find the" + " stats for less than me: "
- + lastChildName.getName());
+ + lastChildName.getName() + ", will retry");
+ id = null;
}
} else {
if (isOwner()) {
