pip/pip-118.md - pulsar - Git at Google

 # PIP-118: Do not restart brokers when ZooKeeper session expires

 After all the work done for PIP-45 that was already included in 2.8 and 2.9
 releases, it enabled the concept of re-acquirable resource locks and leader
 election.

 Another important change was to avoid doing any deferrable metadata operation
 when we know that we are not currently connected to the metadata service.

 Finally, that enabled to stabilize in 2.9 the configuration setting that allows
 brokers to continue operating in a safe mode when the session with ZooKeeper
 expires.

 The way it works is that, when we lose a ZooKeeper session, the data plane will
 continue to work undisturbed, relying on the BookKeeper fencing to avoid any
 inconsistencies.

 New topics are not able to get started, but existing topics will see no
 impact.

 The original intention for shutting down the brokers was to ensure that we
 would automatically go back to a consistent state, with respect to which
 resources are "owned" in ZooKeeper by a given broker.

 With the re-acquirable resource locks, that problem was solved and thoroughly
 tested to be robust.

 ## Proposed changes

 In 2.10 release, for the setting:

 ```properties
 # There are two policies to apply when broker metadata session expires: session expired happens, "shutdown" or "reconnect".
 # With "shutdown", the broker will be restarted.
 # With "reconnect", the broker will keep serving the topics, while attempting to recreate a new session.
 zookeeperSessionExpiredPolicy=shutdown
 ```

 Deprecate the old setting and rename it to `metadataSessionExpiredPolicy`, with default value set to `reconnect`.
	# PIP-118: Do not restart brokers when ZooKeeper session expires

	After all the work done for PIP-45 that was already included in 2.8 and 2.9
	releases, it enabled the concept of re-acquirable resource locks and leader
	election.

	Another important change was to avoid doing any deferrable metadata operation
	when we know that we are not currently connected to the metadata service.

	Finally, that enabled to stabilize in 2.9 the configuration setting that allows
	brokers to continue operating in a safe mode when the session with ZooKeeper
	expires.

	The way it works is that, when we lose a ZooKeeper session, the data plane will
	continue to work undisturbed, relying on the BookKeeper fencing to avoid any
	inconsistencies.

	New topics are not able to get started, but existing topics will see no
	impact.

	The original intention for shutting down the brokers was to ensure that we
	would automatically go back to a consistent state, with respect to which
	resources are "owned" in ZooKeeper by a given broker.

	With the re-acquirable resource locks, that problem was solved and thoroughly
	tested to be robust.

	## Proposed changes

	In 2.10 release, for the setting:

	```properties
	# There are two policies to apply when broker metadata session expires: session expired happens, "shutdown" or "reconnect".
	# With "shutdown", the broker will be restarted.
	# With "reconnect", the broker will keep serving the topics, while attempting to recreate a new session.
	zookeeperSessionExpiredPolicy=shutdown
	```

	Deprecate the old setting and rename it to `metadataSessionExpiredPolicy`, with default value set to `reconnect`.