If you have questions about upgrades (or need help), please feel free to reach out to us by [mailing list]({{ site.baseurl }}community/mailing-lists) or [Slack Channel]({{ site.baseurl }}community/slack).
Consider the below guidelines in preparation for upgrading.
It is wise to canary an upgraded version in one or small set of bookies before upgrading all bookies in your live cluster.
You can follow below steps on how to canary a upgraded version:
ReadOnly
mode. This can be used to verify if the Bookie of this new version can run well for read workload.ReadOnly
mode successfully for a while, restart the Bookie in Write/Read
mode.If problems occur during canarying an upgraded version, you can simply take down the problematic Bookie node. The remain bookies in the old cluster will repair this problematic bookie node by autorecovery. Nothing needs to be worried about.
Once you determined a version is safe to upgrade in a few nodes in your cluster, you can perform following steps to upgrade all bookies in your cluster.
BookKeeper client
against the new bookkeeper libraries and deploy the new versions.In a rolling upgrade scenario, upgrade one Bookie at a time. In a downtime upgrade scenario, take the entire cluster down, upgrade each Bookie, then start the cluster.
For each Bookie:
We describes the general upgrade method in Apache BookKeeper as above. We will cover the details for individual versions.
There isn't any protocol related backward compabilities changes in 4.7.0. So you can follow the general upgrade sequence to upgrade from 4.6.x to 4.7.0.
However, we list a list of changes that you might want to know.
This section documents the common configuration changes that applied for both clients and servers.
Following settings are newly added in 4.7.0.
Name | Default Value | Description |
---|---|---|
allowShadedLedgerManagerFactoryClass | false | The allows bookkeeper client to connect to a bookkeeper cluster using a shaded ledger manager factory |
shadedLedgerManagerFactoryClassPrefix | dlshade. | The shaded ledger manager factory prefix. This is used when allowShadedLedgerManagerFactoryClass is set to true |
metadataServiceUri | null | metadata service uri that bookkeeper is used for loading corresponding metadata driver and resolving its metadata service location |
permittedStartupUsers | null | The list of users are permitted to run the bookie process. Any users can run the bookie process if it is not set |
There are no common settings deprecated at 4.7.0.
There are no common settings whose default value are changed at 4.7.0.
Following settings are newly added in 4.7.0.
Name | Default Value | Description |
---|---|---|
verifyMetadataOnGC | false | Whether the bookie is configured to double check the ledgers' metadata prior to garbage collecting them |
auditorLedgerVerificationPercentage | 0 | The percentage of a ledger (fragment)'s entries will be verified by Auditor before claiming a ledger (fragment) is missing |
numHighPriorityWorkerThreads | 8 | The number of threads that should be used for high priority requests (i.e. recovery reads and adds, and fencing). If zero, reads are handled by Netty threads directly. |
useShortHostName | false | Whether the bookie should use short hostname or FQDN hostname for registration and ledger metadata when useHostNameAsBookieID is enabled. |
minUsableSizeForEntryLogCreation | 1.2 * logSizeLimit | Minimum safe usable size to be available in ledger directory for bookie to create entry log files (in bytes). |
minUsableSizeForHighPriorityWrites | 1.2 * logSizeLimit | Minimum safe usable size to be available in ledger directory for bookie to accept high priority writes even it is in readonly mode. |
Following settings are deprecated since 4.7.0.
Name | Description |
---|---|
registrationManagerClass | The registration manager class used by server to discover registration manager. It is replaced by metadataServiceUri . |
The default values of following settings are changed since 4.7.0.
Name | Old Default Value | New Default Value | Notes |
---|---|---|---|
numLongPollWorkerThreads | 10 | 0 | If the number of threads is zero or negative, bookie can fallback to use read threads for long poll. This allows not creating threads if application doesn't use long poll feature. |
Following settings are newly added in 4.7.0.
Name | Default Value | Description |
---|---|---|
maxNumEnsembleChanges | Integer.MAX_VALUE | The max allowed ensemble change number before sealing a ledger on failures |
timeoutMonitorIntervalSec | min(addEntryTimeoutSec , addEntryQuorumTimeoutSec , readEntryTimeoutSec ) | The interval between successive executions of the operation timeout monitor, in seconds |
ensemblePlacementPolicyOrderSlowBookies | false | Flag to enable/disable reordering slow bookies in placement policy |
Following settings are deprecated since 4.7.0.
Name | Description |
---|---|
clientKeyStoreType | Replaced by tlsKeyStoreType |
clientKeyStore | Replaced by tlsKeyStore |
clientKeyStorePasswordPath | Replaced by tlsKeyStorePasswordPath |
clientTrustStoreType | Replaced by tlsTrustStoreType |
clientTrustStore | Replaced by tlsTrustStore |
clientTrustStorePasswordPath | Replaced by tlsTrustStorePasswordPath |
registrationClientClass | The registration client class used by client to discover registration service. It is replaced by metadataServiceUri . |
The default values of following settings are changed since 4.7.0.
Name | Old Default Value | New Default Value | Notes |
---|---|---|---|
enableDigestTypeAutodetection | false | true | Autodetect the digest type and passwd when opening a ledger. It will ignore the provided digest type, but still verify the provided passwd. |
In 4.8.x a new feature is added to persist explicitLac in FileInfo and explicitLac entry in Journal. (Note: Currently this feature is not available if your ledgerStorageClass is DbLedgerStorage, ISSUE #1533 is going to address it) Hence current journal format version is bumped to 6 and current FileInfo header version is bumped to 1. But since default config values of ‘journalFormatVersionToWrite’ and ‘fileInfoFormatVersionToWrite’ are set to older versions, this feature is off by default. To enable this feature those config values should be set to current versions. Once this is enabled then we cannot rollback to previous Bookie versions (4.7.x and older), since older version code would not be able to deal with explicitLac entry in Journal file while replaying journal and also reading Header of Index files / FileInfo would fail reading Index files with newer FileInfo version. So in summary, it is a non-rollbackable feature and it applies even if explicitLac is not being used.
There isn't any protocol related backward compabilities changes in 4.6.x. So you can follow the general upgrade sequence to upgrade from 4.5.x to 4.6.x.
There isn't any protocol related backward compabilities changes in 4.5.0. So you can follow the general upgrade sequence to upgrade from 4.4.x to 4.5.x. However, we list a list of things that you might want to know.
multi journals
is a non-rollbackable feature. If you configure a bookie to use multiple journals on 4.5.x you can not roll the bookie back to use 4.4.x. You have to take a bookie out and recover it if you want to rollback to 4.4.x.If you are planning to upgrade a non-secured cluster to a secured cluster enabling security features in 4.5.0, please read BookKeeper Security for more details.