Follow the steps below to upgrade your Accumulo instance and client to 2.0.
IMPORTANT! Before upgrading to Accumulo 2.0, you will need to upgrade to Java 8 and Hadoop 3.x.
Upgrading to Accumulo 2.0 is done by stopping Accumulo 1.8/9 and starting Accumulo 2.0.
Before stopping Accumulo 1.8/9, install Accumulo 2.0 and configure it by following the [2.0 quick start]({% durl getting-started/quickstart %}).
There are several changes to scripts and configuration in 2.0 so be careful when using configuration or automated setup designed for 1.8/9. Below are some changes in 2.0 that you should be aware of:
accumulo.properties
has replaced accumulo-site.xml
. You can either convert accumulo-site.xml
by hand from XML to properties or use the following Accumulo command.accumulo convert-config -x old/accumulo-site.xml -p new/accumulo.properties
accumulo-client.properties
has replaced client.conf
. The [client properties]({% durl configuration/client-properties %}) in the new file are different so take care when customizing.accumulo-cluster
script has replaced the start-all.sh
& stop-all.sh
scripts.masters
, monitor
, gc
) are no longer in conf/
directory of tarball but can be created using accumulo-cluster create-config
tservers
file instead of a slaves
file. To minimize confusion, Accumulo will not start if the old slaves
file is present.accumulo-service
script can be used to start/stop Accumulo services (i.e master, tablet server, monitor) on a single node.accumulo-cluster
script.accumulo-env.sh
constructs environment variables (such as JAVA_OPTS
and CLASSPATH
) used when running Accumulo processes$cmd
, $bin
, $conf
) are set before accumulo-env.sh
is loaded and can be used to customize environment.JAVA_OPTS
variable is constructed in accumulo-env.sh
to pass command-line arguments to the java
command that the starts Accumulo processes (i.e. java $JAVA_OPTS main.class.for.$cmd
).CLASSPATH
variable sets the Java classpath used when running Accumulo processes. It can be modified to upgrade dependencies or use vendor-specific distributions of Hadoop.accumulo-env.sh
for Accumulo processes. The following log4j configuration files in the conf/
directory will be used if accumulo-env.sh
is not modified. These files can be modified to turn on/off logging for Accumulo processes:log4j-service.properties
for all Accumulo services (except monitor)logj4-monitor.properties
for Accumulo monitorlog4j.properties
for Accumulo clients and commandsaccumulo shell
to access the shell using configuration in conf/accumulo-client.properties
When your Accumulo 2.0 installation is properly configured, stop Accumulo 1.8/9 and start Accumulo 2.0:
./accumulo-1.9.3/bin/stop-all.sh ./accumulo-2.0.0/bin/accumulo-cluster start
It is recommended that users test this upgrade on development or test clusters before attempting it on production clusters.
There several client API changes in 2.0. In most cases, new API was introduced and the old API was only deprecated. While it is recommended that users start using the new API, the old API will continue to be supported through 2.x.
Below is a list of client API changes that users are required to make for 2.0:
<dependency> <groupId>org.apache.accumulo</groupId> <artifactId>accumulo-core</artifactId> <version>2.0.0</version> </dependency>
new ClientConfiguration()
.ClientConfiguration.create()
insteadBelow is a list of recommended client API changes:
org.apache.accumulo.hadoop
package of the accumulo-hadoop-mapreduce
jar.org.apache.accumulo.core.client
package of the accumulo-core
has been deprecated and will eventually be removed.Upgrades from 1.7 to 1.8 are possible with little effort as no changes were made at the data layer and RPC changes were made in a backwards-compatible way. The recommended way is to stop Accumulo 1.7, perform the Accumulo upgrade to 1.8, and then start 1.8. Like previous versions, after 1.8 is started on a 1.7 instance, a one-time upgrade will happen by the Master which will prevent a downgrade back to 1.7. Upgrades are still one way. Upgrades from versions prior to 1.7 to 1.8 should follow the below path to 1.7 and then perform the upgrade to 1.8 – direct upgrades to 1.8 for versions other than 1.7 are untested.
Existing configuration files from 1.7 should be compared against the examples provided in 1.8. The 1.7 configuration files should all function with 1.8 code, but you will likely want to include changes found in the 1.8.0 release notes and these release notes for 1.8.1.
For upgrades from prior to 1.7, follow the upgrade instructions to 1.7 first.
The recommended way to upgrade from a prior 1.7.x release is to stop Accumulo, upgrade to 1.7.y and then start 1.7.y.
When upgrading, there is a known issue if the upgrade fails due to outstanding FATE operations, see ACCUMULO-4496 The work around if this situation is encountered:
fate print
to list allfate delete
If any of the FATE operations are not complete, you should rollback the upgrade and troubleshoot completing them with your prior version. When performing an upgrade between major versions, the upgrade is one-way, therefore it is important that you do not have any outstanding FATE operations before starting the upgrade.
Upgrades from 1.6 to 1.7 are possible with little effort as no changes were made at the data layer and RPC changes were made in a backwards-compatible way. The recommended way is to stop Accumulo 1.6, perform the Accumulo upgrade to 1.7, and then start 1.7. Like previous versions, after 1.7.0 is started on a 1.6 instance, a one-time upgrade will happen by the Master which will prevent a downgrade back to 1.6. Upgrades are still one way. Upgrades from versions prior to 1.6 to 1.7 should follow the below path to 1.6 and then perform the upgrade to 1.7 – direct upgrades to 1.7 for versions other than 1.6 are untested.
After upgrading to 1.7.0, users will notice the addition of a replication table in the accumulo namespace. This table is created and put offline to avoid any additional maintenance if the data-center replication feature is not in use.
Existing configuration files from 1.6 should be compared against the examples provided in 1.7. The 1.6 configuration files should all function with 1.7 code, but you will likely want to include a new file (hadoop-metrics2-accumulo.properties) to enable the new metrics subsystem. Read the section on Hadoop Metrics2 in the Administration chapter of the Accumulo User Manual.
For each of the other new features, new configuration properties exist to support the feature. Refer to the added sections in the User Manual for the feature for information on how to properly configure and use the new functionality.
This happens automatically the first time Accumulo 1.6 is started.
If your instance previously upgraded from 1.4 to 1.5, you must verify that your 1.5 instance has no outstanding local write ahead logs. You can do this by ensuring either:
To upgrade from 1.5 to 1.6 you must:
$ACCUMULO_HOME/bin/accumulo org.apache.accumulo.server.fate.Admin print
The upgrade process must make changes to Accumulo‘s internal state in both ZooKeeper and the table metadata. This process may take some time if Tablet Servers have to go through recovery. During this time, the Monitor will claim that the Master is down and some services may send the Monitor log messages about failure to communicate with each other. These messages are safe to ignore. If you need detail on the upgrade’s progress you should view the local logs on the Tablet Servers and active Master.
To upgrade from 1.4 to 1.6 you must perform a manual initial step.
Prior to upgrading you must:
$ACCUMULO_HOME/bin/accumulo org.apache.accumulo.server.fate.Admin print
Prior to starting the 1.6 instance you will need to run the LocalWALRecovery tool on each node that previously ran an instance of the Logger role.
$ACCUMULO_HOME/bin/accumulo org.apache.accumulo.tserver.log.LocalWALRecovery
The recovery tool will rewrite the 1.4 write ahead logs into a format that 1.6 can read. After this step has completed on all nodes, start the 1.6 cluster to continue the upgrade.
The upgrade process must make changes to Accumulo‘s internal state in both ZooKeeper and the table metadata. This process may take some time if Tablet Servers have to go through recovery. During this time, the Monitor will claim that the Master is down and some services may send the Monitor log messages about failure to communicate with each other. While the upgrade is in progress, the Garbage Collector may complain about invalid paths. The Master may also complain about failure to create the trace table because it already exists. These messages are safe to ignore. If other error messages occur, you should seek out support before continuing to use Accumulo. If you need detail on the upgrade’s progress you should view the local logs on the Tablet Servers and active Master.
Note that the LocalWALRecovery tool does not delete the local files. Once you confirm that 1.6 is successfully running, you should delete these files on the local filesystem.
This happens automatically the first time Accumulo 1.5 is started.
[ZooKeeeperInstance]: {% jurl org.apache.accumulo.core.client.ZooKeeperInstance %} [Connector]: {% jurl org.apache.accumulo.core.client.Connector %} [Instance]: {% jurl org.apache.accumulo.core.client.Instance %} [ClientConfiguration]: {% jurl org.apache.accumulo.core.client.ClientConfiguration %} [AccumuloClient]: {% jurl org.apache.accumulo.core.client.AccumuloClient %} [Connector.from()]: {% jurl org.apache.accumulo.core.client.Connector#from-org.apache.accumulo.core.client.AccumuloClient- %}