TESTING - accumulo - Git at Google

 Title: Testing Apache Accumulo
 Notice:    Licensed to the Apache Software Foundation (ASF) under one
            or more contributor license agreements.  See the NOTICE file
            distributed with this work for additional information
            regarding copyright ownership.  The ASF licenses this file
            to you under the Apache License, Version 2.0 (the
            "License"); you may not use this file except in compliance
            with the License.  You may obtain a copy of the License at
            .
              http://www.apache.org/licenses/LICENSE-2.0
            .
            Unless required by applicable law or agreed to in writing,
            software distributed under the License is distributed on an
            "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
            KIND, either express or implied.  See the License for the
            specific language governing permissions and limitations
            under the License.

 # Testing Apache Accumulo

 This document is meant to serve as a quick reference to the automated test suites included in Apache Accumulo for users
 to run which validate the product and developers to continue to iterate upon to ensure that the product is stable and as
 free of bugs as possible.

 The automated testing suite can be categorized as two sets of tests: unit tests and integration tests. These are the
 traditional unit and integrations tests as defined by the Apache Maven lifecycle phases (unit tests run at `test` and
 integration tests run at `integration-test`).

 # Unit tests

 Unit tests can be run by invoking `mvn test` at the top of the Apache Accumulo source tree; however, it is more often
 the case that these tests are automatically run by invoking `mvn package` instead.  Either invocation should work
 successfully.

 The unit tests should run rather quickly (order of minutes for the entire project) and, in nearly all cases, do not
 require any noticable amount of computer resources (the compilation of the files typically exceeds running the tests).
 Maven will automatically generate a report for each unit test run and will give a summary at the end of each Maven
 module for the total run/failed/errored/skipped tests.

 The Apache Accumulo developers expect that these tests are always passing on every revision of the code. If this is not
 the case, it is almost certainly in error.

 # Integration tests

 Integration tests can be run by invoking `mvn integration-test` at the top of the Apache Accumulo source tree; however,
 like `mvn package` being recommended for unit tests, `mvn verify` is often the recommended avenue to run the integration tests.

 The integration tests are medium length tests (order minutes for each test class and order hours for the complete suite
 with single threaded execution) but are very encompassing of checking for regressions that were previously seen in the
 codebase. These tests do require a noticable amount of resources, at least another gigabyte of memory over what Maven
 itself requires. As such, it's recommended to have at least 3-4GB of free memory and 10GB of free disk space.

 Take note that when invoking the `integration-test` lifecycle phase, other functions will also be enabled which include
 static analysis (findbugs) and software license checks (release analysis tool -- RAT).

 ## Accumulo for testing

 The primary reason these tests take so much longer than the unit tests is that most are using an Accumulo instance to
 perform the test. It's a necessary evil; however, there are things we can do to improve this.

 ## MiniAccumuloCluster

 By default, these tests will use a MiniAccumuloCluster which is a multi-process "implementation" of Accumulo, managed
 through Java interfaces. This MiniAccumuloCluster has the ability to use the local filesystem or Apache Hadoop's
 MiniDFSCluster, as well as starting one to many tablet servers. MiniAccumuloCluster tends to be a very useful tool in
 that it can automatically provide a workable instance that mimics how an actual deployment functions.

 The downside of using MiniAccumuloCluster is that a significant portion of each test is now devoted to starting and
 stopping the MiniAccumuloCluster.  While this is a surefire way to isolate tests from interferring with one another, it
 increases the actual runtime of the test by, on average, 10x.

 ## Standalone Cluster

 An alternative to the MiniAccumuloCluster for testing, a standalone Accumulo cluster can also be configured for use by
 most tests. This requires a manual step of building and deploying the Accumulo cluster by hand. The build can then be
 configured to use this cluster instead of always starting a MiniAccumuloCluster.  Not all of the integration tests are
 good candidates to run against a standalone Accumulo cluster, these tests will still launch a MiniAccumuloCluster for
 their use.

 Use of a standalone cluster can be enabled using system properties on the Maven command line or, more concisely, by
 providing a Java properties file on the Maven command line. The use of a properties file is recommended since it is
 typically a fixed file per standalone cluster you want to run the tests against.

 ### Configuration

 The following properties can be used to configure a standalone cluster:

 - `accumulo.it.cluster.type`, Required: The type of cluster is being defined (valid options: MINI and STANDALONE)
 - `accumulo.it.cluster.standalone.principal`, Required: Standalone cluster principal (user)
 - `accumulo.it.cluster.standalone.password`, Required: Password for the principal
 - `accumulo.it.cluster.standalone.zookeepers`, Required: ZooKeeper quorum used by the standalone cluster
 - `accumulo.it.cluster.standalone.instance.name`, Required: Accumulo instance name for the cluster
 - `accumulo.it.cluster.standalone.home`, Optional: `ACCUMULO_HOME`
 - `accumulo.it.cluster.standalone.conf`, Optional: `ACCUMULO_CONF_DIR`
 - `accumulo.it.cluster.standalone.hadoop.conf`, Optional: `HADOOP_CONF_DIR`

 Each of the above properties can be set on the commandline (-Daccumulo.it.cluster.standalone.principal=root), or the
 collection can be placed into a properties file and referenced using "accumulo.it.cluster.properties".  For example, the
 following might be similar to what is executed for a standalone cluster.

   `mvn verify -Daccumulo.it.properties=/home/user/my_cluster.properties`

 For the optional properties, each of them will be extracted from the environment if not explicitly provided.
 Specifically, `ACCUMULO_HOME` and `ACCUMULO_CONF_DIR` are used to ensure the correct version of the bundled
 Accumulo scripts are invoked and, in the event that multiple Accumulo processes exist on the same physical machine,
 but for different instances, the correct version is terminated. `HADOOP_CONF_DIR` is used to ensure that the necessary
 files to construct the FileSystem object for the cluster can be constructed (e.g. core-site.xml and hdfs-site.xml).

 # Manual Distributed Testing

 Apache Accumulo also contains a number of tests which are suitable for running against large clusters for hours to days
 at a time, for example the Continuous Ingest and Randomwalk test suites. These all exist in the repository under
 `test/system` and contain their own README files for configuration and use.
	Title: Testing Apache Accumulo
	Notice: Licensed to the Apache Software Foundation (ASF) under one
	or more contributor license agreements. See the NOTICE file
	distributed with this work for additional information
	regarding copyright ownership. The ASF licenses this file
	to you under the Apache License, Version 2.0 (the
	"License"); you may not use this file except in compliance
	with the License. You may obtain a copy of the License at
	.
	http://www.apache.org/licenses/LICENSE-2.0
	.
	Unless required by applicable law or agreed to in writing,
	software distributed under the License is distributed on an
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	KIND, either express or implied. See the License for the
	specific language governing permissions and limitations
	under the License.

	# Testing Apache Accumulo

	This document is meant to serve as a quick reference to the automated test suites included in Apache Accumulo for users
	to run which validate the product and developers to continue to iterate upon to ensure that the product is stable and as
	free of bugs as possible.

	The automated testing suite can be categorized as two sets of tests: unit tests and integration tests. These are the
	traditional unit and integrations tests as defined by the Apache Maven lifecycle phases (unit tests run at `test` and
	integration tests run at `integration-test`).

	# Unit tests

	Unit tests can be run by invoking `mvn test` at the top of the Apache Accumulo source tree; however, it is more often
	the case that these tests are automatically run by invoking `mvn package` instead. Either invocation should work
	successfully.

	The unit tests should run rather quickly (order of minutes for the entire project) and, in nearly all cases, do not
	require any noticable amount of computer resources (the compilation of the files typically exceeds running the tests).
	Maven will automatically generate a report for each unit test run and will give a summary at the end of each Maven
	module for the total run/failed/errored/skipped tests.

	The Apache Accumulo developers expect that these tests are always passing on every revision of the code. If this is not
	the case, it is almost certainly in error.

	# Integration tests

	Integration tests can be run by invoking `mvn integration-test` at the top of the Apache Accumulo source tree; however,
	like `mvn package` being recommended for unit tests, `mvn verify` is often the recommended avenue to run the integration tests.

	The integration tests are medium length tests (order minutes for each test class and order hours for the complete suite
	with single threaded execution) but are very encompassing of checking for regressions that were previously seen in the
	codebase. These tests do require a noticable amount of resources, at least another gigabyte of memory over what Maven
	itself requires. As such, it's recommended to have at least 3-4GB of free memory and 10GB of free disk space.

	Take note that when invoking the `integration-test` lifecycle phase, other functions will also be enabled which include
	static analysis (findbugs) and software license checks (release analysis tool -- RAT).

	## Accumulo for testing

	The primary reason these tests take so much longer than the unit tests is that most are using an Accumulo instance to
	perform the test. It's a necessary evil; however, there are things we can do to improve this.

	## MiniAccumuloCluster

	By default, these tests will use a MiniAccumuloCluster which is a multi-process "implementation" of Accumulo, managed
	through Java interfaces. This MiniAccumuloCluster has the ability to use the local filesystem or Apache Hadoop's
	MiniDFSCluster, as well as starting one to many tablet servers. MiniAccumuloCluster tends to be a very useful tool in
	that it can automatically provide a workable instance that mimics how an actual deployment functions.

	The downside of using MiniAccumuloCluster is that a significant portion of each test is now devoted to starting and
	stopping the MiniAccumuloCluster. While this is a surefire way to isolate tests from interferring with one another, it
	increases the actual runtime of the test by, on average, 10x.

	## Standalone Cluster

	An alternative to the MiniAccumuloCluster for testing, a standalone Accumulo cluster can also be configured for use by
	most tests. This requires a manual step of building and deploying the Accumulo cluster by hand. The build can then be
	configured to use this cluster instead of always starting a MiniAccumuloCluster. Not all of the integration tests are
	good candidates to run against a standalone Accumulo cluster, these tests will still launch a MiniAccumuloCluster for
	their use.

	Use of a standalone cluster can be enabled using system properties on the Maven command line or, more concisely, by
	providing a Java properties file on the Maven command line. The use of a properties file is recommended since it is
	typically a fixed file per standalone cluster you want to run the tests against.

	### Configuration

	The following properties can be used to configure a standalone cluster:

	- `accumulo.it.cluster.type`, Required: The type of cluster is being defined (valid options: MINI and STANDALONE)
	- `accumulo.it.cluster.standalone.principal`, Required: Standalone cluster principal (user)
	- `accumulo.it.cluster.standalone.password`, Required: Password for the principal
	- `accumulo.it.cluster.standalone.zookeepers`, Required: ZooKeeper quorum used by the standalone cluster
	- `accumulo.it.cluster.standalone.instance.name`, Required: Accumulo instance name for the cluster
	- `accumulo.it.cluster.standalone.home`, Optional: `ACCUMULO_HOME`
	- `accumulo.it.cluster.standalone.conf`, Optional: `ACCUMULO_CONF_DIR`
	- `accumulo.it.cluster.standalone.hadoop.conf`, Optional: `HADOOP_CONF_DIR`

	Each of the above properties can be set on the commandline (-Daccumulo.it.cluster.standalone.principal=root), or the
	collection can be placed into a properties file and referenced using "accumulo.it.cluster.properties". For example, the
	following might be similar to what is executed for a standalone cluster.

	`mvn verify -Daccumulo.it.properties=/home/user/my_cluster.properties`

	For the optional properties, each of them will be extracted from the environment if not explicitly provided.
	Specifically, `ACCUMULO_HOME` and `ACCUMULO_CONF_DIR` are used to ensure the correct version of the bundled
	Accumulo scripts are invoked and, in the event that multiple Accumulo processes exist on the same physical machine,
	but for different instances, the correct version is terminated. `HADOOP_CONF_DIR` is used to ensure that the necessary
	files to construct the FileSystem object for the cluster can be constructed (e.g. core-site.xml and hdfs-site.xml).

	# Manual Distributed Testing

	Apache Accumulo also contains a number of tests which are suitable for running against large clusters for hours to days
	at a time, for example the Continuous Ingest and Randomwalk test suites. These all exist in the repository under
	`test/system` and contain their own README files for configuration and use.