To run integration tests, you have to specify the druid cluster the tests should use.
Druid comes with the mvn profile integration-tests for setting up druid running in docker containers, and using that cluster to run the integration tests.
To use a druid cluster that is already running, use the mvn profile int-tests-config-file, which uses a configuration file describing the cluster.
Please refer to instructions at https://github.com/druid-io/docker-druid/blob/master/docker-install.md.
Create a new VM for integration tests with at least 6GB of memory.
docker-machine create --driver virtualbox --virtualbox-memory 6000 integration
Set the docker environment:
eval "$(docker-machine env integration)" export DOCKER_IP=$(docker-machine ip integration)
To run all the tests using docker and mvn run the following command:
mvn verify -P integration-tests
To run only a single test using mvn run the following command:
mvn verify -P integration-tests -Dit.test=<test_name>
Make sure that you have at least 6GB of memory available before you run the tests.
To run tests on any druid cluster that is already running, create a configuration file:
{ "broker_host": "<broker_ip>", "broker_port": "<broker_port>", "router_host": "<router_ip>", "router_port": "<router_port>", "indexer_host": "<indexer_ip>", "indexer_port": "<indexer_port>", "coordinator_host": "<coordinator_ip>", "coordinator_port": "<coordinator_port>", "middlemanager_host": "<middle_manager_ip>", "zookeeper_hosts": "<comma-separated list of zookeeper_ip:zookeeper_port>", }
Set the environment variable CONFIG_FILE to the name of the configuration file:
export CONFIG_FILE=<config file name>
To run all the tests using mvn run the following command:
mvn verify -P int-tests-config-file
To run only a single test using mvn run the following command:
mvn verify -P int-tests-config-file -Dit.test=<test_name>
The integration test that indexes from hadoop is not run as part of the integration test run discussed above. This is because druid test clusters might not, in general, have access to hadoop. That's the case (for now, at least) when using the docker cluster set up by the integration-tests profile, so the hadoop test has to be run using a cluster specified in a configuration file.
The data file is integration-tests/src/test/resources/hadoop/batch_hadoop.data. Create a directory called batchHadoop1 in the hadoop file system (anywhere you want) and put batch_hadoop.data into that directory (as its only file).
Add this keyword to the configuration file (see above):
"hadoopTestDir": "<name_of_dir_containing_batchHadoop1>"
Run the test using mvn:
mvn verify -P int-tests-config-file -Dit.test=ITHadoopIndexTest
In some test environments, the machine where the tests need to be executed cannot access the outside internet, so mvn cannot be run. In that case, do the following instead of running the tests using mvn:
On a machine that can do mvn builds:
cd druid mvn clean package cd integration_tests mvn dependency:copy-dependencies package
Copy the integration-tests directory to the test cluster.
TDIR=<directory containing integration-tests>/target VER=<version of druid you built> export CLASSPATH=$TDIR/dependency/*:$TDIR/druid-integration-tests-$VER.jar:$TDIR/druid-integration-tests-$VER-tests.jar
java -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Ddruid.test.config.type=configFile -Ddruid.test.config.configFile=<pathname of configuration file> org.testng.TestNG -testrunfactory org.testng.DruidTestRunnerFactory -testclass io.druid.tests.hadoop.ITHadoopIndexTest
For every end-user functionality provided by druid we should have an integration-test verifying the correctness.
A test can access different helper and utility classes provided by test-framework in order to access Coordinator,Broker etc.. To mark a test be able to use Guice Dependency Injection - Annotate the test class with the below annotation
@Guice(moduleFactory = DruidTestModuleFactory.class)
This will tell the test framework that the test class needs to be constructed using guice.
Refer ITIndexerTest as an example on how to use dependency Injection