This document shows you how to configure a complete cluster using a virtual machine. This setup replicates a real cluster in your development machine as closely as possible. After you complete the steps outlined here, you will be ready to create and run your first Aurora job.
The following sections describe these steps in detail:
The Aurora distribution includes a set of scripts that enable you to create a local cluster in your development machine. These scripts use Vagrant and VirtualBox to run and configure a virtual machine. Once the virtual machine is running, the scripts install and initialize Aurora and any required components to create the local cluster.
First, download and install VirtualBox on your development machine.
Then download and install Vagrant. To verify that the installation was successful, open a terminal window and type the
vagrant command. You should see a list of common commands for this tool.
To obtain the Aurora source distribution, clone its Git repository using the following command:
git clone git://git.apache.org/aurora.git
Now change into the
aurora/ directory, which contains the Aurora source code and other scripts and tools:
To start the local cluster, type the following command:
This command uses the configuration scripts in the Aurora distribution to:
This process takes several minutes to complete.
You may notice a warning that guest additions in the VM don't match your version of VirtualBox. This should generally be harmless, but you may wish to install a vagrant plugin to take care of mismatches like this for you:
vagrant plugin install vagrant-vbguest
With this plugin installed, whenever you
vagrant up the plugin will upgrade the guest additions for you when a version mis-match is detected. You can read more about the plugin here.
To verify that Aurora is running on the cluster, visit the following URLs:
To SSH into the VM, run the following command in your development machine:
To verify that Aurora is installed in the VM, type the
aurora command. You should see a list of arguments and possible commands.
/vagrant directory on the VM is mapped to the
aurora/ local directory from which you started the cluster. You can edit files inside this directory in your development machine and access them from the VM under
clusters.json file refers to your local cluster as
devcluster, which you will use in client commands.
Now that your cluster is up and running, you are ready to define and run your first job in Aurora. For more information, see the Aurora Tutorial.
If you are changing Aurora code and would like to rebuild a component, you can use the
aurorabuild command on the VM to build and restart a component. This is considerably faster than destroying and rebuilding your VM.
aurorabuild accepts a list of components to build and update. To get a list of supported components, invoke the
aurorabuild command with no arguments:
vagrant ssh -c 'aurorabuild client'
To shut down your local cluster, run the
vagrant halt command in your development machine. To start it again, run the
vagrant up command.
Once you are finished with your local cluster, or if you would otherwise like to start from scratch, you can use the command
vagrant destroy to turn off and delete the virtual file system.
Most of the Vagrant related problems can be fixed by the following steps:
VBoxManagecommand line tool
git clean -fdx
If that still doesn't solve your problem, make sure to inspect the log files:
sudo journalctl -u aurora-scheduler
sudo journalctl -u thermos-observer