Distcc will speed up compilation by distributing compilation tasks to remote build machines. The scripts in this folder make using distcc easier.
The only requirement you should need to be aware of is, the scripts in this folder were only tested on Linux. If you are using OS X, things probably won't work out of the box.
Assuming you are using Linux, if you use the scripts in this folder, there shouldn't be any other requirements other than setting up your build farm and your BUILD_FARM variable.
Setting up a new distcc server is covered at the bottom of this document. Once your distcc servers are configured, set the environment variable BUILD_FARM on your build machine to to “host1/limit1,lzo host2/limit2,lzo” and so on.
The rest of the setup is done for you; here is a short description of what they do:
You shouldn't need to do any of this, this scripts do this for you.
Symlink /opt/Impala-Toolchain to a directory writable by your user and point IMPALA_TOOLCHAIN at that directory. This ensures toolchain binaries are at the same path locally as on the distcc servers
mkdir -p "$IMPALA_TOOLCHAIN" sudo ln -s "$IMPALA_TOOLCHAIN" /opt/Impala-Toolchain echo 'export IMPALA_TOOLCHAIN=/opt/Impala-Toolchain' >> bin/impala-config-local.sh
Source bin/impala-config.sh in the Impala repo. Step #2 depends on this.
Source “distcc_env.sh” in this directory. The script will attempt to install distcc if needed.
Run buildall.sh. The main purpose is to regenerate cmakefiles.
cd "$IMPALA_HOME" ./buildall.sh -skiptests -so # Do not use -noclean
You should notice that the build runs quite a bit faster.
At this point you no longer need to run the heavyweight buildall.sh. After editing files you can either
make -j$(distcc -j)
If you want to compile a very small change, a local build might be faster.
to switch back
If you open a new terminal and attempt to build with “make” that will fail. To fix:
source "$IMPALA_HOME"/bin/impala-config.sh # Skip if already done source "$IMPALA_HOME"/bin/distcc/distcc_env.sh