A tool to build and run big data pseudo cluster using Docker.
Make sure you have Docker installed. We've tested this using Docker for Mac
Currently supported OS list:
centos-6
debian-8
ubuntu-14.04
Running Hadoop
docker run -ti --privileged -p 50070:50070 bigtop/sandbox:centos-6_hadoop
docker run -ti --privileged -p 8080:8080 bigtop/sandbox:debian-8_spark
docker run -ti --privileged -p 50070:50070 -p 60010:60010 bigtop/sandbox:ubuntu-14.04_hbase
./build.sh -a bigtop -o debian-8 -c hadoop
./build.sh -a bigtop -o debian-8 -c "hadoop, spark"
./build.sh -a bigtop -o debian-8 -c "hadoop, yarn, hbase"
export REPO=http://ci.bigtop.apache.org:8080/job/Bigtop-trunk-repos/BUILD_ENVIRONMENTS=centos-6%2Clabel=docker-slave-06//ws/output ./build.sh -a bigtop -o centos-6 -c "hadoop, spark, ignite"
cp site.yaml.template.centos-6_hadoop site.yaml.template.centos-6_hadoop_ignite vim site.yaml.template.centos-6_hadoop_ignite
... hadoop_cluster_node::cluster_components: [hadoop, yarn, ignite] ...
./build.sh -a bigtop -o centos-6 -f site.yaml.template.centos-6_hadoop_ignite -t my_ignite_stack
Since systemd requires CAP_SYS_ADMIN, currently any OS using systemd can not successfully started up daemons during image build time.
Daemons can be brought up only if --privileged specified using docker run command.