Docker is the easiest way to from 0 to Yarn on Mesos within minutes.
YARN runs as a user and group, and expects consistent uid and gids in HDFS and accross the cluster. Hence it is necessary to edit docker/Dockerfile and modify the following:
ENV YARN_USER="yarn" ENV YARN_UID="107" ENV HADOOP_GROUP="hadoop" ENV HADOOP_GID="112" ENV YARN_GROUP="yarn" ENV YARN_UID="113"
The run the following commands:
./gradlew -P dockerTag=username/myriad buildRMDocker docker push username/myriad
This will build the ResourceManager from src, save, and push the image as username/myriad.
It is assumed you already have hdfs and Mesos running. For more information about Apache Mesos visit the website. If you need to setup hdfs, consider using the hdfs-mesos framework.
In order for the ResourceManager to operate correctly, you will need to provide 5 configuration files. These files will need to mounted from a directory into /usr/local/hadoo/etc of the docker container.
Edit and copy the file templates along with the hdfs.xml and core-site.xml from your hdfs configuration into a directory named config. Additional, files maybe necessary such as rack.sh, log4j.properties, hadoop-env.sh, and yarn-env.sh depending on your configuration. Create a second directoy called dist. Then tar the files using tar -zcf dist\config.tgz config.
From the directories containing dist and config execute:
docker run --net=host -v $PWD/dist -v $PWD/config:/usr/local/hadoop/etc/hadoop --name='myriad-resourcemanager' -t \ <username>/myriad
There is no need to to rebuild the docker image to update YARN's configuration, simply edit the files in config and repackage using tar -zcf dist\config.tgz config. To update the configuration on the Nodemanagers they will need to be flexed down and flexed up, to update the resource manager the docker container will need to be restarted:
docker kill myriad-resourcemanager docker run --net=host -v $PWD/dist -v $PWD/config:/usr/local/hadoop/etc/hadoop --name='myriad-resourcemanager' -t \ <username>/myriad