only the executor uses the application configuration
2 files changed
tree: 5b514b9b972700340808e3b645cffa247251b5e9
  1. config/
  2. docker/
  3. docs/
  4. gradle/
  5. myriad-commons/
  6. myriad-executor/
  7. myriad-scheduler/
  8. .gitignore
  9. .travis.yml
  10. build.gradle
  11. gradle.properies
  12. gradlew
  13. gradlew.bat
  14. LICENSE
  15. NOTICE
  16. README.md
  17. settings.gradle
  18. setup-yarn-1.sh
  19. setup-yarn-2.sh
  20. shutdown.sh
  21. Vagrantfile
README.md

Myriad

Build Status

Myriad is a mesos framework designed for scaling a YARN cluster on Mesos. Myriad can expand or shrink the resources managed by a YARN cluster in response to events as per configured rules and policies.

The name Myriad means, countless or extremely great number. In context of the project, it allows one to expand overall resources managed by Mesos, even when the cluster under mesos management runs other cluster managers like YARN.

Please note: Myriad is a work in progress, and should not be used in production at this point.

Getting started

Roadmap

Myriad is a work in progress, please keep checking this section for updates.

  • [x] Custom Executor for managing NodeManager
  • [x] Support unique constraint to let only one node-manager run on a slave
  • [x] Framework checkpointing
  • [x] NodeManager Profiles
  • [ ] High Availability mode for framework
  • [ ] Framework re-conciliation
  • [ ] Fine-grained scaling
  • [ ] Support multi-tenancy for node-managers
  • [ ] Configuration store for storing rules and policies for clusters managed by Myriad

Videos and Slides

  • MesosCon 2014 - Running YARN alongside Mesos (video) (slides)
  • Mesos User Group, March 2015 - Myriad: Integrating Hadoop into the Datacenter (video)