| commit | 18c25fdcfabb7ba1c56ea374d0d7f6777d99d7fd | [log] [tgz] |
|---|---|---|
| author | Julia Wang <jwang98052@yahoo.com> | Fri Oct 14 14:26:36 2016 -0700 |
| committer | Mariia Mykhailova <mariia@apache.org> | Mon Oct 17 11:21:41 2016 -0700 |
| tree | 1dafa62575ad97dbbe868f7f50a7924e945c5328 | |
| parent | 08f096a2778a731c4de40c13edb1e7141f39a02b [diff] |
[REEF-1641] Remove batch id and change the way to request maser/slave evalautors Currently in IMRU, we have different spec for master vs mapper elevators. We use EvaluatorBatchId to distinguish which one we requested when receiving IAllocatedEvaluator. However, EvaluatorBatchId may not be supported in some environments such as HDInsight. So we need a more reliable way to decide which evaluator is for master. The change in this PR is to request the first evaluator for master. After receiving it, record it as master, then request the mappers. An assumption is that if master fails at any stage, we fail the system. * IMRU Driver and Evaluator Manager are updated * Existing test cases are updated * A new test case is added to simulate master evaluator failure during WaitingForEvaluators phase. JIRA: [REEF-1641](https://issues.apache.org/jira/browse/REEF-1641) Pull request: This closes #1156
Apache REEF™ (Retainable Evaluator Execution Framework) is a library for developing portable applications for cluster resource managers such as Apache Hadoop YARN or Apache Mesos. For example, Microsoft Azure Stream Analytics is built on REEF and Hadoop.
Detailed information on REEF can be found in the following places:
The developer mailing list is the best way to reach REEF‘s developers when the above aren’t sufficient.
| Component | OS | Status |
|---|---|---|
| REEF Java | Ubuntu |  |
| REEF.NET | Windows |
| Java | .NET | |
|---|---|---|
| Build & run unit tests | java\BUILD.md | cs\BUILD.md |
