| commit | 765d7f616844ed947549465ce991b9ee71b4c190 | [log] [tgz] |
|---|---|---|
| author | Julia Wang <juliaw@apache.org> | Tue Dec 06 12:50:41 2016 -0800 |
| committer | Mariia Mykhailova <mariia@apache.org> | Mon Dec 12 17:17:15 2016 -0800 |
| tree | 1a642729f63d5e067f20ca59f6fca0ae6fc0af1e | |
| parent | eb66b2e9d473418b8df9415b26809016c526114a [diff] |
[REEF-1685] Complete the Job properly if update/master task is completed from running state In stress testing, we have seen the following scenario: master/update task is successfully completed, and most of mapper tasks are also completed, but then driver receives an IFailedEvaluator event. This would result in system transitioning to ShuttingDown state and doing an unnecessary retry. When the driver receives ICompletedTask from master task which was in running state, that means we have completed the calculation and result has been written to the output. The driver should execute DoneAction to dispose of all the contexts and shut down the system. After that, any FailedEvalutor/FailedTask events received should be ignored. JIRA: [REEF-1685](https://issues.apache.org/jira/browse/REEF-1685) Pull request: This closes #1200
Apache REEF™ (Retainable Evaluator Execution Framework) is a library for developing portable applications for cluster resource managers such as Apache Hadoop YARN or Apache Mesos. For example, Microsoft Azure Stream Analytics is built on REEF and Hadoop.
Detailed information on REEF can be found in the following places:
The developer mailing list is the best way to reach REEF‘s developers when the above aren’t sufficient.
| Component | OS | Status |
|---|---|---|
| REEF Java | Ubuntu |  |
| REEF.NET | Windows |
| Java | .NET | |
|---|---|---|
| Build & run unit tests | java\BUILD.md | cs\BUILD.md |
