commit | 7adaa97d3ddf4e192ae42fea88f6911a7f747343 | [log] [tgz] |
---|---|---|
author | Julia Wang <juliaw@apache.org> | Tue Dec 13 12:05:02 2016 -0800 |
committer | Mariia Mykhailova <mariia@apache.org> | Tue Dec 13 14:39:09 2016 -0800 |
tree | 9785049ab909f16fdc011e5ac7a852fede6d4cf3 | |
parent | 55cde7d623ae2096248fea58e84ac211a04de2ad [diff] |
[REEF-1687] Throw exception if task dispose throws exception after Call() Currently in TaskRuntime, when task Dispose in Close method throws exception, we throw exception. But when task Dispose in main flow (after Call method) throws exception, we log it but don't re-throw. This behavior is inconsistent. It also results in random failures of TestFailMapperTasksOnDispose, because the test depends on which code path is quicker and where the task Dispose is called. When task Dispose throws exception, that means some resources are not released properly. For fault tolerance work, even if the task returns from the Call(), either having really completed the task or having been closed by driver, if we want to reuse the evaluator to resubmit a task on it, we can't afford that. So the correct way is to throw exception to make evaluator fail, same as what we do in Close method. This change: * updates TaskRuntime to throw exception in both code paths. * updates TestFailMapperTasksOnDispose to account for the time-sensitive nature of the test JIRA: [REEF-1687](https://issues.apache.org/jira/browse/REEF-1687) Pull request: This closes #1203
Apache REEF™ (Retainable Evaluator Execution Framework) is a library for developing portable applications for cluster resource managers such as Apache Hadoop YARN or Apache Mesos. For example, Microsoft Azure Stream Analytics is built on REEF and Hadoop.
Detailed information on REEF can be found in the following places:
The developer mailing list is the best way to reach REEF‘s developers when the above aren’t sufficient.
Component | OS | Status |
---|---|---|
REEF Java | Ubuntu | |
REEF.NET | Windows |
Java | .NET | |
---|---|---|
Build & run unit tests | java\BUILD.md | cs\BUILD.md |