| commit | 6c597f140184471969ce8bac4c96fc41ca8ea36d | [log] [tgz] |
|---|---|---|
| author | Julia Wang <jwang98052@yahoo.com> | Tue Dec 13 17:33:37 2016 -0800 |
| committer | Mariia Mykhailova <mariia@apache.org> | Fri Dec 16 11:36:56 2016 -0800 |
| tree | 3d0be75077d82c19c744a02e26014ed8ede705f6 | |
| parent | 7adaa97d3ddf4e192ae42fea88f6911a7f747343 [diff] |
[REEF-1511] Add timeout for Task shutdown during IMRU recovery During IMRU FT recovery sometimes the tasks that are supposed to be closed by driver don't report back, causing the system hang. This change adds a timeout for tasks closed by driver, so that evaluators of unresponsive tasks are shut down after timeout. The average task closing time is recorded in the TaskManager. This number is a reference to define the timeout on the fly. In case the average number is accessed before the data is accumulated, or the average number is too low in some scenarios, we have a configurable MinTaskWaitingForCloseTimeout to ensure that the driver waits long enough before killing the evaluators. JIRA: [REEF-1511](https://issues.apache.org/jira/browse/REEF-1511) Pull request: This closes #1201
Apache REEF™ (Retainable Evaluator Execution Framework) is a library for developing portable applications for cluster resource managers such as Apache Hadoop YARN or Apache Mesos. For example, Microsoft Azure Stream Analytics is built on REEF and Hadoop.
Detailed information on REEF can be found in the following places:
The developer mailing list is the best way to reach REEF‘s developers when the above aren’t sufficient.
| Component | OS | Status |
|---|---|---|
| REEF Java | Ubuntu |  |
| REEF.NET | Windows |
| Java | .NET | |
|---|---|---|
| Build & run unit tests | java\BUILD.md | cs\BUILD.md |
