| commit | 78fc2635992ad7fa164c97cf59ddfd8b0cb0aef9 | [log] [tgz] |
|---|---|---|
| author | martin <martin@attivio.com> | Sat Feb 04 11:40:36 2017 -0500 |
| committer | Henry Saputra <hsaputra@apache.org> | Tue Feb 07 10:37:45 2017 -0800 |
| tree | 1c5d89e4a0c07409c821c8c69a559f29851b7512 | |
| parent | db7f8d6438f8a0f3c7128aace24813a9af9ecff4 [diff] |
(TWILL-181) allow setting the maximum number of retries per runnable Patch by @serranom Each runnable can have a configured number of max retries. If not set, then retries are unlimited as before. Retries are scaled according to the number of instances for each Runnable. add withMaxTries(runnableName, int) to TwillPreparer add withMaxTries(runnableName, int) to YarnTwillPreparer. This stores a map from runnableName to maxRetries. this map becomes part of the twillRuntimeSpecification and RuntimeSpecification interface and is added to TwillRuntimeSpecificationCodec ApplicationMasterService.initRunningContainers is updated to pass a map of runnables to maxretries. updated RunningContainers so that it keeps count of the number of retries per runnable and uses this in handleCompleted() to determine if it should retry. Since every instance is the same as any other, if I'm starting 10 instances of a Runnable, and wanted a max retry count of 3, then that would scale the total number of retries to 30. Each instance gets (on average) 3 tries. Since the instances are interchangeable, there is no concept of a discrete instance being retried. updated logging to not have anything special if max wasn't set and to log the number of retries left and when they have been exhausted. This closes #23 from Github Signed-off-by: Henry Saputra <hsaputra@apache.org>
Twill is an abstraction over Apache Hadoop® YARN that reduces the complexity of developing distributed applications, allowing developers to focus more on their business logic. Twill allows you to use YARN’s distributed capabilities with a programming model that is similar to running threads.
You can build and install the Apache Twill by:
git clone https://git-wip-us.apache.org/repos/asf/twill.git cd twill mvn install
After the maven installation completes, you can include the artifact org.apache.twill:twill-yarn as a dependency on your other projects.
This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.
The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.
The following provides more details on the included cryptographic software:
Apache Twill uses the built-in java cryptography libraries for unique ID generation. See http://www.oracle.com/us/products/export/export-regulations-345813.html for more details on Java's cryptography features.