Hadoop YARN Change Log

Release 0.23.11 - UNRELEASED

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

  OPTIMIZATIONS

  BUG FIXES

    YARN-1053. Diagnostic message from ContainerExitEvent is ignored in
    ContainerImpl (Omkar Vinit Joshi via jlowe)

    YARN-1145. Fixed a potential file-handle leak in the web interface for
    displaying aggregated logs. (Rohith Sharma via jlowe)

    YARN-1180. Update capacity scheduler docs to include types on the configs
    (Chen He via jeagles)

    YARN-1592. CapacityScheduler tries to reserve more than a node's total
    memory on branch-0.23 (Omkar Vinit Joshi and Thomas Graves via jlowe)

    YARN-1575. Public localizer crashes with "Localized unkown resource"
    (jlowe)

    YARN-500. Fixed YARN webapps to not roll-over ports when explicitly asked
    to use non-ephemeral ports. (Kenji Kikushima via vinodkv)

    YARN-1670. aggregated log writer can write more log data then it says is
    the log length (Mit Desai via jeagles)

    YARN-1932. Javascript injection on the job status page (Mit Desai via
    jlowe)

    YARN-853. maximum-am-resource-percent doesn't work after refreshQueues
    command (Devaraj K via jlowe)

Release 0.23.10 - 2013-12-09

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    YARN-985. Nodemanager should log where a resource was localized (Ravi
    Prakash via jeagles)

    YARN-707. Add user info in the YARN ClientToken (jlowe)

    YARN-1119. Add ClusterMetrics checks to tho TestRMNodeTransitions tests
    (Mit Desai via jeagles)

  OPTIMIZATIONS

  BUG FIXES

    YARN-949. Failed log aggregation can leave a file open. (Kihwal Lee via
    jlowe)

    YARN-1036. Distributed Cache gives inconsistent result if cache files get
    deleted from task tracker (Mayank Bansal and Ravi Prakash via jlowe)

    YARN-543. Shared data structures in Public Localizer and Private
    Localizer are not Thread safe. (Omkar Vinit Joshi and Mit Desai via jlowe)

    YARN-337. RM handles killed application tracking URL poorly (jlowe)

    YARN-1101. Active nodes can be decremented below 0 (Robert Parker
    via tgraves)

    YARN-1176. RM web services ClusterMetricsInfo total nodes doesn't include
    unhealthy nodes (Jonathan Eagles via tgraves)

    YARN-1243. ResourceManager: Error in handling event type NODE_UPDATE to the
    scheduler - NPE at SchedulerApp.java:411 (Jason Lowe via jeagles)

    YARN-155. TestAppManager intermittently fails with jdk7 (Thomas Graves via
    jlowe)

    YARN-1031. JQuery UI components reference external css in branch-23
    (Jonathan Eagles and Jason Lowe via jlowe)

    YARN-1386. NodeManager mistakenly loses resources and relocalizes them
    (Jason Lowe via jeagles)

    YARN-1419. TestFifoScheduler.testAppAttemptMetrics fails intermittently
    under jdk7 (Jonathan Eagles via jlowe)

Release 0.23.9 - 2013-07-08

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    YARN-427. Coverage fix for org.apache.hadoop.yarn.server.api.* (Aleksey
    Gorshkov via jeagles)

    YARN-478. fix coverage org.apache.hadoop.yarn.webapp.log (Aleksey Gorshkov
    via jeagles)

  OPTIMIZATIONS

  BUG FIXES

    YARN-742. Log aggregation causes a lot of redundant setPermission calls.
    (jlowe via kihwal)

Release 0.23.8 - 2013-06-05
  
  INCOMPATIBLE CHANGES
  
  NEW FEATURES
  
  IMPROVEMENTS

    YARN-548. Add tests for YarnUncaughtExceptionHandler (Vadim Bondarev via
    jeagles)

    YARN-169. Update log4j.appender.EventCounter to use
    org.apache.hadoop.log.metrics.EventCounter (Anthony Rojas via jeagles)

  
  OPTIMIZATIONS
 
  BUG FIXES

    YARN-476. ProcfsBasedProcessTree info message confuses users. 
    (sandyr via tucu)

    YARN-71. Fix the NodeManager to clean up local-dirs on restart.
    (Xuan Gong via sseth)

    YARN-363. Add webapps/proxy directory without which YARN proxy-server fails
    when started in stand-alone mode. (Kenji Kikushima via vinodkv)

    YARN-690. RM exits on token cancel/renew problems (daryn via bobby)

Release 0.23.7 - 2013-04-18

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    YARN-133 Update web services docs for RM clusterMetrics (Ravi Prakash via
    kihwal)

    YARN-286. Add a YARN ApplicationClassLoader. (tomwhite)

    YARN-249. Capacity Scheduler web page should show list of active users per 
    queue like it used to (in 1.x) (Ravi Prakash via tgraves)

    YARN-236. RM should point tracking URL to RM web page when app fails to
    start (Jason Lowe via jeagles)

    YARN-269. Resource Manager not logging the health_check_script result when
    taking it out (Jason Lowe via kihwal)

    YARN-227. Application expiration difficult to debug for end-users
    (Jason Lowe via jeagles)

    YARN-410. Fixed RM UI so that the new lines diagnostics for a failed app on
    the per-application page are translated to html line breaks. (Omkar Vinit
    Joshi via vinodkv)

    YARN-443. allow OS scheduling priority of NM to be different than the 
    containers it launches (tgraves)

    YARN-468. coverage fix for org.apache.hadoop.yarn.server.webproxy.amfilter
    (Aleksey Gorshkov via bobby)

    YARN-200. yarn log does not output all needed information, and is in a
    binary format (Ravi Prakash via jlowe)

    YARN-525. make CS node-locality-delay refreshable (Thomas Graves via jlowe)

  OPTIMIZATIONS

    YARN-357. App submission should not be synchronized (daryn)

  BUG FIXES

    YARN-343. Capacity Scheduler maximum-capacity value -1 is invalid (Xuan
    Gong via tgraves)

    YARN-364. AggregatedLogDeletionService can take too long to delete logs
    (jlowe)

    YARN-29. Add a yarn-client module (Vinod Kumar Vavilapalli via tgraves)

    YARN-83. Change package of YarnClient to include apache (Bikas Saha via
    tgraves)

    YARN-40. Provide support for missing yarn commands (Devaraj K via tgraves)

    YARN-355. Fixes a bug where RM app submission could jam under load.
    (Daryn Sharp via sseth)

    YARN-150. AppRejectedTransition does not unregister app from master service
    and scheduler (Bikas Shah via tgraves)

    YARN-362. Unexpected extra results when using webUI table search (Ravi
    Prakash via jlowe)

    YARN-400. RM can return null application resource usage report leading to 
    NPE in client (Jason Lowe via tgraves)

    YARN-426. Failure to download a public resource prevents further downloads
    (Jason Lowe via bobby)

    YARN-376. Fixes a bug which would prevent the NM knowing about completed
    containers and applications. (Jason Lowe via sseth)

    YARN-448. Remove unnecessary hflush from log aggregation (Kihwal Lee via
    bobby)

    YARN-345. Many InvalidStateTransitonException errors for ApplicationImpl
    in Node Manager (Robert Parker via jlowe)

    YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal 
    via bobby)

    YARN-460. CS user left in list of active users for the queue even when
    application finished (tgraves)

Release 0.23.6 - 2013-02-06

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    YARN-285. Added a temporary plugin interface for RM to be able to redirect to
    JobHistory server for apps that it no longer tracks. Contributed by
    Derek Dagit.

  OPTIMIZATIONS

  BUG FIXES

    YARN-204. test coverage for org.apache.hadoop.tools (Aleksey Gorshkov via
    bobby)

    YARN-251. Proxy URI generation fails for blank tracking URIs (Tom White
    via jlowe)

    YARN-72. NM should handle cleaning up containers when it shuts down.
    (Sandy Ryza via tomwhite)

    YARN-258. RM web page UI shows Invalid Date for start and finish times
    (Ravi Prakash via jlowe)

    YARN-266. RM and JHS Web UIs are blank because AppsBlock is not escaping
    string properly (Ravi Prakash via jlowe)

    YARN-223. Change processTree interface to work better with native code
    (Radim Kolar via tgraves)

    YARN-280. RM does not reject app submission with invalid tokens 
    (Daryn Sharp via tgraves)

    YARN-225. Proxy Link in RM UI thows NPE in Secure mode 
    (Devaraj K via bobby)

    YARN-293. Node Manager leaks LocalizerRunner object for every Container
    (Robert Joseph Evans via jlowe)

    YARN-50. Implement renewal / cancellation of Delegation Tokens 
    (Siddharth Seth via tgraves)

    YARN-320. RM should always be able to renew its own tokens.
    (Daryn Sharp via sseth)

    YARN-325. RM CapacityScheduler can deadlock when getQueueInfo() is 
    called and a container is completing (Arun C Murthy via tgraves)

    YARN-334. Maven RAT plugin is not checking all source files (tgraves)

    YARN-170. NodeManager stop() gets called twice on shutdown (Sandy Ryza
    via tgraves)

    YARN-354. WebAppProxyServer exits immediately after startup (Liang Xie via
    jlowe)

Release 0.23.5 - 2012-11-28

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

    YARN-138. Ensure default values for minimum/maximum container sizes is 
    sane. (harsh & sseth via acmurthy)

    YARN-137. Change the default YARN scheduler to be the CapacityScheduler. 
    (sseth via acmurthy) 

    YARN-116. Add the ability to change the RM include/exclude file without
    a restart. (xieguiming and Harsh J via sseth)

    YARN-28. Fixed TestCompositeService to not depend on test-order and thus
    made it pass on JDK7 (Thomas Graves via vinodkv).

    YARN-161. Fix multiple compiler warnings for unchecked operations in YARN
    common. (Chris Nauroth via vinodkv)

    YARN-43. Fix TestResourceTrackerService to not depend on test order and thus
    pass on JDK7. (Thomas Graves via vinodkv)

    YARN-32. Fix TestApplicationTokens to not depend on test order and thus pass
    on JDK7. (vinodkv)

    YARN-186. Coverage fixing LinuxContainerExecutor (Aleksey Gorshkov via
    bobby)

    YARN-216. Remove jquery theming support. (Robert Joseph Evans via jlowe)

    YARN-80. Add support for delaying rack-local containers in
    CapacityScheduler. (acmurthy) 

  OPTIMIZATIONS

  BUG FIXES
    
    YARN-131. Fix incorrect ACL properties in capacity scheduler documentation.
    (Ahmed Radwan via sseth)

    YARN-163. Retrieving container log via NM webapp can hang with multibyte
    characters in log (jlowe via bobby)

    YARN-102. Move the apache header to the top of the file in MemStore.java.
    (Devaraj K via sseth)

    YARN-174. Modify NodeManager to pass the user's configuration even when
    rebooting. (vinodkv)

    YARN-177. CapacityScheduler - adding a queue while the RM is running has 
    wacky results (acmurthy vai tgraves)

    YARN-178. Fix custom ProcessTree instance creation (Radim Kolar via bobby)

    YARN-180. Capacity scheduler - containers that get reserved create container
    token to early (acmurthy and bobby)

    YARN-139. Interrupted Exception within AsyncDispatcher leads to user
    confusion. (Vinod Kumar Vavilapalli via jlowe)

    YARN-165. RM should point tracking URL to RM web page for app when AM fails
    (jlowe via bobby)

    YARN-159. RM web ui applications page should be sorted to display last app
    first (tgraves via bobby)

    YARN-166. capacity scheduler doesn't allow capacity < 1.0 (tgraves via
    bobby)

    YARN-189. Fixed a deadlock between RM's ApplicationMasterService and the
    dispatcher. (Thomas Graves via vinodkv)

    YARN-202. Log Aggregation generates a storm of fsync() for namenode 
    (Kihwal Lee via bobby)

    YARN-201. Fix CapacityScheduler to be less conservative for starved 
    off-switch requests. (jlowe via acmurthy) 

    YARN-206. TestApplicationCleanup.testContainerCleanup occasionally fails
    (jlowe via bobby)

    YARN-212. NM state machine ignores an APPLICATION_CONTAINER_FINISHED event
    when it shouldn't (Nathan Roberts via jlowe)

    YARN-144. MiniMRYarnCluster launches RM and JHS on default ports (Robert
    Parker via jlowe)

    YARN-219. NM should aggregate logs when application finishes. (bobby)

    YARN-188. Coverage fixing for CapacityScheduler (Aleksey Gorshkov via
    bobby)

    YARN-214. RMContainerImpl does not handle event EXPIRE at state RUNNING
    (jeagles via bobby)

    YARN-151. Browser thinks RM main page JS is taking too long 
    (Ravi Prakash via bobby)

Release 0.23.4

  INCOMPATIBLE CHANGES

  NEW FEATURES

  IMPROVEMENTS

  OPTIMIZATIONS

  BUG FIXES

    YARN-108. FSDownload can create cache directories with the wrong
    permissions (Jason Lowe via bobby)

    YARN-88. DefaultContainerExecutor can fail to set proper permissions.
    (Jason Lowe via sseth)

    YARN-42. Modify NM's non-aggregating logs' handler to stop properly so that
    NMs don't get NPEs on startup errors. (Devaraj K via vinodkv)

    YARN-75. Modified ResourceManager's RMContainer to handle a valid RELEASE
    event at RUNNING state. (Siddharth Seth via vinodkv)

    YARN-57. Allow process-tree based resource calculation et al. to be
    pluggable to support it on multiple platforms. (Radim Kolar via acmurthy)

    YARN-93. Diagnostics missing from applications that have finished but
    failed (jlowe via bobby)

    YARN-106. Nodemanager needs to set permissions of local directories (jlowe
    via bobby)

Release 0.23.3

  INCOMPATIBLE CHANGES

  NEW FEATURES

    YARN-1. Promote YARN to be a sub-project of Apache Hadoop. (acmurthy)

  IMPROVEMENTS

  BUG FIXES

    YARN-14. Symlinks to peer distributed cache files no longer work 
    (Jason Lowe via bobby) 

    YARN-25. remove old aggregated logs  (Robert Evans via tgraves)

    YARN-27. Failed refreshQueues due to misconfiguration prevents further 
    refreshing of queues (Arun Murthy via tgraves)

    MAPREDUCE-4323. NM leaks filesystems (Jason Lowe via jeagles)

    MAPREDUCE-2374. "Text File Busy" errors launching MR tasks. (Andy Isaacson
    via atm)

    YARN-39. RM-NM secret-keys should be randomly generated and rolled every
    so often. (vinodkv and sseth via sseth)

    YARN-31. Fix TestDelegationTokenRenewer to not depend on test order so as to
    pass tests on jdk7. (Thomas Graves via vinodkv)

    YARN-63. RMNodeImpl is missing valid transitions from the UNHEALTHY state
    (Jason Lowe via bobby)

    YARN-66. aggregated logs permissions not set properly (tgraves via bobby)

    YARN-60. Fixed a bug in ResourceManager which causes all NMs to get NPEs and
    thus causes all containers to be rejected. (vinodkv)

    YARN-68. NodeManager will refuse to shutdown indefinitely due to container
    log aggregation (daryn via bobby)

    YARN-87. NM ResourceLocalizationService does not set permissions of local 
    cache directories (Jason Lowe via tgraves)