| <META http-equiv="Content-Type" content="text/html; charset=UTF-8"> |
| <title>Hadoop 2.7.2 Release Notes</title> |
| <STYLE type="text/css"> |
| H1 {font-family: sans-serif} |
| H2 {font-family: sans-serif; margin-left: 7mm} |
| TABLE {margin-left: 7mm} |
| </STYLE> |
| </head> |
| <body> |
| <h1>Hadoop 2.7.2 Release Notes</h1> |
| These release notes include new developer and user-facing incompatibilities, features, and major improvements. |
| <a name="changes"/> |
| <h2>Changes since Hadoop 2.7.1</h2> |
| <ul> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4321">YARN-4321</a>. |
| Major bug reported by Varun Saxena and fixed by Varun Saxena (resourcemanager)<br> |
| <b>Incessant retries if NoAuthException is thrown by Zookeeper in non HA mode</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4320">YARN-4320</a>. |
| Major bug reported by Varun Saxena and fixed by Varun Saxena <br> |
| <b>TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4313">YARN-4313</a>. |
| Major bug reported by Jian He and fixed by Jian He <br> |
| <b>Race condition in MiniMRYarnCluster when getting history server address</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4312">YARN-4312</a>. |
| Major bug reported by Varun Saxena and fixed by Varun Saxena <br> |
| <b>TestSubmitApplicationWithRMHA fails on branch-2.7 and branch-2.6 as some of the test cases time out </b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4281">YARN-4281</a>. |
| Blocker bug reported by Chang Li and fixed by Chang Li <br> |
| <b>2.7 RM app page is broken</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4209">YARN-4209</a>. |
| Critical bug reported by zhihai xu and fixed by zhihai xu (resourcemanager)<br> |
| <b>RMStateStore FENCED state doesn’t work due to updateFencedState called by stateMachine.doTransition</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4180">YARN-4180</a>. |
| Critical bug reported by Anubhav Dhoot and fixed by Anubhav Dhoot (resourcemanager)<br> |
| <b>AMLauncher does not retry on failures when talking to NM </b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4158">YARN-4158</a>. |
| Minor improvement reported by zhihai xu and fixed by zhihai xu (nodemanager)<br> |
| <b>Remove duplicate close for LogWriter in AppLogAggregatorImpl#uploadLogsForContainers</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4127">YARN-4127</a>. |
| Major bug reported by Jian He and fixed by Varun Saxena (resourcemanager)<br> |
| <b>RM fail with noAuth error if switched from failover mode to non-failover mode </b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4105">YARN-4105</a>. |
| Major bug reported by Chang Li and fixed by Chang Li (capacityscheduler)<br> |
| <b>Capacity Scheduler headroom for DRF is wrong</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4103">YARN-4103</a>. |
| Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles <br> |
| <b>RM WebServices missing scheme for appattempts logLinks</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4101">YARN-4101</a>. |
| Critical sub-task reported by Yesha Vora and fixed by Xuan Gong (yarn)<br> |
| <b>RM should print alert messages if Zookeeper and Resourcemanager gets connection issue</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4096">YARN-4096</a>. |
| Major bug reported by Jason Lowe and fixed by Jason Lowe (log-aggregation , nodemanager)<br> |
| <b>App local logs are leaked if log aggregation fails to initialize for the app</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4092">YARN-4092</a>. |
| Major sub-task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager)<br> |
| <b>RM HA UI redirection needs to be fixed when both RMs are in standby mode</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4087">YARN-4087</a>. |
| Major bug reported by Jian He and fixed by Jian He <br> |
| <b>Followup fixes after YARN-2019 regarding RM behavior when state-store error occurs</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4047">YARN-4047</a>. |
| Major bug reported by Jason Lowe and fixed by Jason Lowe (resourcemanager)<br> |
| <b>ClientRMService getApplications has high scheduler lock contention</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4041">YARN-4041</a>. |
| Major bug reported by Jason Lowe and fixed by Sunil G (resourcemanager)<br> |
| <b>Slow delegation token renewal can severely prolong RM recovery</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4009">YARN-4009</a>. |
| Major bug reported by Prakash Ramachandran and fixed by Varun Vasudev <br> |
| <b>CORS support for ResourceManager REST API</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4005">YARN-4005</a>. |
| Major bug reported by Jun Gong and fixed by Jun Gong <br> |
| <b>Completed container whose app is finished is not removed from NMStateStore</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-4000">YARN-4000</a>. |
| Major bug reported by Jason Lowe and fixed by Varun Saxena (capacityscheduler , resourcemanager)<br> |
| <b>RM crashes with NPE if leaf queue becomes parent queue during restart</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3999">YARN-3999</a>. |
| Major bug reported by Jian He and fixed by Jian He <br> |
| <b>RM hangs on draining events</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3990">YARN-3990</a>. |
| Critical bug reported by Rohith Sharma K S and fixed by Bibin A Chundatt (resourcemanager)<br> |
| <b>AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3978">YARN-3978</a>. |
| Major improvement reported by Eric Payne and fixed by Eric Payne (timelineserver , yarn)<br> |
| <b>Configurably turn off the saving of container info in Generic AHS</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3975">YARN-3975</a>. |
| Major bug reported by Mit Desai and fixed by Mit Desai <br> |
| <b>WebAppProxyServlet should not redirect to RM page if AHS is enabled</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3969">YARN-3969</a>. |
| Major sub-task reported by Subru Krishnan and fixed by Subru Krishnan (capacityscheduler , fairscheduler , resourcemanager)<br> |
| <b>Allow jobs to be submitted to reservation that is active but does not have any allocations</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3967">YARN-3967</a>. |
| Major bug reported by Mit Desai and fixed by Mit Desai <br> |
| <b>Fetch the application report from the AHS if the RM does not know about it</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3925">YARN-3925</a>. |
| Critical bug reported by zhihai xu and fixed by zhihai xu (nodemanager)<br> |
| <b>ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks.</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3905">YARN-3905</a>. |
| Major bug reported by Eric Payne and fixed by Eric Payne (timelineserver)<br> |
| <b>Application History Server UI NPEs when accessing apps run after RM restart</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3896">YARN-3896</a>. |
| Major bug reported by Jun Gong and fixed by Jun Gong (resourcemanager)<br> |
| <b>RMNode transitioned from RUNNING to REBOOTED because its response id had not been reset synchronously</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3893">YARN-3893</a>. |
| Critical sub-task reported by Bibin A Chundatt and fixed by Bibin A Chundatt (resourcemanager)<br> |
| <b>Both RM in active state when Admin#transitionToActive failure from refeshAll()</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3878">YARN-3878</a>. |
| Critical bug reported by Varun Saxena and fixed by Varun Saxena <br> |
| <b>AsyncDispatcher can hang while stopping if it is configured for draining events on stop</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3857">YARN-3857</a>. |
| Critical bug reported by mujunchao and fixed by mujunchao (resourcemanager)<br> |
| <b>Memory leak in ResourceManager with SIMPLE mode</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3802">YARN-3802</a>. |
| Major bug reported by zhihai xu and fixed by zhihai xu (resourcemanager)<br> |
| <b>Two RMNodes for the same NodeId are used in RM sometimes after NM is reconnected.</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3798">YARN-3798</a>. |
| Blocker bug reported by Bibin A Chundatt and fixed by Varun Saxena (resourcemanager)<br> |
| <b>ZKRMStateStore shouldn't create new session without occurrance of SESSIONEXPIED</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3793">YARN-3793</a>. |
| Major bug reported by Karthik Kambatla and fixed by Varun Saxena (nodemanager)<br> |
| <b>Several NPEs when deleting local files on NM recovery</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3780">YARN-3780</a>. |
| Minor bug reported by zhihai xu and fixed by zhihai xu (resourcemanager)<br> |
| <b>Should use equals when compare Resource in RMNodeImpl#ReconnectNodeTransition</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3740">YARN-3740</a>. |
| Major sub-task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager , webapp , yarn)<br> |
| <b>Fixed the typo with the configuration name: APPLICATION_HISTORY_PREFIX_MAX_APPS</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3727">YARN-3727</a>. |
| Major improvement reported by zhihai xu and fixed by zhihai xu (nodemanager)<br> |
| <b>For better error recovery, check if the directory exists before using it for localization.</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3700">YARN-3700</a>. |
| Major sub-task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager , webapp , yarn)<br> |
| <b>ATS Web Performance issue at load time when large number of jobs</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3697">YARN-3697</a>. |
| Critical bug reported by zhihai xu and fixed by zhihai xu (fairscheduler)<br> |
| <b>FairScheduler: ContinuousSchedulingThread can fail to shutdown</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3690">YARN-3690</a>. |
| Major bug reported by Akira AJISAKA and fixed by Brahma Reddy Battula (api , site)<br> |
| <b>[JDK8] 'mvn site' fails</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3624">YARN-3624</a>. |
| Major bug reported by Mit Desai and fixed by Mit Desai (timelineserver)<br> |
| <b>ApplicationHistoryServer reverses the order of the filters it gets</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3619">YARN-3619</a>. |
| Major bug reported by Jason Lowe and fixed by zhihai xu (nodemanager)<br> |
| <b>ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3580">YARN-3580</a>. |
| Major test reported by Robert Kanter and fixed by Robert Kanter (test)<br> |
| <b>[JDK 8] TestClientRMService.testGetLabelsToNodes fails</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3535">YARN-3535</a>. |
| Critical bug reported by Peng Zhang and fixed by Peng Zhang (capacityscheduler , fairscheduler , resourcemanager)<br> |
| <b>Scheduler must re-request container resources when RMContainer transitions from ALLOCATED to KILLED</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3508">YARN-3508</a>. |
| Major bug reported by Jason Lowe and fixed by Varun Saxena (resourcemanager , scheduler)<br> |
| <b>Prevent processing preemption events on the main RM dispatcher</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3248">YARN-3248</a>. |
| Major improvement reported by Varun Vasudev and fixed by Varun Vasudev (capacityscheduler , resourcemanager)<br> |
| <b>Display count of nodes blacklisted by apps in the web UI</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3170">YARN-3170</a>. |
| Major improvement reported by Allen Wittenauer and fixed by Brahma Reddy Battula (documentation)<br> |
| <b>YARN architecture document needs updating</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-3136">YARN-3136</a>. |
| Major sub-task reported by Jason Lowe and fixed by Sunil G (scheduler)<br> |
| <b>getTransferredContainers can be a bottleneck during AM registration</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-2902">YARN-2902</a>. |
| Major sub-task reported by Jason Lowe and fixed by Varun Saxena (nodemanager)<br> |
| <b>Killing a container that is localizing can orphan resources in the DOWNLOADING state</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-2890">YARN-2890</a>. |
| Major bug reported by Mit Desai and fixed by Mit Desai <br> |
| <b>MiniYarnCluster should turn on timeline service if configured to do so</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-2859">YARN-2859</a>. |
| Critical bug reported by Hitesh Shah and fixed by Vinod Kumar Vavilapalli (timelineserver)<br> |
| <b>ApplicationHistoryServer binds to default port 8188 in MiniYARNCluster</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-2801">YARN-2801</a>. |
| Major sub-task reported by Gururaj Shetty and fixed by Wangda Tan (documentation)<br> |
| <b>Add documentation for node labels feature</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-2513">YARN-2513</a>. |
| Major sub-task reported by Jonathan Eagles and fixed by Jonathan Eagles (timelineserver)<br> |
| <b>Host framework UIs in YARN for use with the ATS</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/YARN-2019">YARN-2019</a>. |
| Critical bug reported by Junping Du and fixed by Jian He <br> |
| <b>Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6528">MAPREDUCE-6528</a>. |
| Critical bug reported by Junping Du and fixed by Junping Du (jobhistoryserver)<br> |
| <b>Memory leak for HistoryFileManager.getJobSummary()</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6518">MAPREDUCE-6518</a>. |
| Major bug reported by Nathan Roberts and fixed by Chang Li (mrv2 , nodemanager)<br> |
| <b>Set SO_KEEPALIVE on shuffle connections</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6497">MAPREDUCE-6497</a>. |
| Major bug reported by Shinichi Yamashita and fixed by Shinichi Yamashita <br> |
| <b>Fix wrong value of JOB_FINISHED event in JobHistoryEventHandler</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6492">MAPREDUCE-6492</a>. |
| Critical bug reported by Bibin A Chundatt and fixed by Bibin A Chundatt <br> |
| <b>AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6481">MAPREDUCE-6481</a>. |
| Critical bug reported by zhihai xu and fixed by zhihai xu (mrv2)<br> |
| <b>LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes.</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6474">MAPREDUCE-6474</a>. |
| Major bug reported by Nathan Roberts and fixed by Kuhu Shukla (mrv2 , nodemanager)<br> |
| <b>ShuffleHandler can possibly exhaust nodemanager file descriptors</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6472">MAPREDUCE-6472</a>. |
| Major bug reported by Jason Lowe and fixed by Naganarasimha G R (mr-am)<br> |
| <b>MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6454">MAPREDUCE-6454</a>. |
| Critical bug reported by Junping Du and fixed by Junping Du <br> |
| <b>MapReduce doesn't set the HADOOP_CLASSPATH for jar lib in distributed cache.</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6451">MAPREDUCE-6451</a>. |
| Major bug reported by Kuhu Shukla and fixed by Kuhu Shukla (distcp)<br> |
| <b>DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6442">MAPREDUCE-6442</a>. |
| Major bug reported by Chang Li and fixed by Chang Li (client)<br> |
| <b>Stack trace is missing when error occurs in client protocol provider's constructor</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6439">MAPREDUCE-6439</a>. |
| Critical bug reported by Anubhav Dhoot and fixed by Anubhav Dhoot <br> |
| <b>AM may fail instead of retrying if RM shuts down during the allocate call</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6426">MAPREDUCE-6426</a>. |
| Major bug reported by Devaraj K and fixed by zhihai xu (test)<br> |
| <b>TestShuffleHandler#testGetMapOutputInfo is failing</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6425">MAPREDUCE-6425</a>. |
| Major bug reported by zhihai xu and fixed by zhihai xu (mrv2 , nodemanager)<br> |
| <b>ShuffleHandler passes wrong "base" parameter to getMapOutputInfo if mapId is not in the cache.</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-6273">MAPREDUCE-6273</a>. |
| Minor bug reported by zhihai xu and fixed by zhihai xu (jobhistoryserver)<br> |
| <b>HistoryFileManager should check whether summaryFile exists to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5982">MAPREDUCE-5982</a>. |
| Major bug reported by Jason Lowe and fixed by Chang Li (mr-am)<br> |
| <b>Task attempts that fail from the ASSIGNED state can disappear</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5948">MAPREDUCE-5948</a>. |
| Critical bug reported by Kris Geusebroek and fixed by Akira AJISAKA <br> |
| <b>org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-5649">MAPREDUCE-5649</a>. |
| Major bug reported by stanley shi and fixed by Gera Shegalov (mrv2)<br> |
| <b>Reduce cannot use more than 2G memory for the final merge</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-9317">HDFS-9317</a>. |
| Major bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)<br> |
| <b>Document fsck -blockId and -storagepolicy options in branch-2.7</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-9305">HDFS-9305</a>. |
| Major bug reported by Chris Nauroth and fixed by Arpit Agarwal (datanode)<br> |
| <b>Delayed heartbeat processing causes storm of subsequent heartbeats</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-9290">HDFS-9290</a>. |
| Blocker bug reported by Tony Wu and fixed by Tony Wu <br> |
| <b>DFSClient#callAppend() is not backward compatible for slightly older NameNodes</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-9221">HDFS-9221</a>. |
| Major improvement reported by Staffan Friberg and fixed by Staffan Friberg (performance)<br> |
| <b>HdfsServerConstants#ReplicaState#getState should avoid calling values() since it creates a temporary array</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-9220">HDFS-9220</a>. |
| Blocker bug reported by Bogdan Raducanu and fixed by Jing Zhao <br> |
| <b>Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-9178">HDFS-9178</a>. |
| Critical bug reported by Kihwal Lee and fixed by Kihwal Lee <br> |
| <b>Slow datanode I/O can cause a wrong node to be marked bad</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-9106">HDFS-9106</a>. |
| Critical bug reported by Kihwal Lee and fixed by Kihwal Lee <br> |
| <b>Transfer failure during pipeline recovery causes permanent write failures</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-9083">HDFS-9083</a>. |
| Blocker bug reported by Rushabh S Shah and fixed by Rushabh S Shah (HDFS , namenode)<br> |
| <b>Replication violates block placement policy.</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-9043">HDFS-9043</a>. |
| Minor bug reported by J.Andreina and fixed by J.Andreina (documentation)<br> |
| <b>Doc updation for commands in HDFS Federation</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-9042">HDFS-9042</a>. |
| Minor bug reported by J.Andreina and fixed by J.Andreina (documentation)<br> |
| <b>Update document for the Storage policy name</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-9033">HDFS-9033</a>. |
| Major bug reported by Archana T and fixed by Brahma Reddy Battula <br> |
| <b>dfsadmin -metasave prints "NaN" for cache used%</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8995">HDFS-8995</a>. |
| Critical bug reported by Kihwal Lee and fixed by Kihwal Lee <br> |
| <b>Flaw in registration bookeeping can make DN die on reconnect</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8950">HDFS-8950</a>. |
| Major bug reported by Daniel Templeton and fixed by Daniel Templeton (datanode , namenode)<br> |
| <b>NameNode refresh doesn't remove DataNodes that are no longer in the allowed list</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8891">HDFS-8891</a>. |
| Blocker bug reported by Yong Zhang and fixed by Yong Zhang <br> |
| <b>HDFS concat should keep srcs order</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8879">HDFS-8879</a>. |
| Major bug reported by Kihwal Lee and fixed by Xiaoyu Yao (namenode)<br> |
| <b>Quota by storage type usage incorrectly initialized upon namenode restart</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8867">HDFS-8867</a>. |
| Major bug reported by Rushabh S Shah and fixed by Daryn Sharp <br> |
| <b>Enable optimized block reports</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8863">HDFS-8863</a>. |
| Critical bug reported by Kihwal Lee and fixed by Kihwal Lee <br> |
| <b>The remaining space check in BlockPlacementPolicyDefault is flawed</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8852">HDFS-8852</a>. |
| Major bug reported by Hong Dai Thanh and fixed by Ajith S (documentation)<br> |
| <b>HDFS architecture documentation of version 2.x is outdated about append write support</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8850">HDFS-8850</a>. |
| Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (datanode)<br> |
| <b>VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8846">HDFS-8846</a>. |
| Major bug reported by Zhe Zhang and fixed by Zhe Zhang (namenode)<br> |
| <b>Add a unit test for INotify functionality across a layout version upgrade</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8806">HDFS-8806</a>. |
| Major bug reported by Zhe Zhang and fixed by Zhe Zhang <br> |
| <b>Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8767">HDFS-8767</a>. |
| Critical bug reported by Haohui Mai and fixed by Kanaka Kumar Avvaru <br> |
| <b>RawLocalFileSystem.listStatus() returns null for UNIX pipefile</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8727">HDFS-8727</a>. |
| Major improvement reported by Andrew Baptist and fixed by Andrew Baptist (HDFS)<br> |
| <b>Allow using path style addressing for accessing the s3 endpoint</b><br> |
| <blockquote>Add new flag to allow supporting path style addressing for s3a</blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8722">HDFS-8722</a>. |
| Critical improvement reported by Kihwal Lee and fixed by Kihwal Lee <br> |
| <b>Optimize datanode writes for small writes and flushes</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8676">HDFS-8676</a>. |
| Critical bug reported by Kihwal Lee and fixed by Walter Su <br> |
| <b>Delayed rolling upgrade finalization can cause heartbeat expiration and write failures</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8659">HDFS-8659</a>. |
| Major improvement reported by Yongjun Zhang and fixed by Yongjun Zhang (datanode)<br> |
| <b>Block scanner INFO message is spamming logs</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8656">HDFS-8656</a>. |
| Critical bug reported by Andrew Wang and fixed by Andrew Wang (rolling upgrades)<br> |
| <b>Preserve compatibility of ClientProtocol#rollingUpgrade after finalization</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8431">HDFS-8431</a>. |
| Critical bug reported by Sumana Sathish and fixed by Anu Engineer (HDFS)<br> |
| <b>hdfs crypto class not found in Windows</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8384">HDFS-8384</a>. |
| Minor improvement reported by Tsz Wo Nicholas Sze and fixed by Jing Zhao (namenode)<br> |
| <b>Allow NN to startup if there are files having a lease but are not under construction</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8219">HDFS-8219</a>. |
| Major bug reported by Peter Shi and fixed by Surendra Singh Lilhore <br> |
| <b>setStoragePolicy with folder behavior is different after cluster restart</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8099">HDFS-8099</a>. |
| Minor bug reported by Charles Lamb and fixed by Charles Lamb (hdfs-client)<br> |
| <b>Change "DFSInputStream has been closed already" message to debug log level</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-8046">HDFS-8046</a>. |
| Major bug reported by Kihwal Lee and fixed by Kihwal Lee <br> |
| <b>Allow better control of getContentSummary</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-7725">HDFS-7725</a>. |
| Major bug reported by Ming Ma and fixed by Ming Ma <br> |
| <b>Incorrect "nodes in service" metrics caused all writes to fail</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-7645">HDFS-7645</a>. |
| Major improvement reported by Nathan Roberts and fixed by Keisuke Ogiwara (datanode)<br> |
| <b>Rolling upgrade is restoring blocks from trash multiple times</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-7609">HDFS-7609</a>. |
| Critical bug reported by Carrey Zhan and fixed by Ming Ma (namenode)<br> |
| <b>Avoid retry cache collision when Standby NameNode loading edits</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-7314">HDFS-7314</a>. |
| Major improvement reported by Ming Ma and fixed by Ming Ma <br> |
| <b>When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HDFS-6945">HDFS-6945</a>. |
| Critical bug reported by Akira AJISAKA and fixed by Akira AJISAKA (namenode)<br> |
| <b>BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-12465">HADOOP-12465</a>. |
| Minor bug reported by Martin Petricek and fixed by Jagadesh Kiran N (documentation)<br> |
| <b>Incorrect javadoc in WritableUtils.java</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-12464">HADOOP-12464</a>. |
| Major bug reported by Kihwal Lee and fixed by Kihwal Lee (ipc)<br> |
| <b>Interrupted client may try to fail-over and retry</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-12451">HADOOP-12451</a>. |
| Blocker bug reported by Karthik Kambatla and fixed by Karthik Kambatla (scripts)<br> |
| <b>[Branch-2] Setting HADOOP_HOME explicitly should be allowed</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-12359">HADOOP-12359</a>. |
| Major bug reported by Daniel Templeton and fixed by Jagadesh Kiran N (documentation)<br> |
| <b>hadoop fs -getmerge doc is wrong</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-12304">HADOOP-12304</a>. |
| Blocker bug reported by Chris Nauroth and fixed by Chris Nauroth (fs)<br> |
| <b>Applications using FileContext fail with the default file system configured to be wasb/s3/etc.</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-12280">HADOOP-12280</a>. |
| Minor improvement reported by Masatake Iwasaki and fixed by Masatake Iwasaki (test)<br> |
| <b>Skip unit tests based on maven profile rather than NativeCodeLoader.isNativeCodeLoaded</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-12232">HADOOP-12232</a>. |
| Major improvement reported by Chris Nauroth and fixed by Chris Nauroth (build)<br> |
| <b>Upgrade Tomcat dependency to 6.0.44.</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-12230">HADOOP-12230</a>. |
| Minor bug reported by Steve Loughran and fixed by Rakesh R (build)<br> |
| <b>hadoop-project declares duplicate, conflicting curator dependencies</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-12213">HADOOP-12213</a>. |
| Minor bug reported by Oleg Zhurakousky and fixed by Kuhu Shukla <br> |
| <b>Interrupted exception can occur when Client#stop is called</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-12191">HADOOP-12191</a>. |
| Major bug reported by Jason Lowe and fixed by Brahma Reddy Battula (io)<br> |
| <b>Bzip2Factory is not thread safe</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-12186">HADOOP-12186</a>. |
| Major bug reported by zhihai xu and fixed by zhihai xu (ha)<br> |
| <b>ActiveStandbyElector shouldn't call monitorLockNodeAsync multiple times</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-12006">HADOOP-12006</a>. |
| Major bug reported by Akira AJISAKA and fixed by Akira AJISAKA <br> |
| <b>Remove unimplemented option for `hadoop fs -ls` from document in branch-2.7</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-11932">HADOOP-11932</a>. |
| Critical bug reported by Jian He and fixed by Brahma Reddy Battula <br> |
| <b> MetricsSinkAdapter hangs when being stopped</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-11812">HADOOP-11812</a>. |
| Blocker improvement reported by Gera Shegalov and fixed by Gera Shegalov (fs)<br> |
| <b>Implement listLocatedStatus for ViewFileSystem to speed up split calculation</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-11491">HADOOP-11491</a>. |
| Critical bug reported by Gera Shegalov and fixed by Brahma Reddy Battula (fs)<br> |
| <b>HarFs incorrectly declared as requiring an authority</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-10365">HADOOP-10365</a>. |
| Minor bug reported by Ted Yu and fixed by Kiran Kumar M R (util)<br> |
| <b>BufferedOutputStream in FileUtil#unpackEntries() should be closed in finally block</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-9242">HADOOP-9242</a>. |
| Major bug reported by Andrey Klochkov and fixed by Andrey Klochkov (test)<br> |
| <b>Duplicate surefire plugin config in hadoop-common</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-8151">HADOOP-8151</a>. |
| Major bug reported by Todd Lipcon and fixed by Matt Foley (io , native)<br> |
| <b>Error handling in snappy decompressor throws invalid exceptions</b><br> |
| <blockquote></blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-7139">HADOOP-7139</a>. |
| Major improvement reported by Stephen Rose and fixed by Kanaka Kumar Avvaru (io)<br> |
| <b>Allow appending to existing SequenceFiles</b><br> |
| <blockquote>Existing sequence files can be appended.</blockquote></li> |
| <li> <a href="https://issues.apache.org/jira/browse/HADOOP-5323">HADOOP-5323</a>. |
| Minor improvement reported by Suman Sehgal and fixed by Weiwei Yang (documentation)<br> |
| <b>Trash documentation should describe its directory structure and configurations</b><br> |
| <blockquote></blockquote></li> |
| </ul> |
| </body></html> |
