hadoop-2.0.2-alpha rc1
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.2-alpha-rc1@1392055 13f79535-47bb-0310-9956-ffa450edef68
diff --git a/hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html b/hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html
index 50db08c..8ac55c1 100644
--- a/hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html
+++ b/hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html
@@ -12,6 +12,28 @@
<a name="changes"/>
<h2>Changes since Hadoop 2.0.1-alpha</h2>
<ul>
+<li> <a href="https://issues.apache.org/jira/browse/YARN-137">YARN-137</a>.
+ Major improvement reported by Siddharth Seth and fixed by Siddharth Seth (scheduler)<br>
+ <b>Change the default scheduler to the CapacityScheduler</b><br>
+ <blockquote>There's some bugs in the FifoScheduler atm - doesn't distribute tasks across nodes and some headroom (available resource) issues.
+That's not the best experience for users trying out the 2.0 branch. The CS with the default configuration of a single queue behaves the same as the FifoScheduler and doesn't have these issues.
+</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/YARN-108">YARN-108</a>.
+ Critical bug reported by Jason Lowe and fixed by Jason Lowe (nodemanager)<br>
+ <b>FSDownload can create cache directories with the wrong permissions</b><br>
+ <blockquote>When the cluster is configured with a restrictive umask, e.g.: {{fs.permissions.umask-mode=0077}}, the nodemanager can end up creating directory entries in the public cache with the wrong permissions. The permissions can end up where only the nodemanager user can access files in the public cache, preventing jobs from running properly.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/YARN-106">YARN-106</a>.
+ Major bug reported by Jason Lowe and fixed by Jason Lowe (nodemanager)<br>
+ <b>Nodemanager needs to set permissions of local directories</b><br>
+ <blockquote>If the nodemanager process is running with a restrictive default umask (e.g.: 0077) then it will create its local directories with permissions that are too restrictive to allow containers from other users to run.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/YARN-88">YARN-88</a>.
+ Major bug reported by Jason Lowe and fixed by Jason Lowe (nodemanager)<br>
+ <b>DefaultContainerExecutor can fail to set proper permissions</b><br>
+ <blockquote>{{DefaultContainerExecutor}} can fail to set the proper permissions on its local directories if the cluster has been configured with a restrictive umask, e.g.: fs.permissions.umask-mode=0077. The configured umask ends up defeating the permissions requested by {{DefaultContainerExecutor}} when it creates directories.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/YARN-87">YARN-87</a>.
+ Critical bug reported by Jason Lowe and fixed by Jason Lowe (nodemanager)<br>
+ <b>NM ResourceLocalizationService does not set permissions of local cache directories</b><br>
+ <blockquote>{{ResourceLocalizationService}} creates a file cache and user cache directory when it starts up but doesn't specify the permissions for them when they are created. If the cluster configs are set to limit the default permissions (e.g.: fs.permissions.umask-mode=0077 instead of the default 0022), then the cache directories are created with too-restrictive permissions and no jobs are able to run.</blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/YARN-83">YARN-83</a>.
Major bug reported by Bikas Saha and fixed by Bikas Saha (client)<br>
<b>Change package of YarnClient to include apache</b><br>
@@ -250,10 +272,22 @@
Major task reported by Arun C Murthy and fixed by Arun C Murthy <br>
<b>Move YARN out of hadoop-mapreduce</b><br>
<blockquote>Move YARN out of hadoop-mapreduce-project into hadoop-yarn-project in hadoop trunk</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4691">MAPREDUCE-4691</a>.
+ Critical bug reported by Jason Lowe and fixed by Robert Joseph Evans (jobhistoryserver , mrv2)<br>
+ <b>Historyserver can report "Unknown job" after RM says job has completed</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4689">MAPREDUCE-4689</a>.
+ Major bug reported by Jason Lowe and fixed by Jason Lowe (client)<br>
+ <b>JobClient.getMapTaskReports on failed job results in NPE</b><br>
+ <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4649">MAPREDUCE-4649</a>.
Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (jobhistoryserver)<br>
<b>mr-jobhistory-daemon.sh needs to be updated post YARN-1</b><br>
<blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4647">MAPREDUCE-4647</a>.
+ Major bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>We should only unjar jobjar if there is a lib directory in it.</b><br>
+ <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4646">MAPREDUCE-4646</a>.
Major bug reported by Jason Lowe and fixed by Jason Lowe (mrv2)<br>
<b>client does not receive job diagnostics for failed jobs</b><br>
@@ -518,10 +552,6 @@
Major new feature reported by Karthik Kambatla and fixed by Karthik Kambatla (mrv1 , mrv2)<br>
<b>Add RunningJob.getJobStatus()</b><br>
<blockquote></blockquote></li>
-<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4342">MAPREDUCE-4342</a>.
- Major bug reported by Mayank Bansal and fixed by Mayank Bansal <br>
- <b>Distributed Cache gives inconsistent result if cache files get deleted from task tracker </b><br>
- <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4341">MAPREDUCE-4341</a>.
Major bug reported by Thomas Graves and fixed by Karthik Kambatla (capacity-sched , mrv2)<br>
<b>add types to capacity scheduler properties documentation</b><br>
@@ -550,6 +580,10 @@
Major bug reported by Ahmed Radwan and fixed by Ahmed Radwan (mrv2)<br>
<b>Problem running Distributed Shell applications as a user other than the one started the daemons</b><br>
<blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4302">MAPREDUCE-4302</a>.
+ Critical bug reported by Daryn Sharp and fixed by Daryn Sharp (nodemanager)<br>
+ <b>NM goes down if error encountered during log aggregation</b><br>
+ <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4301">MAPREDUCE-4301</a>.
Major improvement reported by Robert Joseph Evans and fixed by Robert Joseph Evans (applicationmaster)<br>
<b>Dedupe some strings in MRAM for memory savings</b><br>
@@ -918,10 +952,6 @@
Critical bug reported by MengWang and fixed by Bhallamudi Venkata Siva Kamesh (mrv2)<br>
<b>Shuffle hang when set map.failures.percent</b><br>
<blockquote></blockquote></li>
-<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3921">MAPREDUCE-3921</a>.
- Major improvement reported by Vinod Kumar Vavilapalli and fixed by Bikas Saha (mr-am , mrv2)<br>
- <b>MR AM should act on the nodes liveliness information when nodes go up/down/unhealthy</b><br>
- <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3907">MAPREDUCE-3907</a>.
Minor improvement reported by Eugene Koontz and fixed by Eugene Koontz (documentation)<br>
<b>Document entries mapred-default.xml for the jobhistory server.</b><br>
@@ -950,6 +980,10 @@
Major bug reported by Bhallamudi Venkata Siva Kamesh and fixed by Bhallamudi Venkata Siva Kamesh (mrv2)<br>
<b>Invalid App Metrics</b><br>
<blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3850">MAPREDUCE-3850</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (security)<br>
+ <b>Avoid redundant calls for tokens in TokenCache</b><br>
+ <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3842">MAPREDUCE-3842</a>.
Critical improvement reported by Alejandro Abdelnur and fixed by Thomas Graves (mrv2 , webapps)<br>
<b>stop webpages from automatic refreshing</b><br>
@@ -1064,14 +1098,14 @@
Minor new feature reported by Philip Zeyliger and fixed by Ahmed Radwan (build , test)<br>
<b>Exposing MiniDFS and MiniMR clusters as a single process command-line</b><br>
<blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3972">HDFS-3972</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Trash emptier fails in secure HA cluster</b><br>
+ <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HDFS-3928">HDFS-3928</a>.
Major bug reported by Eli Collins and fixed by Eli Collins (test)<br>
<b>MiniDFSCluster should reset the first ExitException on shutdown</b><br>
<blockquote></blockquote></li>
-<li> <a href="https://issues.apache.org/jira/browse/HDFS-3910">HDFS-3910</a>.
- Minor improvement reported by Eli Collins and fixed by Eli Collins (test)<br>
- <b>DFSTestUtil#waitReplication should timeout</b><br>
- <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HDFS-3902">HDFS-3902</a>.
Minor bug reported by Andy Isaacson and fixed by Andy Isaacson <br>
<b>TestDatanodeBlockScanner#testBlockCorruptionPolicy is broken</b><br>
@@ -1236,6 +1270,10 @@
Minor bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (fuse-dfs)<br>
<b>fuse_dfs: incorrect configuration value checked for connection expiry timer period</b><br>
<blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3731">HDFS-3731</a>.
+ Blocker bug reported by Suresh Srinivas and fixed by Kihwal Lee (data-node)<br>
+ <b>2.0 release upgrade must handle blocks being written from 1.0</b><br>
+ <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HDFS-3724">HDFS-3724</a>.
Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur <br>
<b>add InterfaceAudience annotations to HttpFS classes and making inner enum static</b><br>
@@ -1328,10 +1366,6 @@
Major improvement reported by Eli Collins and fixed by Eli Collins (test)<br>
<b>MiniDFSCluster should capture the code path that led to the first ExitException </b><br>
<blockquote></blockquote></li>
-<li> <a href="https://issues.apache.org/jira/browse/HDFS-3659">HDFS-3659</a>.
- Minor improvement reported by Brandon Li and fixed by Brandon Li (documentation)<br>
- <b>Add missing @Override to methods across the hadoop-hdfs project</b><br>
- <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HDFS-3658">HDFS-3658</a>.
Major bug reported by Eli Collins and fixed by Tsz Wo (Nicholas), SZE <br>
<b>TestDFSClientRetries#testNamenodeRestart failed</b><br>
@@ -1364,6 +1398,10 @@
Trivial bug reported by Brandon Li and fixed by Brandon Li (name-node)<br>
<b>fix the typo in the error message about inconsistent storage layout version</b><br>
<blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3622">HDFS-3622</a>.
+ Major bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans <br>
+ <b>Backport HDFS-3541 to branch-0.23</b><br>
+ <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HDFS-3615">HDFS-3615</a>.
Major bug reported by Eli Collins and fixed by Aaron T. Myers (security)<br>
<b>Two BlockTokenSecretManager findbugs warnings</b><br>
@@ -1732,10 +1770,6 @@
Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (hdfs client)<br>
<b>DFSClient.close should close cached sockets</b><br>
<blockquote></blockquote></li>
-<li> <a href="https://issues.apache.org/jira/browse/HDFS-3343">HDFS-3343</a>.
- Major improvement reported by Todd Lipcon and fixed by Andrew Wang (data-node)<br>
- <b>Improve metrics for DN read latency</b><br>
- <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HDFS-3341">HDFS-3341</a>.
Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon <br>
<b>Change minimum RPC versions to 2.0.0-SNAPSHOT instead of 2.0.0</b><br>
@@ -1768,6 +1802,10 @@
Critical bug reported by Daryn Sharp and fixed by Daryn Sharp (webhdfs)<br>
<b>hftp/webhdfs can't get tokens if authority has no port</b><br>
<blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3306">HDFS-3306</a>.
+ Minor bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe <br>
+ <b>fuse_dfs: don't lock release operations</b><br>
+ <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HDFS-3291">HDFS-3291</a>.
Major test reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur <br>
<b>add test that covers HttpFS working w/ a non-HDFS Hadoop filesystem</b><br>
@@ -2132,10 +2170,6 @@
Major improvement reported by Eli Collins and fixed by Eli Collins (fs)<br>
<b>Make trash a server side configuration option</b><br>
<blockquote>If fs.trash.interval is configured on the server then the client's value for this configuration is ignored. </blockquote></li>
-<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8687">HADOOP-8687</a>.
- Minor improvement reported by Eli Collins and fixed by Eli Collins <br>
- <b>Upgrade log4j to 1.2.17</b><br>
- <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8686">HADOOP-8686</a>.
Minor bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (native)<br>
<b>Fix warnings in native code</b><br>
@@ -2168,6 +2202,10 @@
Critical bug reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
<b>FilterFileSystem#setWriteChecksum is broken</b><br>
<blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8635">HADOOP-8635</a>.
+ Critical improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Cannot cancel paths registered deleteOnExit</b><br>
+ <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8634">HADOOP-8634</a>.
Critical bug reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
<b>Ensure FileSystem#close doesn't squawk for deleteOnExit paths</b><br>
@@ -2532,10 +2570,6 @@
Major bug reported by Radim Kolar and fixed by Radim Kolar (build)<br>
<b>A few pom.xml across Hadoop project may fail XML validation</b><br>
<blockquote></blockquote></li>
-<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8249">HADOOP-8249</a>.
- Major bug reported by bc Wong and fixed by Alejandro Abdelnur (security)<br>
- <b>invalid hadoop-auth cookies should trigger authentication if info is avail before returning HTTP 401</b><br>
- <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8244">HADOOP-8244</a>.
Major improvement reported by Henry Robinson and fixed by Henry Robinson <br>
<b>Improve comments on ByteBufferReadable.read</b><br>
@@ -2568,6 +2602,10 @@
Critical bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (conf)<br>
<b>Configuration logs WARNs on every use of a deprecated key</b><br>
<blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8180">HADOOP-8180</a>.
+ Major bug reported by Ravi Prakash and fixed by Ravi Prakash <br>
+ <b>Remove hsqldb since its not needed from pom.xml</b><br>
+ <blockquote></blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8179">HADOOP-8179</a>.
Minor bug reported by Steve Loughran and fixed by Daryn Sharp (fs)<br>
<b>risk of NPE in CopyCommands processArguments()</b><br>
@@ -2670,3 +2708,8019 @@
<blockquote></blockquote></li>
</ul>
</body></html>
+<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
+<title>Hadoop 2.0.1-alpha Release Notes</title>
+<STYLE type="text/css">
+ H1 {font-family: sans-serif}
+ H2 {font-family: sans-serif; margin-left: 7mm}
+ TABLE {margin-left: 7mm}
+</STYLE>
+</head>
+<body>
+<h1>Hadoop 2.0.1-alpha Release Notes</h1>
+These release notes include new developer and user-facing incompatibilities, features, and major improvements.
+<a name="changes"/>
+<h2>Changes since Hadoop 2.0.0-alpha</h2>
+<ul>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8552">HADOOP-8552</a>.
+ Major bug reported by Karthik Kambatla and fixed by Karthik Kambatla (conf , security)<br>
+ <b>Conflict: Same security.log.file for multiple users. </b><br>
+ <blockquote></blockquote></li>
+</ul>
+</body></html>
+<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
+<title>Hadoop 2.0.0-alpha Release Notes</title>
+<STYLE type="text/css">
+ H1 {font-family: sans-serif}
+ H2 {font-family: sans-serif; margin-left: 7mm}
+ TABLE {margin-left: 7mm}
+</STYLE>
+</head>
+<body>
+<h1>Hadoop 2.0.0-alpha Release Notes</h1>
+These release notes include new developer and user-facing incompatibilities, features, and major improvements.
+<a name="changes"/>
+<h2>Changes since Hadoop 0.23.2</h2>
+<ul>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4274">MAPREDUCE-4274</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (performance , task)<br>
+ <b>MapOutputBuffer should use native byte order for kvmeta</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4231">MAPREDUCE-4231</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (contrib/raid)<br>
+ <b>Update RAID to not to use FSInodeInfo</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4219">MAPREDUCE-4219</a>.
+ Major improvement reported by Roman Shaposhnik and fixed by Roman Shaposhnik (security)<br>
+ <b>make default container-executor.conf.dir be a path relative to the container-executor binary</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4202">MAPREDUCE-4202</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (test)<br>
+ <b>TestYarnClientProtocolProvider is broken</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4193">MAPREDUCE-4193</a>.
+ Major bug reported by Patrick Hunt and fixed by Patrick Hunt (documentation)<br>
+ <b>broken doc link for yarn-default.xml in site.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4147">MAPREDUCE-4147</a>.
+ Major bug reported by Tom White and fixed by Tom White <br>
+ <b>YARN should not have a compile-time dependency on HDFS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4138">MAPREDUCE-4138</a>.
+ Major improvement reported by Tom White and fixed by Tom White <br>
+ <b>Reduce memory usage of counters due to non-static nested classes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4113">MAPREDUCE-4113</a>.
+ Major sub-task reported by Devaraj K and fixed by Devaraj K (mrv2 , test)<br>
+ <b>Fix tests org.apache.hadoop.mapred.TestClusterMRNotification</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4112">MAPREDUCE-4112</a>.
+ Major sub-task reported by Devaraj K and fixed by Devaraj K (mrv2 , test)<br>
+ <b>Fix tests org.apache.hadoop.mapred.TestClusterMapReduceTestCase</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4111">MAPREDUCE-4111</a>.
+ Major sub-task reported by Devaraj K and fixed by Devaraj K (mrv2 , test)<br>
+ <b>Fix tests in org.apache.hadoop.mapred.TestJobName</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4110">MAPREDUCE-4110</a>.
+ Major sub-task reported by Devaraj K and fixed by Devaraj K (mrv2 , test)<br>
+ <b>Fix tests in org.apache.hadoop.mapred.TestMiniMRClasspath & org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4108">MAPREDUCE-4108</a>.
+ Major sub-task reported by Devaraj K and fixed by Devaraj K (mrv2)<br>
+ <b>Fix tests in org.apache.hadoop.util.TestRunJar</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4107">MAPREDUCE-4107</a>.
+ Major sub-task reported by Devaraj K and fixed by Devaraj K (mrv2)<br>
+ <b>Fix tests in org.apache.hadoop.ipc.TestSocketFactory</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4105">MAPREDUCE-4105</a>.
+ Major bug reported by Ahmed Radwan and fixed by Ahmed Radwan (mrv2)<br>
+ <b>Yarn RackResolver ignores rack configurations</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4103">MAPREDUCE-4103</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (documentation)<br>
+ <b>Fix HA docs for changes to shell command fencer args</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4098">MAPREDUCE-4098</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (test)<br>
+ <b>TestMRApps testSetClasspath fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4093">MAPREDUCE-4093</a>.
+ Major improvement reported by Devaraj K and fixed by Devaraj K (mrv2)<br>
+ <b>Improve RM WebApp start up when proxy address is not set</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4081">MAPREDUCE-4081</a>.
+ Blocker bug reported by Jason Lowe and fixed by Jason Lowe (build , mrv2)<br>
+ <b>TestMROutputFormat.java does not compile</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4076">MAPREDUCE-4076</a>.
+ Blocker bug reported by Devaraj K and fixed by Devaraj K (mrv2)<br>
+ <b>Stream job fails with ZipException when use yarn jar command</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4066">MAPREDUCE-4066</a>.
+ Minor bug reported by xieguiming and fixed by xieguiming (job submission , mrv2)<br>
+ <b>To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4057">MAPREDUCE-4057</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Devaraj K (contrib/raid)<br>
+ <b>Compilation error in RAID </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4008">MAPREDUCE-4008</a>.
+ Major bug reported by Devaraj K and fixed by Devaraj K (mrv2 , scheduler)<br>
+ <b>ResourceManager throws MetricsException on start up saying QueueMetrics MBean already exists</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4007">MAPREDUCE-4007</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (mrv2)<br>
+ <b>JobClient getJob(JobID) should return NULL if the job does not exist (for backwards compatibility)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3991">MAPREDUCE-3991</a>.
+ Trivial improvement reported by Harsh J and fixed by Harsh J (documentation)<br>
+ <b>Streaming FAQ has some wrong instructions about input files splitting</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3989">MAPREDUCE-3989</a>.
+ Major improvement reported by Patrick Hunt and fixed by Patrick Hunt <br>
+ <b>cap space usage of default log4j rolling policy (mr specific changes)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3974">MAPREDUCE-3974</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Aaron T. Myers <br>
+ <b>TestSubmitJob in MR1 tests doesn't compile after HDFS-1623 merge</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3958">MAPREDUCE-3958</a>.
+ Major bug reported by Bikas Saha and fixed by Bikas Saha (mrv2)<br>
+ <b>RM: Remove RMNodeState and replace it with NodeState</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3955">MAPREDUCE-3955</a>.
+ Blocker improvement reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey (mrv2)<br>
+ <b>Replace ProtoOverHadoopRpcEngine with ProtobufRpcEngine.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3952">MAPREDUCE-3952</a>.
+ Major bug reported by Zhenxiao Luo and fixed by Bhallamudi Venkata Siva Kamesh (mrv2)<br>
+ <b>In MR2, when Total input paths to process == 1, CombinefileInputFormat.getSplits() returns 0 split.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3935">MAPREDUCE-3935</a>.
+ Major improvement reported by Tom White and fixed by Tom White (client)<br>
+ <b>Annotate Counters.Counter and Counters.Group as @Public</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3933">MAPREDUCE-3933</a>.
+ Major bug reported by Ahmed Radwan and fixed by Ahmed Radwan (mrv2 , test)<br>
+ <b>Failures because MALLOC_ARENA_MAX is not set</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3916">MAPREDUCE-3916</a>.
+ Critical bug reported by Roman Shaposhnik and fixed by Devaraj K (mrv2 , resourcemanager , webapps)<br>
+ <b>various issues with running yarn proxyserver</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3909">MAPREDUCE-3909</a>.
+ Trivial improvement reported by Steve Loughran and fixed by Steve Loughran (mrv2)<br>
+ <b>javadoc the Service interfaces</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3885">MAPREDUCE-3885</a>.
+ Major improvement reported by Devaraj Das and fixed by Devaraj Das (mrv2)<br>
+ <b>Apply the fix similar to HADOOP-8084</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3883">MAPREDUCE-3883</a>.
+ Minor improvement reported by Eugene Koontz and fixed by Eugene Koontz (documentation , mrv2)<br>
+ <b>Document yarn.nodemanager.delete.debug-delay-sec configuration property</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3869">MAPREDUCE-3869</a>.
+ Blocker bug reported by Devaraj K and fixed by Devaraj K (mrv2)<br>
+ <b>Distributed shell application fails with NoClassDefFoundError</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3867">MAPREDUCE-3867</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (test)<br>
+ <b>MiniMRYarn/MiniYarn uses fixed ports</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3818">MAPREDUCE-3818</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Suresh Srinivas (build , test)<br>
+ <b>Trunk MRV1 compilation is broken.</b><br>
+ <blockquote>Fixed broken compilation in TestSubmitJob after the patch for HDFS-2895.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3740">MAPREDUCE-3740</a>.
+ Blocker bug reported by Devaraj K and fixed by Devaraj K (mrv2)<br>
+ <b>Mapreduce Trunk compilation fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3578">MAPREDUCE-3578</a>.
+ Major bug reported by Gilad Wolff and fixed by Tom White (nodemanager)<br>
+ <b>starting nodemanager as 'root' gives "Unknown -jvm option"</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3545">MAPREDUCE-3545</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>Remove Avro RPC</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3431">MAPREDUCE-3431</a>.
+ Minor bug reported by Steve Loughran and fixed by Steve Loughran (resourcemanager)<br>
+ <b>NPE in Resource Manager shutdown</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3377">MAPREDUCE-3377</a>.
+ Major bug reported by Jane Chen and fixed by Jane Chen <br>
+ <b>Compatibility issue with 0.20.203.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3353">MAPREDUCE-3353</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Bikas Saha (applicationmaster , mrv2 , resourcemanager)<br>
+ <b>Need a RM->AM channel to inform AMs about faulty/unhealthy/lost nodes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3173">MAPREDUCE-3173</a>.
+ Critical bug reported by Devaraj K and fixed by Devaraj K (mrv2)<br>
+ <b>MRV2 UI doesn't work properly without internet</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2942">MAPREDUCE-2942</a>.
+ Critical bug reported by Vinod Kumar Vavilapalli and fixed by Thomas Graves <br>
+ <b>TestNMAuditLogger.testNMAuditLoggerWithIP failing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2934">MAPREDUCE-2934</a>.
+ Major improvement reported by Aaron T. Myers and fixed by Aaron T. Myers (mrv2)<br>
+ <b>MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup process</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2887">MAPREDUCE-2887</a>.
+ Major improvement reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>MR changes to match HADOOP-7524 (multiple RPC protocols)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3418">HDFS-3418</a>.
+ Minor improvement reported by Eli Collins and fixed by Eli Collins <br>
+ <b>Rename BlockWithLocationsProto datanodeIDs field to storageIDs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3396">HDFS-3396</a>.
+ Minor bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (fuse-dfs)<br>
+ <b>FUSE build fails on Ubuntu 12.04</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3395">HDFS-3395</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (name-node)<br>
+ <b>NN doesn't start with HA+security enabled and HTTP address set to 0.0.0.0</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3378">HDFS-3378</a>.
+ Trivial improvement reported by Eli Collins and fixed by Eli Collins <br>
+ <b>Remove DFS_NAMENODE_SECONDARY_HTTPS_PORT_KEY and DEFAULT</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3376">HDFS-3376</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (hdfs client)<br>
+ <b>DFSClient fails to make connection to DN if there are many unusable cached sockets</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3375">HDFS-3375</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>Put client name in DataXceiver thread name for readBlock and keepalive</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3365">HDFS-3365</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (hdfs client)<br>
+ <b>Enable users to disable socket caching in DFS client configuration</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3363">HDFS-3363</a>.
+ Minor sub-task reported by John George and fixed by John George (name-node)<br>
+ <b>blockmanagement should stop using INodeFile & INodeFileUC </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3357">HDFS-3357</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>DataXceiver reads from client socket with incorrect/no timeout</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3351">HDFS-3351</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (name-node)<br>
+ <b>NameNode#initializeGenericKeys should always set fs.defaultFS regardless of whether HA or Federation is enabled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3350">HDFS-3350</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>findbugs warning: INodeFileUnderConstruction doesn't override INodeFile.equals(Object)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3339">HDFS-3339</a>.
+ Minor sub-task reported by John George and fixed by John George (name-node)<br>
+ <b>change INode to package private</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3336">HDFS-3336</a>.
+ Minor bug reported by Roman Shaposhnik and fixed by Roman Shaposhnik (scripts)<br>
+ <b>hdfs launcher script will be better off not special casing namenode command with regards to hadoop.security.logger</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3332">HDFS-3332</a>.
+ Major bug reported by amith and fixed by amith (data-node)<br>
+ <b>NullPointerException in DN when directoryscanner is trying to report bad blocks</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3330">HDFS-3330</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>If GetImageServlet throws an Error or RTE, response has HTTP "OK" status</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3328">HDFS-3328</a>.
+ Minor bug reported by Uma Maheswara Rao G and fixed by Eli Collins (data-node)<br>
+ <b>NPE in DataNode.getIpcPort</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3326">HDFS-3326</a>.
+ Trivial bug reported by J.Andreina and fixed by Matthew Jacobs (name-node)<br>
+ <b>Append enabled log message uses the wrong variable</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3322">HDFS-3322</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (hdfs client)<br>
+ <b>Update file context to use HdfsDataInputStream and HdfsDataOutputStream</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3319">HDFS-3319</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (hdfs client)<br>
+ <b>DFSOutputStream should not start a thread in constructors</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3314">HDFS-3314</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur <br>
+ <b>HttpFS operation for getHomeDirectory is incorrect</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3309">HDFS-3309</a>.
+ Major bug reported by Romain Rigaux and fixed by Alejandro Abdelnur <br>
+ <b>HttpFS (Hoop) chmod not supporting octal and sticky bit permissions</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3305">HDFS-3305</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (ha , name-node)<br>
+ <b>GetImageServlet should consider SBN a valid requestor in a secure HA setup</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3303">HDFS-3303</a>.
+ Minor bug reported by Brandon Li and fixed by Brandon Li (name-node)<br>
+ <b>RemoteEditLogManifest doesn't need to implements Writable</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3298">HDFS-3298</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (hdfs client)<br>
+ <b>Add HdfsDataOutputStream as a public API</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3294">HDFS-3294</a>.
+ Trivial improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node , name-node)<br>
+ <b>Fix indentation in NamenodeWebHdfsMethods and DatanodeWebHdfsMethods</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3286">HDFS-3286</a>.
+ Major bug reported by J.Andreina and fixed by Ashish Singhi (balancer)<br>
+ <b>When the threshold value for balancer is 0(zero) ,unexpected output is displayed</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3284">HDFS-3284</a>.
+ Minor bug reported by Todd Lipcon and fixed by Todd Lipcon (ha , security)<br>
+ <b>bootstrapStandby fails in secure cluster</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3282">HDFS-3282</a>.
+ Major sub-task reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (hdfs client)<br>
+ <b>Add HdfsDataInputStream as a public API</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3280">HDFS-3280</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (hdfs client)<br>
+ <b>DFSOutputStream.sync should not be synchronized</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3279">HDFS-3279</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Arpit Gupta (name-node)<br>
+ <b>One of the FSEditLog constructors should be moved to TestEditLog</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3275">HDFS-3275</a>.
+ Major bug reported by Vinithra Varadharajan and fixed by amith (ha , name-node)<br>
+ <b>Format command overwrites contents of non-empty shared edits dir if name dirs are empty without any prompting</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3268">HDFS-3268</a>.
+ Critical bug reported by Daryn Sharp and fixed by Daryn Sharp (ha , hdfs client)<br>
+ <b>Hdfs mishandles token service & incompatible with HA</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3263">HDFS-3263</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur <br>
+ <b>HttpFS should read HDFS config from Hadoop site.xml files</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3260">HDFS-3260</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers <br>
+ <b>TestDatanodeRegistration should set minimum DN version in addition to minimum NN version</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3259">HDFS-3259</a>.
+ Major improvement reported by Aaron T. Myers and fixed by Aaron T. Myers (ha , name-node)<br>
+ <b>NameNode#initializeSharedEdits should populate shared edits dir with edit log segments</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3256">HDFS-3256</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers <br>
+ <b>HDFS considers blocks under-replicated if topology script is configured with only 1 rack</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3255">HDFS-3255</a>.
+ Critical bug reported by Daryn Sharp and fixed by Daryn Sharp (ha , hdfs client)<br>
+ <b>HA DFS returns wrong token service</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3254">HDFS-3254</a>.
+ Major bug reported by Anupam Seth and fixed by Anupam Seth (fuse-dfs)<br>
+ <b>Branch-2 build broken due to wrong version number in fuse-dfs' pom.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3249">HDFS-3249</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Use ToolRunner.confirmPrompt in NameNode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3248">HDFS-3248</a>.
+ Minor bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe <br>
+ <b>bootstrapstanby repeated twice in hdfs namenode usage message</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3247">HDFS-3247</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (ha)<br>
+ <b>Improve bootstrapStandby behavior when original NN is not active</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3244">HDFS-3244</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins <br>
+ <b>Remove dead writable code from hdfs/protocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3240">HDFS-3240</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>Drop log level of "heartbeat: ..." in BPServiceActor to DEBUG</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3238">HDFS-3238</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins <br>
+ <b>ServerCommand and friends don't need to be writables</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3236">HDFS-3236</a>.
+ Minor bug reported by Aaron T. Myers and fixed by Aaron T. Myers (ha , name-node)<br>
+ <b>NameNode does not initialize generic conf keys when started with -initializeSharedEditsDir</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3234">HDFS-3234</a>.
+ Trivial bug reported by Todd Lipcon and fixed by Todd Lipcon (tools)<br>
+ <b>Accidentally left log message in GetConf after HDFS-3226</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3226">HDFS-3226</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (tools)<br>
+ <b>Allow GetConf tool to print arbitrary keys</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3222">HDFS-3222</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (hdfs client)<br>
+ <b>DFSInputStream#openInfo should not silently get the length as 0 when locations length is zero for last partial block.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3214">HDFS-3214</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>InterDatanodeProtocolServerSideTranslatorPB doesn't handle null response from initReplicaRecovery</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3211">HDFS-3211</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (ha , name-node)<br>
+ <b>JournalProtocol changes required for introducing epoch and fencing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3210">HDFS-3210</a>.
+ Major bug reported by Eli Collins and fixed by Eli Collins <br>
+ <b>JsonUtil#toJsonMap for for a DatanodeInfo should use "ipAddr" instead of "name"</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3208">HDFS-3208</a>.
+ Major bug reported by Eli Collins and fixed by Eli Collins (name-node)<br>
+ <b>Bogus entries in hosts files are incorrectly displayed in the report </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3204">HDFS-3204</a>.
+ Major improvement reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>Minor modification to JournalProtocol.proto to make it generic</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3202">HDFS-3202</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (data-node)<br>
+ <b>NamespaceInfo PB translation drops build version</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3199">HDFS-3199</a>.
+ Major bug reported by Eli Collins and fixed by Todd Lipcon <br>
+ <b>TestValidateConfigurationSettings is failing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3187">HDFS-3187</a>.
+ Minor sub-task reported by Todd Lipcon and fixed by Todd Lipcon (build)<br>
+ <b>Upgrade guava to 11.0.2</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3181">HDFS-3181</a>.
+ Minor bug reported by Colin Patrick McCabe and fixed by Tsz Wo (Nicholas), SZE (test)<br>
+ <b>testHardLeaseRecoveryAfterNameNodeRestart fails when length before restart is 1 byte less than CRC chunk size</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3179">HDFS-3179</a>.
+ Major improvement reported by Zhanwei.Wang and fixed by Tsz Wo (Nicholas), SZE (hdfs client)<br>
+ <b>Improve the error message: DataStreamer throw an exception, "nodes.length != original.length + 1" on single datanode cluster</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3172">HDFS-3172</a>.
+ Trivial improvement reported by Eli Collins and fixed by Eli Collins (name-node)<br>
+ <b>dfs.upgrade.permission is dead code</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3171">HDFS-3171</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins (data-node)<br>
+ <b>The DatanodeID "name" field is overloaded </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3169">HDFS-3169</a>.
+ Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (test)<br>
+ <b>TestFsck should test multiple -move operations in a row</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3167">HDFS-3167</a>.
+ Minor new feature reported by Henry Robinson and fixed by Henry Robinson (test)<br>
+ <b>CLI-based driver for MiniDFSCluster</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3164">HDFS-3164</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins (data-node)<br>
+ <b>Move DatanodeInfo#hostName to DatanodeID</b><br>
+ <blockquote>This change modifies DatanodeID, which is part of the client to server protocol, therefore clients must be upgraded with servers.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3160">HDFS-3160</a>.
+ Major bug reported by Roman Shaposhnik and fixed by Roman Shaposhnik (scripts)<br>
+ <b>httpfs should exec catalina instead of forking it</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3158">HDFS-3158</a>.
+ Major improvement reported by Aaron T. Myers and fixed by Aaron T. Myers (name-node)<br>
+ <b>LiveNodes member of NameNodeMXBean should list non-DFS used space and capacity per DN</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3156">HDFS-3156</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (test)<br>
+ <b>TestDFSHAAdmin is failing post HADOOP-8202</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3155">HDFS-3155</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Clean up FSDataset implemenation related code.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3148">HDFS-3148</a>.
+ Major new feature reported by Eli Collins and fixed by Eli Collins (hdfs client , performance)<br>
+ <b>The client should be able to use multiple local interfaces for data transfer</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3144">HDFS-3144</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins (data-node)<br>
+ <b>Refactor DatanodeID#getName by use</b><br>
+ <blockquote>This change modifies DatanodeID, which is part of the client to server protocol, therefore clients must be upgraded with servers.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3143">HDFS-3143</a>.
+ Major bug reported by Eli Collins and fixed by Arpit Gupta (test)<br>
+ <b>TestGetBlocks.testGetBlocks is failing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3142">HDFS-3142</a>.
+ Blocker bug reported by Eli Collins and fixed by Brandon Li (test)<br>
+ <b>TestHDFSCLI.testAll is failing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3139">HDFS-3139</a>.
+ Minor improvement reported by Eli Collins and fixed by Eli Collins (data-node)<br>
+ <b>Minor Datanode logging improvement</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3138">HDFS-3138</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins <br>
+ <b>Move DatanodeInfo#ipcPort to DatanodeID</b><br>
+ <blockquote>This change modifies DatanodeID, which is part of the client to server protocol, therefore clients must be upgraded with servers.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3137">HDFS-3137</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins (name-node)<br>
+ <b>Bump LAST_UPGRADABLE_LAYOUT_VERSION to -16</b><br>
+ <blockquote>Upgrade from Hadoop versions earlier than 0.18 is not supported as of 2.0. To upgrade from an earlier release, first upgrade to 0.18, and then upgrade again from there.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3132">HDFS-3132</a>.
+ Minor bug reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Findbugs warning on HDFS trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3130">HDFS-3130</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Move FSDataset implemenation to a package</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3129">HDFS-3129</a>.
+ Minor test reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe <br>
+ <b>NetworkTopology: add test that getLeaf should check for invalid topologies</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3121">HDFS-3121</a>.
+ Major bug reported by John George and fixed by John George <br>
+ <b>hdfs tests for HADOOP-8014</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3120">HDFS-3120</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins <br>
+ <b>Enable hsync and hflush by default</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3119">HDFS-3119</a>.
+ Minor bug reported by J.Andreina and fixed by Ashish Singhi (name-node)<br>
+ <b>Overreplicated block is not deleted even after the replication factor is reduced after sync follwed by closing that file</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3111">HDFS-3111</a>.
+ Trivial task reported by Todd Lipcon and fixed by Uma Maheswara Rao G <br>
+ <b>Missing license headers in trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3109">HDFS-3109</a>.
+ Major bug reported by Ravi Prakash and fixed by Ravi Prakash <br>
+ <b>Remove hsqldb exclusions from pom.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3105">HDFS-3105</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node , hdfs client)<br>
+ <b>Add DatanodeStorage information to block recovery</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3102">HDFS-3102</a>.
+ Major new feature reported by Todd Lipcon and fixed by Aaron T. Myers (ha , name-node)<br>
+ <b>Add CLI tool to initialize the shared-edits dir</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3101">HDFS-3101</a>.
+ Major bug reported by Zhanwei.Wang and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>cannot read empty file using webhdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3100">HDFS-3100</a>.
+ Major bug reported by Zhanwei.Wang and fixed by Brandon Li (data-node)<br>
+ <b>failed to append data</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3099">HDFS-3099</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (name-node)<br>
+ <b>SecondaryNameNode does not properly initialize metrics system</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3094">HDFS-3094</a>.
+ Major improvement reported by Arpit Gupta and fixed by Arpit Gupta <br>
+ <b>add -nonInteractive and -force option to namenode -format command</b><br>
+ <blockquote>The 'namenode -format' command now supports the flags '-nonInteractive' and '-force' to improve usefulness without user input.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3093">HDFS-3093</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>TestAllowFormat is trying to be interactive</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3091">HDFS-3091</a>.
+ Major improvement reported by Uma Maheswara Rao G and fixed by Tsz Wo (Nicholas), SZE (data-node , hdfs client , name-node)<br>
+ <b>Update the usage limitations of ReplaceDatanodeOnFailure policy in the config description for the smaller clusters.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3089">HDFS-3089</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Move FSDatasetInterface and other related classes/interfaces to a package</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3088">HDFS-3088</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Move FSDatasetInterface inner classes to a package</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3086">HDFS-3086</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Change Datanode not to send storage list in registration - it will be sent in block report</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3084">HDFS-3084</a>.
+ Major improvement reported by Philip Zeyliger and fixed by Todd Lipcon (ha)<br>
+ <b>FenceMethod.tryFence() and ShellCommandFencer should pass namenodeId as well as host:port</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3083">HDFS-3083</a>.
+ Critical bug reported by Mingjie Lai and fixed by Aaron T. Myers (ha , security)<br>
+ <b>Cannot run an MR job with HA and security enabled when second-listed NN active</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3082">HDFS-3082</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Clean up FSDatasetInterface</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3071">HDFS-3071</a>.
+ Major improvement reported by Philip Zeyliger and fixed by Todd Lipcon (ha)<br>
+ <b>haadmin failover command does not provide enough detail for when target NN is not ready to be active</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3070">HDFS-3070</a>.
+ Major bug reported by Stephen Chu and fixed by Aaron T. Myers (balancer)<br>
+ <b>HDFS balancer doesn't ensure that hdfs-site.xml is loaded</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3066">HDFS-3066</a>.
+ Major improvement reported by Patrick Hunt and fixed by Patrick Hunt (scripts)<br>
+ <b>cap space usage of default log4j rolling policy (hdfs specific changes)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3062">HDFS-3062</a>.
+ Critical bug reported by Mingjie Lai and fixed by Mingjie Lai (ha , security)<br>
+ <b>Fail to submit mapred job on a secured-HA-HDFS: logic URI cannot be picked up by job submission.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3057">HDFS-3057</a>.
+ Major bug reported by Roman Shaposhnik and fixed by Roman Shaposhnik (scripts)<br>
+ <b>httpfs and hdfs launcher scripts should honor CATALINA_HOME and HADOOP_LIBEXEC_DIR</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3056">HDFS-3056</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Add an interface for DataBlockScanner logging</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3050">HDFS-3050</a>.
+ Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (name-node)<br>
+ <b>rework OEV to share more code with the NameNode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3044">HDFS-3044</a>.
+ Major improvement reported by Eli Collins and fixed by Colin Patrick McCabe (name-node)<br>
+ <b>fsck move should be non-destructive by default</b><br>
+ <blockquote>The fsck "move" option is no longer destructive. It copies the accessible blocks of corrupt files to lost and found as before, but no longer deletes the corrupt files after copying the blocks. The original, destructive behavior can be enabled by specifying both the "move" and "delete" options. </blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3038">HDFS-3038</a>.
+ Trivial bug reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Add FSEditLog.metrics to findbugs exclude list</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3036">HDFS-3036</a>.
+ Trivial improvement reported by Aaron T. Myers and fixed by Aaron T. Myers (name-node)<br>
+ <b>Remove unused method DFSUtil#isDefaultNamenodeAddress</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3032">HDFS-3032</a>.
+ Major bug reported by Kihwal Lee and fixed by Kihwal Lee (hdfs client)<br>
+ <b>Lease renewer tries forever even if renewal is not possible</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3030">HDFS-3030</a>.
+ Major improvement reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Remove getProtocolVersion and getProtocolSignature from translators</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3026">HDFS-3026</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (ha , name-node)<br>
+ <b>HA: Handle failure during HA state transition</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3024">HDFS-3024</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Improve performance of stringification in addStoredBlock</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3021">HDFS-3021</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Use generic type to declare FSDatasetInterface</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3020">HDFS-3020</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Auto-logSync based on edit log buffer size broken</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3014">HDFS-3014</a>.
+ Major improvement reported by Sho Shimauchi and fixed by Sho Shimauchi (name-node)<br>
+ <b>FSEditLogOp and its subclasses should have toString() method</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3005">HDFS-3005</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>ConcurrentModificationException in FSDataset$FSVolume.getDfsUsed(..)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3004">HDFS-3004</a>.
+ Major new feature reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (tools)<br>
+ <b>Implement Recovery Mode</b><br>
+ <blockquote>This is a new feature. It is documented in hdfs_user_guide.xml.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3003">HDFS-3003</a>.
+ Trivial improvement reported by Brandon Li and fixed by Brandon Li (name-node)<br>
+ <b>Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString()</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3000">HDFS-3000</a>.
+ Major new feature reported by Aaron T. Myers and fixed by Aaron T. Myers (hdfs client)<br>
+ <b>Add a public API for setting quotas</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2995">HDFS-2995</a>.
+ Major bug reported by Todd Lipcon and fixed by Eli Collins (scripts)<br>
+ <b>start-dfs.sh should only start the 2NN for namenodes with dfs.namenode.secondary.http-address configured</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2983">HDFS-2983</a>.
+ Major improvement reported by Eli Collins and fixed by Aaron T. Myers <br>
+ <b>Relax the build version check to permit rolling upgrades within a release</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2968">HDFS-2968</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Todd Lipcon (data-node , name-node)<br>
+ <b>Protocol translator for BlockRecoveryCommand broken when multiple blocks need recovery</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2941">HDFS-2941</a>.
+ Major new feature reported by Aaron T. Myers and fixed by Aaron T. Myers (hdfs client , name-node)<br>
+ <b>Add an administrative command to download a copy of the fsimage from the NN</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2899">HDFS-2899</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>Service protocol change to support multiple storages added in HDFS-2880</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2895">HDFS-2895</a>.
+ Major improvement reported by Suresh Srinivas and fixed by Suresh Srinivas (data-node , name-node)<br>
+ <b>Remove Writable wire protocol related code that is no longer necessary</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2880">HDFS-2880</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (data-node , name-node)<br>
+ <b>Protocol buffer changes to add support multiple storages</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2878">HDFS-2878</a>.
+ Blocker bug reported by Eli Collins and fixed by Todd Lipcon (test)<br>
+ <b>TestBlockRecovery does not compile</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2815">HDFS-2815</a>.
+ Critical bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (name-node)<br>
+ <b>Namenode is not coming out of safemode when we perform ( NN crash + restart ) . Also FSCK report shows blocks missed.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2801">HDFS-2801</a>.
+ Major sub-task reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Provide a method in client side translators to check for a methods supported in underlying protocol.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2799">HDFS-2799</a>.
+ Major bug reported by Eli Collins and fixed by amith (name-node)<br>
+ <b>Trim fs.checkpoint.dir values</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2768">HDFS-2768</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (name-node)<br>
+ <b>BackupNode stop can not close proxy connections because it is not a proxy instance.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2765">HDFS-2765</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (test)<br>
+ <b>TestNameEditsConfigs is incorrectly swallowing IOE</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2739">HDFS-2739</a>.
+ Critical bug reported by Sho Shimauchi and fixed by Jitendra Nath Pandey <br>
+ <b>SecondaryNameNode doesn't start up</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2731">HDFS-2731</a>.
+ Major new feature reported by Aaron T. Myers and fixed by Todd Lipcon (ha)<br>
+ <b>HA: Autopopulate standby name dirs if they're empty</b><br>
+ <blockquote>The HA NameNode may now be started with the "-bootstrapStandby" flag. This causes it to copy the namespace information and most recent checkpoint from its HA pair, and save it to local storage, allowing an HA setup to be bootstrapped without use of rsync or external tools.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2708">HDFS-2708</a>.
+ Minor improvement reported by Eli Collins and fixed by Aaron T. Myers (data-node , name-node)<br>
+ <b>Stats for the # of blocks per DN</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2700">HDFS-2700</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G <br>
+ <b>TestDataNodeMultipleRegistrations is failing in trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2697">HDFS-2697</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Jitendra Nath Pandey <br>
+ <b>Move RefreshAuthPolicy, RefreshUserMappings, GetUserMappings protocol to protocol buffers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2696">HDFS-2696</a>.
+ Major bug reported by Petru Dimulescu and fixed by Bruno Mahé (build , fuse-dfs)<br>
+ <b>Fix the fuse-fds build</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2694">HDFS-2694</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (name-node)<br>
+ <b>Removal of Avro broke non-PB NN services</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2687">HDFS-2687</a>.
+ Major sub-task reported by Uma Maheswara Rao G and fixed by Suresh Srinivas (test)<br>
+ <b>Tests are failing with ClassCastException, due to new protocol changes </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2676">HDFS-2676</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>Remove Avro RPC</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2669">HDFS-2669</a>.
+ Major sub-task reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>Enable protobuf rpc for ClientNamenodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2666">HDFS-2666</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (test)<br>
+ <b>TestBackupNode fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2663">HDFS-2663</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>Optional parameters are not handled correctly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2661">HDFS-2661</a>.
+ Major sub-task reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Enable protobuf RPC for DatanodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2651">HDFS-2651</a>.
+ Major sub-task reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>ClientNameNodeProtocol Translators for Protocol Buffers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2650">HDFS-2650</a>.
+ Minor improvement reported by Hari Mankude and fixed by Hari Mankude <br>
+ <b>Replace @inheritDoc with @Override </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2647">HDFS-2647</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (balancer , data-node , hdfs client , name-node)<br>
+ <b>Enable protobuf RPC for InterDatanodeProtocol, ClientDatanodeProtocol, JournalProtocol and NamenodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2642">HDFS-2642</a>.
+ Major sub-task reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Protobuf translators for DatanodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2636">HDFS-2636</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>Implement protobuf service for ClientDatanodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2629">HDFS-2629</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (data-node)<br>
+ <b>Implement protobuf service for InterDatanodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2618">HDFS-2618</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>Implement protobuf service for NamenodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2597">HDFS-2597</a>.
+ Major sub-task reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b> ClientNameNodeProtocol in Protocol Buffers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2581">HDFS-2581</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>Implement protobuf service for JournalProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2532">HDFS-2532</a>.
+ Critical bug reported by Todd Lipcon and fixed by Uma Maheswara Rao G (test)<br>
+ <b>TestDfsOverAvroRpc timing out in trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2526">HDFS-2526</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (hdfs client , name-node)<br>
+ <b>(Client)NamenodeProtocolTranslatorR23 do not need to keep a reference to rpcProxyWithoutRetry</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2520">HDFS-2520</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (data-node)<br>
+ <b>Protobuf - Add protobuf service for InterDatanodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2519">HDFS-2519</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (data-node , name-node)<br>
+ <b>Protobuf - Add protobuf service for DatanodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2518">HDFS-2518</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>Protobuf - Add protobuf service for NamenodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2517">HDFS-2517</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>Protobuf - Add protocol service for JournalProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2507">HDFS-2507</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>HA: Allow saveNamespace operations to be canceled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2505">HDFS-2505</a>.
+ Minor test reported by Ravi Prakash and fixed by Ravi Prakash (test)<br>
+ <b>Add a test to verify getFileChecksum works with ViewFS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2499">HDFS-2499</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>Fix RPC client creation bug from HDFS-2459</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2497">HDFS-2497</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>Fix TestBackupNode failure</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2496">HDFS-2496</a>.
+ Major improvement reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>Separate datatypes for DatanodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2495">HDFS-2495</a>.
+ Major sub-task reported by Tomasz Nykiel and fixed by Tomasz Nykiel (name-node)<br>
+ <b>Increase granularity of write operations in ReplicationMonitor thus reducing contention for write lock</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2489">HDFS-2489</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>Move commands Finalize and Register out of DatanodeCommand class.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2488">HDFS-2488</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (data-node)<br>
+ <b>Separate datatypes for InterDatanodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2481">HDFS-2481</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Sanjay Radia <br>
+ <b>Unknown protocol: org.apache.hadoop.hdfs.protocol.ClientProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2480">HDFS-2480</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>Separate datatypes for NamenodeProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2479">HDFS-2479</a>.
+ Major sub-task reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>HDFS Client Data Types in Protocol Buffers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2477">HDFS-2477</a>.
+ Major sub-task reported by Tomasz Nykiel and fixed by Tomasz Nykiel (name-node)<br>
+ <b>Optimize computing the diff between a block report and the namenode state.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2476">HDFS-2476</a>.
+ Major sub-task reported by Tomasz Nykiel and fixed by Tomasz Nykiel (name-node)<br>
+ <b>More CPU efficient data structure for under-replicated/over-replicated/invalidate blocks</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2459">HDFS-2459</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>Separate datatypes for Journal protocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2430">HDFS-2430</a>.
+ Major new feature reported by Aaron T. Myers and fixed by Aaron T. Myers (name-node)<br>
+ <b>The number of failed or low-resource volumes the NN can tolerate should be configurable</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2413">HDFS-2413</a>.
+ Major improvement reported by Todd Lipcon and fixed by Harsh J (hdfs client)<br>
+ <b>Add public APIs for safemode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2410">HDFS-2410</a>.
+ Minor improvement reported by Suresh Srinivas and fixed by Suresh Srinivas (data-node , name-node , test)<br>
+ <b>Further clean up hard-coded configuration keys</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2351">HDFS-2351</a>.
+ Major improvement reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>Change Namenode and Datanode to register each of their protocols seperately </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2337">HDFS-2337</a>.
+ Major improvement reported by Aaron T. Myers and fixed by Aaron T. Myers (hdfs client)<br>
+ <b>DFSClient shouldn't keep multiple RPC proxy references</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2334">HDFS-2334</a>.
+ Major sub-task reported by Ivan Kelly and fixed by Ivan Kelly (name-node)<br>
+ <b>Add Closeable to JournalManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2303">HDFS-2303</a>.
+ Major improvement reported by Roman Shaposhnik and fixed by Mingjie Lai (build , scripts)<br>
+ <b>Unbundle jsvc</b><br>
+ <blockquote>To run secure Datanodes users must install jsvc for their platform and set JSVC_HOME to point to the location of jsvc in their environment.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2223">HDFS-2223</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Untangle depencencies between NN components</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2181">HDFS-2181</a>.
+ Major sub-task reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>Separate HDFS Client wire protocol data types</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2158">HDFS-2158</a>.
+ Major sub-task reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Add JournalSet to manage the set of journals.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2038">HDFS-2038</a>.
+ Critical test reported by Daryn Sharp and fixed by Kihwal Lee (test)<br>
+ <b>Update test to handle relative paths with globs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2018">HDFS-2018</a>.
+ Major sub-task reported by Ivan Kelly and fixed by Ivan Kelly <br>
+ <b>1073: Move all journal stream management code into one place</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1765">HDFS-1765</a>.
+ Major bug reported by Hairong Kuang and fixed by Uma Maheswara Rao G (name-node)<br>
+ <b>Block Replication should respect under-replication block priority</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1623">HDFS-1623</a>.
+ Major new feature reported by Sanjay Radia and fixed by <br>
+ <b>High Availability Framework for HDFS NN</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1580">HDFS-1580</a>.
+ Major improvement reported by Ivan Kelly and fixed by Jitendra Nath Pandey (name-node)<br>
+ <b>Add interface for generic Write Ahead Logging mechanisms</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-891">HDFS-891</a>.
+ Minor bug reported by Steve Loughran and fixed by Harsh J (data-node)<br>
+ <b>DataNode no longer needs to check for dfs.network.script</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-860">HDFS-860</a>.
+ Minor wish reported by Brian Bockelman and fixed by Brian Bockelman (fuse-dfs)<br>
+ <b>fuse-dfs truncate behavior causes issues with scp</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-395">HDFS-395</a>.
+ Major sub-task reported by dhruba borthakur and fixed by Tomasz Nykiel (data-node , name-node)<br>
+ <b>DFS Scalability: Incremental block reports</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-309">HDFS-309</a>.
+ Major improvement reported by Todd Lipcon and fixed by Sho Shimauchi <br>
+ <b>FSEditLog should log progress during replay</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-234">HDFS-234</a>.
+ Major new feature reported by Luca Telloli and fixed by Ivan Kelly <br>
+ <b>Integration with BookKeeper logging system</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-208">HDFS-208</a>.
+ Minor improvement reported by Allen Wittenauer and fixed by Uma Maheswara Rao G (name-node)<br>
+ <b>name node should warn if only one dir is listed in dfs.name.dir</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8619">HADOOP-8619</a>.
+ Major improvement reported by Radim Kolar and fixed by Chris Douglas (io)<br>
+ <b>WritableComparator must implement no-arg constructor</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8398">HADOOP-8398</a>.
+ Minor improvement reported by Eli Collins and fixed by Eli Collins <br>
+ <b>Cleanup BlockLocation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8388">HADOOP-8388</a>.
+ Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe <br>
+ <b>Remove unused BlockLocation serialization</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8366">HADOOP-8366</a>.
+ Blocker improvement reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>Use ProtoBuf for RpcResponseHeader</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8359">HADOOP-8359</a>.
+ Trivial task reported by Harsh J and fixed by Anupam Seth (conf)<br>
+ <b>Clear up javadoc warnings in hadoop-common-project</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8356">HADOOP-8356</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (fs)<br>
+ <b>FileSystem service loading mechanism should print the FileSystem impl it is failing to load</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8355">HADOOP-8355</a>.
+ Minor bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)<br>
+ <b>SPNEGO filter throws/logs exception when authentication fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8353">HADOOP-8353</a>.
+ Major improvement reported by Roman Shaposhnik and fixed by Roman Shaposhnik (scripts)<br>
+ <b>hadoop-daemon.sh and yarn-daemon.sh can be misleading on stop</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8350">HADOOP-8350</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (util)<br>
+ <b>Improve NetUtils.getInputStream to return a stream which has a tunable timeout</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8349">HADOOP-8349</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (viewfs)<br>
+ <b>ViewFS doesn't work when the root of a file system is mounted</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8347">HADOOP-8347</a>.
+ Major bug reported by Philip Zeyliger and fixed by Philip Zeyliger (security)<br>
+ <b>Hadoop Common logs misspell 'successful'</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8343">HADOOP-8343</a>.
+ Major new feature reported by Philip Zeyliger and fixed by Alejandro Abdelnur (util)<br>
+ <b>Allow configuration of authorization for JmxJsonServlet and MetricsServlet</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8314">HADOOP-8314</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)<br>
+ <b>HttpServer#hasAdminAccess should return false if authorization is enabled but user is not authenticated</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8310">HADOOP-8310</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (fs)<br>
+ <b>FileContext#checkPath should handle URIs with no port</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8309">HADOOP-8309</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)<br>
+ <b>Pseudo & Kerberos AuthenticationHandler should use getType() to create token</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8296">HADOOP-8296</a>.
+ Minor bug reported by Thomas Graves and fixed by Devaraj K <br>
+ <b>hadoop/yarn daemonlog usage wrong </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8285">HADOOP-8285</a>.
+ Major improvement reported by Sanjay Radia and fixed by Sanjay Radia (ipc)<br>
+ <b>Use ProtoBuf for RpcPayLoadHeader</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8282">HADOOP-8282</a>.
+ Minor bug reported by Devaraj K and fixed by Devaraj K (scripts)<br>
+ <b>start-all.sh refers incorrectly start-dfs.sh existence for starting start-yarn.sh</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8280">HADOOP-8280</a>.
+ Major improvement reported by Ahmed Radwan and fixed by Ahmed Radwan (test , util)<br>
+ <b> Move VersionUtil/TestVersionUtil and GenericTestUtils from HDFS into Common.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8275">HADOOP-8275</a>.
+ Minor bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe <br>
+ <b>Range check DelegationKey length </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8270">HADOOP-8270</a>.
+ Minor bug reported by Roman Shaposhnik and fixed by Roman Shaposhnik (scripts)<br>
+ <b>hadoop-daemon.sh stop action should return 0 for an already stopped service </b><br>
+ <blockquote>The daemon stop action no longer returns failure when stopping an already stopped service. </blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8264">HADOOP-8264</a>.
+ Trivial bug reported by Bernd Fondermann and fixed by Bernd Fondermann <br>
+ <b>Remove irritating double double quotes in front of hostname </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8263">HADOOP-8263</a>.
+ Minor bug reported by Todd Lipcon and fixed by Todd Lipcon (ipc)<br>
+ <b>Stringification of IPC calls not useful</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8261">HADOOP-8261</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (fs)<br>
+ <b>Har file system doesn't deal with FS URIs with a host but no port</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8251">HADOOP-8251</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Todd Lipcon (security)<br>
+ <b>SecurityUtil.fetchServiceTicket broken after HADOOP-6941</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8243">HADOOP-8243</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (ha , security)<br>
+ <b>Security support broken in CLI (manual) failover controller</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8238">HADOOP-8238</a>.
+ Major bug reported by Eli Collins and fixed by Eli Collins <br>
+ <b>NetUtils#getHostNameOfIP blows up if given ip:port string w/o port</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8236">HADOOP-8236</a>.
+ Major improvement reported by Philip Zeyliger and fixed by Todd Lipcon (ha)<br>
+ <b>haadmin should have configurable timeouts for failover commands</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8218">HADOOP-8218</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (ipc , test)<br>
+ <b>RPC.closeProxy shouldn't throw error when closing a mock</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8214">HADOOP-8214</a>.
+ Major improvement reported by Roman Shaposhnik and fixed by Roman Shaposhnik (scripts)<br>
+ <b>make hadoop script recognize a full set of deprecated commands</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8211">HADOOP-8211</a>.
+ Major sub-task reported by Eli Collins and fixed by Eli Collins (io , performance)<br>
+ <b>Update commons-net version to 3.1</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8210">HADOOP-8210</a>.
+ Major sub-task reported by Eli Collins and fixed by Eli Collins (io , performance)<br>
+ <b>Common side of HDFS-3148</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8206">HADOOP-8206</a>.
+ Major new feature reported by Todd Lipcon and fixed by Todd Lipcon (ha)<br>
+ <b>Common portion of ZK-based failover controller</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8204">HADOOP-8204</a>.
+ Major bug reported by Tom White and fixed by Todd Lipcon <br>
+ <b>TestHealthMonitor fails occasionally </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8202">HADOOP-8202</a>.
+ Minor bug reported by Hari Mankude and fixed by Hari Mankude (ipc)<br>
+ <b>stopproxy() is not closing the proxies correctly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8200">HADOOP-8200</a>.
+ Minor improvement reported by Eli Collins and fixed by Eli Collins (conf)<br>
+ <b>Remove HADOOP_[JOBTRACKER|TASKTRACKER]_OPTS </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8199">HADOOP-8199</a>.
+ Major bug reported by Nishan Shetty and fixed by Devaraj K <br>
+ <b>Fix issues in start-all.sh and stop-all.sh</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8193">HADOOP-8193</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (ha)<br>
+ <b>Refactor FailoverController/HAAdmin code to add an abstract class for "target" services</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8191">HADOOP-8191</a>.
+ Major bug reported by Philip Zeyliger and fixed by Todd Lipcon (ha)<br>
+ <b>SshFenceByTcpPort uses netcat incorrectly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8189">HADOOP-8189</a>.
+ Major bug reported by Jonathan Natkins and fixed by Jonathan Natkins (security)<br>
+ <b>LdapGroupsMapping shouldn't throw away IOException</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8185">HADOOP-8185</a>.
+ Major improvement reported by Arpit Gupta and fixed by Arpit Gupta (documentation)<br>
+ <b>Update namenode -format documentation and add -nonInteractive and -force</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8184">HADOOP-8184</a>.
+ Major improvement reported by Sanjay Radia and fixed by Sanjay Radia (ipc)<br>
+ <b>ProtoBuf RPC engine does not need it own reply packet - it can use the IPC layer reply packet.</b><br>
+ <blockquote>This change will affect the output of errors for some Hadoop CLI commands. Specifically, the name of the exception class will no longer appear, and instead only the text of the exception message will appear.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8183">HADOOP-8183</a>.
+ Minor improvement reported by Harsh J and fixed by Harsh J (util)<br>
+ <b>Stop using "mapred.used.genericoptionsparser" to avoid unnecessary warnings</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8169">HADOOP-8169</a>.
+ Critical bug reported by Thomas Graves and fixed by Thomas Graves (build)<br>
+ <b>javadoc generation fails with java.lang.OutOfMemoryError: Java heap space</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8164">HADOOP-8164</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Daryn Sharp (fs)<br>
+ <b>Handle paths using back slash as path separator for windows only</b><br>
+ <blockquote>This jira only allows providing paths using back slash as separator on Windows. The back slash on *nix system will be used as escape character. The support for paths using back slash as path separator will be removed in HADOOP-8139 in release 23.3.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8163">HADOOP-8163</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (ha)<br>
+ <b>Improve ActiveStandbyElector to provide hooks for fencing old active</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8159">HADOOP-8159</a>.
+ Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe <br>
+ <b>NetworkTopology: getLeaf should check for invalid topologies</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8154">HADOOP-8154</a>.
+ Major bug reported by Eli Collins and fixed by Eli Collins (conf)<br>
+ <b>DNS#getIPs shouldn't silently return the local host IP for bogus interface names</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8152">HADOOP-8152</a>.
+ Major improvement reported by Aaron T. Myers and fixed by Aaron T. Myers (security)<br>
+ <b>Expand public APIs for security library classes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8149">HADOOP-8149</a>.
+ Major improvement reported by Patrick Hunt and fixed by Patrick Hunt (conf)<br>
+ <b>cap space usage of default log4j rolling policy </b><br>
+ <blockquote>Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings of hadoop.root.logger and hadoop.security.logger can use their current log4j.properties files and update the HADOOP_ROOT_LOGGER and HADOOP_SECURITY_LOGGER environment variables to use DRFA and DRFAS respectively.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8142">HADOOP-8142</a>.
+ Major task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (build)<br>
+ <b>Update versions from 0.23.2 to 0.23.3</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8141">HADOOP-8141</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon (security)<br>
+ <b>Add method to init krb5 cipher suites</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8121">HADOOP-8121</a>.
+ Major new feature reported by Jonathan Natkins and fixed by Jonathan Natkins (security)<br>
+ <b>Active Directory Group Mapping Service</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8119">HADOOP-8119</a>.
+ Minor bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (test)<br>
+ <b>Fix javac warnings in TestAuthenticationFilter</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8118">HADOOP-8118</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (metrics)<br>
+ <b>Print the stack trace of InstanceAlreadyExistsException in trace level</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8117">HADOOP-8117</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon (build , test)<br>
+ <b>Upgrade test build to Surefire 2.12</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8113">HADOOP-8113</a>.
+ Trivial improvement reported by Eugene Koontz and fixed by Eugene Koontz (documentation)<br>
+ <b>Correction to BUILDING.txt: HDFS needs ProtocolBuffer, too (not just MapReduce)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8098">HADOOP-8098</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)<br>
+ <b>KerberosAuthenticatorHandler should use _HOST replacement to resolve principal name</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8086">HADOOP-8086</a>.
+ Minor improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)<br>
+ <b>KerberosName silently sets defaultRealm to "" if the Kerberos config is not found, it should log a WARN</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8084">HADOOP-8084</a>.
+ Major improvement reported by Devaraj Das and fixed by Devaraj Das (ipc)<br>
+ <b>Protobuf RPC engine can be optimized to not do copying for the RPC request/response</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8077">HADOOP-8077</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (ha)<br>
+ <b>HA: fencing method should be able to be configured on a per-NN or per-NS basis</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8070">HADOOP-8070</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (benchmarks , ipc)<br>
+ <b>Add standalone benchmark of protobuf IPC</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8007">HADOOP-8007</a>.
+ Major improvement reported by Aaron T. Myers and fixed by Todd Lipcon (ha)<br>
+ <b>HA: use substitution token for fencing argument</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7994">HADOOP-7994</a>.
+ Major sub-task reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Remove getProtocolVersion and getProtocolSignature from the client side translator and server side implementation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7968">HADOOP-7968</a>.
+ Minor bug reported by Todd Lipcon and fixed by Sho Shimauchi (ipc)<br>
+ <b>Errant println left in RPC.getHighestSupportedProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7965">HADOOP-7965</a>.
+ Major sub-task reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey (ipc)<br>
+ <b>Support for protocol version and signature in PB</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7957">HADOOP-7957</a>.
+ Major improvement reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Classes deriving GetGroupsBase should be able to override proxy creation.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7940">HADOOP-7940</a>.
+ Major bug reported by Aaron, and fixed by Csaba Miklos (io)<br>
+ <b>method clear() in org.apache.hadoop.io.Text does not work</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7931">HADOOP-7931</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (ipc)<br>
+ <b>o.a.h.ipc.WritableRpcEngine should have a way to force initialization</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7920">HADOOP-7920</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas (ipc)<br>
+ <b>Remove Avro RPC</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7913">HADOOP-7913</a>.
+ Major sub-task reported by Sanjay Radia and fixed by Sanjay Radia (ipc)<br>
+ <b>Fix bug in ProtoBufRpcEngine - </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7900">HADOOP-7900</a>.
+ Major bug reported by Ravi Gummadi and fixed by Ravi Gummadi (fs)<br>
+ <b>LocalDirAllocator confChanged() accesses conf.get() twice</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7899">HADOOP-7899</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Generate proto java files as part of the build</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7897">HADOOP-7897</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas (ipc)<br>
+ <b>ProtobufRPCEngine client side exception mechanism is not consistent with WritableRpcEngine</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7892">HADOOP-7892</a>.
+ Trivial bug reported by Todd Lipcon and fixed by Todd Lipcon (ipc)<br>
+ <b>IPC logs too verbose after "RpcKind" introduction</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7888">HADOOP-7888</a>.
+ Major bug reported by Jason Lowe and fixed by Jason Lowe (test)<br>
+ <b>TestFailoverProxy fails intermittently on trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7876">HADOOP-7876</a>.
+ Major new feature reported by Suresh Srinivas and fixed by Suresh Srinivas (ipc)<br>
+ <b>Allow access to BlockKey/DelegationKey endoded key for RPC over protobuf</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7875">HADOOP-7875</a>.
+ Major improvement reported by Suresh Srinivas and fixed by Suresh Srinivas (ipc)<br>
+ <b>Add helper class to unwrap RemoteException from ServiceException thrown on protobuf based RPC</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7862">HADOOP-7862</a>.
+ Major sub-task reported by Sanjay Radia and fixed by Sanjay Radia (ipc)<br>
+ <b>Move the support for multiple protocols to lower layer so that Writable, PB and Avro can all use it</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7833">HADOOP-7833</a>.
+ Major bug reported by John Lee and fixed by John Lee (ipc)<br>
+ <b>Inner classes of org.apache.hadoop.ipc.protobuf.HadoopRpcProtos generates findbugs warnings which results in -1 for findbugs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7806">HADOOP-7806</a>.
+ Major new feature reported by Harsh J and fixed by Harsh J (util)<br>
+ <b>Support binding to sub-interfaces</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7788">HADOOP-7788</a>.
+ Major new feature reported by Todd Lipcon and fixed by Todd Lipcon (ha)<br>
+ <b>HA: Simple HealthMonitor class to watch an HAService</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7776">HADOOP-7776</a>.
+ Major sub-task reported by Sanjay Radia and fixed by Sanjay Radia (ipc)<br>
+ <b>Make the Ipc-Header in a RPC-Payload an explicit header</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7773">HADOOP-7773</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (ipc)<br>
+ <b>Add support for protocol buffer based RPC engine</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7729">HADOOP-7729</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (ipc)<br>
+ <b>Send back valid HTTP response if user hits IPC port with HTTP GET</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7717">HADOOP-7717</a>.
+ Major improvement reported by Aaron T. Myers and fixed by Aaron T. Myers (ipc)<br>
+ <b>Move handling of concurrent client fail-overs to RetryInvocationHandler</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7716">HADOOP-7716</a>.
+ Minor improvement reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>RPC protocol registration on SS does not log the protocol name (only the class which may be different)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7695">HADOOP-7695</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (ipc)<br>
+ <b>RPC.stopProxy can throw unintended exception while logging error</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7693">HADOOP-7693</a>.
+ Major improvement reported by Doug Cutting and fixed by Doug Cutting (ipc)<br>
+ <b>fix RPC.Server#addProtocol to work in AvroRpcEngine</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7687">HADOOP-7687</a>.
+ Minor improvement reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>Make getProtocolSignature public </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7635">HADOOP-7635</a>.
+ Major improvement reported by Aaron T. Myers and fixed by Aaron T. Myers (ipc)<br>
+ <b>RetryInvocationHandler should release underlying resources on close</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7621">HADOOP-7621</a>.
+ Critical bug reported by Alejandro Abdelnur and fixed by Aaron T. Myers (security)<br>
+ <b>alfredo config should be in a file not readable by users</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7607">HADOOP-7607</a>.
+ Major improvement reported by Aaron T. Myers and fixed by Aaron T. Myers (ipc)<br>
+ <b>Simplify the RPC proxy cleanup process</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7557">HADOOP-7557</a>.
+ Major sub-task reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>Make IPC header be extensible</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7549">HADOOP-7549</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (fs)<br>
+ <b>Use JDK ServiceLoader mechanism to find FileSystem implementations</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7524">HADOOP-7524</a>.
+ Major sub-task reported by Sanjay Radia and fixed by Sanjay Radia (ipc)<br>
+ <b>Change RPC to allow multiple protocols including multiple versions of the same protocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7454">HADOOP-7454</a>.
+ Major new feature reported by Aaron T. Myers and fixed by <br>
+ <b>Common side of High Availability Framework (HDFS-1623)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7358">HADOOP-7358</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (ipc)<br>
+ <b>Improve log levels when exceptions caught in RPC handler</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7350">HADOOP-7350</a>.
+ Major improvement reported by Tom White and fixed by Tom White (conf , io)<br>
+ <b>Use ServiceLoader to discover compression codec classes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7069">HADOOP-7069</a>.
+ Major improvement reported by Jakob Homan and fixed by (documentation)<br>
+ <b>Replace forrest with supported framework</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7030">HADOOP-7030</a>.
+ Major new feature reported by Patrick Angeles and fixed by Tom White <br>
+ <b>Add TableMapping topology implementation to read host to rack mapping from a file</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6941">HADOOP-6941</a>.
+ Major bug reported by Stephen Watt and fixed by Devaraj Das <br>
+ <b>Support non-SUN JREs in UserGroupInformation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6924">HADOOP-6924</a>.
+ Major bug reported by Stephen Watt and fixed by Devaraj Das <br>
+ <b>Build fails with non-Sun JREs due to different pathing to the operating system architecture shared libraries</b><br>
+ <blockquote></blockquote></li>
+</ul>
+</body></html>
+<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
+<title>Hadoop 0.23.2 Release Notes</title>
+<STYLE type="text/css">
+ H1 {font-family: sans-serif}
+ H2 {font-family: sans-serif; margin-left: 7mm}
+ TABLE {margin-left: 7mm}
+</STYLE>
+</head>
+<body>
+<h1>Hadoop 0.23.2 Release Notes</h1>
+These release notes include new developer and user-facing incompatibilities, features, and major improvements.
+<a name="changes"/>
+<h2>Changes since Hadoop 0.23.1</h2>
+<ul>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4043">MAPREDUCE-4043</a>.
+ Blocker bug reported by Jason Lowe and fixed by Jason Lowe (mrv2 , security)<br>
+ <b>Secret keys set in Credentials are not seen by tasks</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4034">MAPREDUCE-4034</a>.
+ Blocker bug reported by Jason Lowe and fixed by Jason Lowe (mrv2)<br>
+ <b>Unable to view task logs on history server with mapreduce.job.acl-view-job=*</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4025">MAPREDUCE-4025</a>.
+ Blocker bug reported by Jason Lowe and fixed by Jason Lowe (mr-am , mrv2)<br>
+ <b>AM can crash if task attempt reports bogus progress value</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4006">MAPREDUCE-4006</a>.
+ Major bug reported by Jason Lowe and fixed by Siddharth Seth (jobhistoryserver , mrv2)<br>
+ <b>history server container log web UI sometimes combines stderr/stdout/syslog contents together</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-4005">MAPREDUCE-4005</a>.
+ Major bug reported by Jason Lowe and fixed by Jason Lowe (mrv2)<br>
+ <b>AM container logs URL is broken for completed apps when log aggregation is enabled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3982">MAPREDUCE-3982</a>.
+ Critical bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>TestEmptyJob fails with FileNotFound</b><br>
+ <blockquote>Fixed FileOutputCommitter to not err out for an 'empty-job' whose tasks don't write any outputs.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3977">MAPREDUCE-3977</a>.
+ Critical bug reported by Jason Lowe and fixed by Jason Lowe (mrv2 , nodemanager)<br>
+ <b>LogAggregationService leaks log aggregator objects</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3976">MAPREDUCE-3976</a>.
+ Major bug reported by Bikas Saha and fixed by Jason Lowe (mrv2)<br>
+ <b>TestRMContainerAllocator failing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3975">MAPREDUCE-3975</a>.
+ Blocker bug reported by Eric Payne and fixed by Eric Payne (mrv2)<br>
+ <b>Default value not set for Configuration parameter mapreduce.job.local.dir</b><br>
+ <blockquote>Exporting mapreduce.job.local.dir for mapreduce tasks to use as job-level shared scratch space.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3964">MAPREDUCE-3964</a>.
+ Critical bug reported by Jason Lowe and fixed by Jason Lowe (mrv2 , resourcemanager)<br>
+ <b>ResourceManager does not have JVM metrics</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3961">MAPREDUCE-3961</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>Map/ReduceSlotMillis computation incorrect</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3960">MAPREDUCE-3960</a>.
+ Critical bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>web proxy doesn't forward request to AM with configured hostname/IP</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3954">MAPREDUCE-3954</a>.
+ Blocker bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>Clean up passing HEAPSIZE to yarn and mapred commands.</b><br>
+ <blockquote>Added new envs to separate heap size for different daemons started via bin scripts.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3944">MAPREDUCE-3944</a>.
+ Blocker sub-task reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>JobHistory web services are slower then the UI and can easly overload the JH</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3931">MAPREDUCE-3931</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR tasks failing due to changing timestamps on Resources to download</b><br>
+ <blockquote>Changed PB implementation of LocalResource to take locks so that race conditions don't fail tasks by inadvertantly changing the timestamps.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3930">MAPREDUCE-3930</a>.
+ Critical bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>The AM page for a Reducer that has not been launched causes an NPE</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3929">MAPREDUCE-3929</a>.
+ Major bug reported by John George and fixed by John George (mrv2)<br>
+ <b>output of mapred -showacl is not clear</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3922">MAPREDUCE-3922</a>.
+ Minor improvement reported by Eugene Koontz and fixed by Hitesh Shah (build , mrv2)<br>
+ <b>Fix the potential problem compiling 32 bit binaries on a x86_64 host.</b><br>
+ <blockquote>Fixed build to not compile 32bit container-executor binary by default on all platforms.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3920">MAPREDUCE-3920</a>.
+ Major bug reported by Dave Thompson and fixed by Dave Thompson (nodemanager , resourcemanager)<br>
+ <b>Revise yarn default port number selection</b><br>
+ <blockquote>port number changes for resourcemanager and nodemanager</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3918">MAPREDUCE-3918</a>.
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>proc_historyserver no longer in command line arguments for HistoryServer</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3913">MAPREDUCE-3913</a>.
+ Critical bug reported by Jason Lowe and fixed by Jason Lowe (mrv2 , webapps)<br>
+ <b>RM application webpage is unresponsive after 2000 jobs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3910">MAPREDUCE-3910</a>.
+ Blocker bug reported by John George and fixed by John George (mrv2)<br>
+ <b>user not allowed to submit jobs even though queue -showacls shows it allows</b><br>
+ <blockquote>Fixed a bug in CapacityScheduler LeafQueue which was causing app-submission to fail.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3904">MAPREDUCE-3904</a>.
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>[NPE] Job history produced with mapreduce.cluster.acls.enabled false can not be viewed with mapreduce.cluster.acls.enabled true</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3903">MAPREDUCE-3903</a>.
+ Critical bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>no admin override to view jobs on mr app master and job history server</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3901">MAPREDUCE-3901</a>.
+ Major improvement reported by Siddharth Seth and fixed by Siddharth Seth (jobhistoryserver , mrv2)<br>
+ <b>lazy load JobHistory Task and TaskAttempt details</b><br>
+ <blockquote>Modified JobHistory records in YARN to lazily load job and task reports so as to improve UI response times.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3897">MAPREDUCE-3897</a>.
+ Critical bug reported by Thomas Graves and fixed by Eric Payne (mrv2)<br>
+ <b>capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3896">MAPREDUCE-3896</a>.
+ Blocker bug reported by John George and fixed by Vinod Kumar Vavilapalli (jobhistoryserver , mrv2)<br>
+ <b>pig job through oozie hangs </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3884">MAPREDUCE-3884</a>.
+ Critical bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (mrv2)<br>
+ <b>PWD should be first in the classpath of MR tasks</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3878">MAPREDUCE-3878</a>.
+ Critical bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>Null user on filtered jobhistory job page</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3877">MAPREDUCE-3877</a>.
+ Minor test reported by Steve Loughran and fixed by Steve Loughran (mrv2)<br>
+ <b>Add a test to formalise the current state transitions of the yarn lifecycle</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3866">MAPREDUCE-3866</a>.
+ Minor bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>bin/yarn prints the command line unnecessarily</b><br>
+ <blockquote>Fixed the bin/yarn script to not print the command line unnecessarily.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3864">MAPREDUCE-3864</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (documentation , security)<br>
+ <b>Fix cluster setup docs for correct SNN HTTPS parameters</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3862">MAPREDUCE-3862</a>.
+ Major bug reported by Jason Lowe and fixed by Jason Lowe (mrv2 , nodemanager)<br>
+ <b>Nodemanager can appear to hang on shutdown due to lingering DeletionService threads</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3852">MAPREDUCE-3852</a>.
+ Blocker bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>test TestLinuxResourceCalculatorPlugin failing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3849">MAPREDUCE-3849</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (security)<br>
+ <b>Change TokenCache's reading of the binary token file</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3816">MAPREDUCE-3816</a>.
+ Critical bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>capacity scheduler web ui bar graphs for used capacity wrong</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3798">MAPREDUCE-3798</a>.
+ Major test reported by Ravi Prakash and fixed by Ravi Prakash (test)<br>
+ <b>TestJobCleanup testCustomCleanup is failing</b><br>
+ <blockquote>Fixed failing TestJobCleanup.testCusomCleanup() and moved it to the maven build.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3792">MAPREDUCE-3792</a>.
+ Critical bug reported by Ramya Sunil and fixed by Jason Lowe (mrv2)<br>
+ <b>job -list displays only the jobs submitted by a particular user</b><br>
+ <blockquote>Fix "bin/mapred job -list" to display all jobs instead of only the jobs owned by the user. </blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3790">MAPREDUCE-3790</a>.
+ Major bug reported by Jason Lowe and fixed by Jason Lowe (contrib/streaming , mrv2)<br>
+ <b>Broken pipe on streaming job can lead to truncated output for a successful job</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3738">MAPREDUCE-3738</a>.
+ Critical bug reported by Jason Lowe and fixed by Jason Lowe (mrv2 , nodemanager)<br>
+ <b>NM can hang during shutdown if AppLogAggregatorImpl thread dies unexpectedly</b><br>
+ <blockquote>Committed to trunk and branch-0.23. Thanks Jason.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3730">MAPREDUCE-3730</a>.
+ Minor improvement reported by Jason Lowe and fixed by Jason Lowe (mrv2 , resourcemanager)<br>
+ <b>Allow restarted NM to rejoin cluster before RM expires it</b><br>
+ <blockquote>Modified RM to allow restarted NMs to be able to join the cluster without waiting for expiry.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3706">MAPREDUCE-3706</a>.
+ Critical bug reported by Thomas Graves and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>HTTP Circular redirect error on the job attempts page</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3687">MAPREDUCE-3687</a>.
+ Major bug reported by David Capwell and fixed by Ravi Prakash (mrv2)<br>
+ <b>If AM dies before it returns new tracking URL, proxy redirects to http://N/A/ and doesn't return error code</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3686">MAPREDUCE-3686</a>.
+ Critical bug reported by Thomas Graves and fixed by Bhallamudi Venkata Siva Kamesh (mrv2)<br>
+ <b>history server web ui - job counter values for map/reduce not shown properly</b><br>
+ <blockquote>Fixed two bugs in Counters because of which web app displays zero counter values for framework counters.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3680">MAPREDUCE-3680</a>.
+ Major bug reported by Thomas Graves and fixed by (mrv2)<br>
+ <b>FifoScheduler web service rest API can print out invalid JSON</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3634">MAPREDUCE-3634</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>All daemons should crash instead of hanging around when their EventHandlers get exceptions</b><br>
+ <blockquote>Fixed all daemons to crash instead of hanging around when their EventHandlers get exceptions.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3614">MAPREDUCE-3614</a>.
+ Major bug reported by Ravi Prakash and fixed by Ravi Prakash (mrv2)<br>
+ <b> finalState UNDEFINED if AM is killed by hand</b><br>
+ <blockquote>Fixed MR AM to close history file quickly and send a correct final state to the RM when it is killed.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3583">MAPREDUCE-3583</a>.
+ Critical bug reported by Ted Yu and fixed by Ted Yu <br>
+ <b>ProcfsBasedProcessTree#constructProcessInfo() may throw NumberFormatException</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3497">MAPREDUCE-3497</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves (documentation , mrv2)<br>
+ <b>missing documentation for yarn cli and subcommands - similar to commands_manual.html</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3034">MAPREDUCE-3034</a>.
+ Critical bug reported by Vinod Kumar Vavilapalli and fixed by Devaraj K (mrv2 , nodemanager)<br>
+ <b>NM should act on a REBOOT command from RM</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3009">MAPREDUCE-3009</a>.
+ Major bug reported by chackaravarthy and fixed by chackaravarthy (jobhistoryserver , mrv2)<br>
+ <b>RM UI -> Applications -> Application(Job History) -> Map Tasks -> Task ID -> Node link is not working</b><br>
+ <blockquote>Fixed node link on JobHistory webapp.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2855">MAPREDUCE-2855</a>.
+ Major bug reported by Todd Lipcon and fixed by Siddharth Seth <br>
+ <b>ResourceBundle lookup during counter name resolution takes a lot of time</b><br>
+ <blockquote>Passing a cached class-loader to ResourceBundle creator to minimize counter names lookup time.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2793">MAPREDUCE-2793</a>.
+ Critical bug reported by Ramya Sunil and fixed by Bikas Saha (mrv2)<br>
+ <b>[MR-279] Maintain consistency in naming appIDs, jobIDs and attemptIDs </b><br>
+ <blockquote>Corrected AppIDs, JobIDs, TaskAttemptIDs to be of correct format on the web pages.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3853">HDFS-3853</a>.
+ Minor bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (name-node)<br>
+ <b>Port MiniDFSCluster enableManagedDfsDirsRedundancy option to branch-2</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3104">HDFS-3104</a>.
+ Major test reported by Daryn Sharp and fixed by Daryn Sharp (test)<br>
+ <b>Add tests for mkdir -p</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3101">HDFS-3101</a>.
+ Major bug reported by Zhanwei.Wang and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>cannot read empty file using webhdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3098">HDFS-3098</a>.
+ Major test reported by Daryn Sharp and fixed by Daryn Sharp (test)<br>
+ <b>Update FsShell tests for quoted metachars</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3060">HDFS-3060</a>.
+ Minor test reported by Eli Collins and fixed by Eli Collins (test)<br>
+ <b>Bump TestDistributedUpgrade#testDistributedUpgrade timeout</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3012">HDFS-3012</a>.
+ Critical bug reported by Ramya Sunil and fixed by Robert Joseph Evans <br>
+ <b>Exception while renewing delegation token</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3008">HDFS-3008</a>.
+ Major bug reported by Eli Collins and fixed by Eli Collins (hdfs client)<br>
+ <b>Negative caching of local addrs doesn't work</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-3006">HDFS-3006</a>.
+ Major bug reported by bc Wong and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>Webhdfs "SETOWNER" call returns incorrect content-type</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2985">HDFS-2985</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Improve logging when replicas are marked as corrupt</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2981">HDFS-2981</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE <br>
+ <b>The default value of dfs.client.block.write.replace-datanode-on-failure.enable should be true</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2969">HDFS-2969</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>ExtendedBlock.equals is incorrectly implemented</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2950">HDFS-2950</a>.
+ Minor bug reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Secondary NN HTTPS address should be listed as a NAMESERVICE_SPECIFIC_KEY</b><br>
+ <blockquote>The configuration dfs.secondary.https.port has been renamed to dfs.namenode.secondary.https-port for consistency. The old configuration is still supported via a deprecation path.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2944">HDFS-2944</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (hdfs client)<br>
+ <b>Typo in hdfs-default.xml causes dfs.client.block.write.replace-datanode-on-failure.enable to be mistakenly disabled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2943">HDFS-2943</a>.
+ Major new feature reported by Aaron T. Myers and fixed by Aaron T. Myers (name-node)<br>
+ <b>Expose last checkpoint time and transaction stats as JMX metrics</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2938">HDFS-2938</a>.
+ Major bug reported by Suresh Srinivas and fixed by Hari Mankude (name-node)<br>
+ <b>Recursive delete of a large directory makes namenode unresponsive</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2931">HDFS-2931</a>.
+ Minor task reported by Harsh J and fixed by Harsh J (data-node)<br>
+ <b>Switch the DataNode's BlockVolumeChoosingPolicy to be a private-audience interface</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2907">HDFS-2907</a>.
+ Minor improvement reported by Sanjay Radia and fixed by Tsz Wo (Nicholas), SZE <br>
+ <b>Make FSDataset in Datanode Pluggable</b><br>
+ <blockquote>Add a private conf property dfs.datanode.fsdataset.factory to make FSDataset in Datanode pluggable.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2887">HDFS-2887</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Define a FSVolume interface</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2764">HDFS-2764</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (name-node , test)<br>
+ <b>TestBackupNode is racy</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2725">HDFS-2725</a>.
+ Major bug reported by Prashant Sharma and fixed by (hdfs client)<br>
+ <b>hdfs script usage information is missing the information about "dfs" command</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2506">HDFS-2506</a>.
+ Major improvement reported by Suresh Srinivas and fixed by Suresh Srinivas (data-node , name-node)<br>
+ <b>Umbrella jira for tracking separation of wire protocol datatypes from the implementation types</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1217">HDFS-1217</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Laxman (name-node)<br>
+ <b>Some methods in the NameNdoe should not be public</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-776">HDFS-776</a>.
+ Critical bug reported by Owen O'Malley and fixed by Uma Maheswara Rao G (balancer)<br>
+ <b>Fix exception handling in Balancer</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8176">HADOOP-8176</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Disambiguate the destination of FsShell copies</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8175">HADOOP-8175</a>.
+ Major sub-task reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Add mkdir -p flag</b><br>
+ <blockquote>FsShell mkdir now accepts a -p flag. Like unix, mkdir -p will not fail if the directory already exists. Unlike unix, intermediate directories are always created, regardless of the flag, to avoid incompatibilities at this time.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8173">HADOOP-8173</a>.
+ Major sub-task reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>FsShell needs to handle quoted metachars</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8157">HADOOP-8157</a>.
+ Major test reported by Eli Collins and fixed by Todd Lipcon <br>
+ <b>TestRPCCallBenchmark#testBenchmarkWithWritable fails with RTE</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8146">HADOOP-8146</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>FsShell commands cannot be interrupted</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8140">HADOOP-8140</a>.
+ Major bug reported by arkady borkovsky and fixed by Daryn Sharp <br>
+ <b>dfs -getmerge should process its argments better </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8137">HADOOP-8137</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Thomas Graves (documentation)<br>
+ <b>Site side links for commands manual (MAPREDUCE-3497)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8131">HADOOP-8131</a>.
+ Critical bug reported by Daryn Sharp and fixed by Daryn Sharp <br>
+ <b>FsShell put doesn't correctly handle a non-existent dir</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8123">HADOOP-8123</a>.
+ Critical bug reported by Jonathan Eagles and fixed by Jonathan Eagles (build)<br>
+ <b>hadoop-project invalid pom warnings prevent transitive dependency resolution</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8083">HADOOP-8083</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>javadoc generation for some modules is not done under target/</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8082">HADOOP-8082</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>add hadoop-client and hadoop-minicluster to the dependency-management section</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8074">HADOOP-8074</a>.
+ Trivial bug reported by Eli Collins and fixed by Colin Patrick McCabe (scripts)<br>
+ <b>Small bug in hadoop error message for unknown commands</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8071">HADOOP-8071</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (ipc)<br>
+ <b>Avoid an extra packet in client code when nagling is disabled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8066">HADOOP-8066</a>.
+ Major bug reported by Aaron T. Myers and fixed by Andrew Bayer (build)<br>
+ <b>The full docs build intermittently fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8064">HADOOP-8064</a>.
+ Major bug reported by Kihwal Lee and fixed by Kihwal Lee (build)<br>
+ <b>Remove unnecessary dependency on w3c.org in document processing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8057">HADOOP-8057</a>.
+ Major bug reported by Vinay and fixed by Vinay (scripts)<br>
+ <b>hadoop-setup-conf.sh not working because of some extra spaces.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8051">HADOOP-8051</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (documentation)<br>
+ <b>HttpFS documentation it is not wired to the generated site</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8050">HADOOP-8050</a>.
+ Major bug reported by Kihwal Lee and fixed by Kihwal Lee (metrics)<br>
+ <b>Deadlock in metrics</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8048">HADOOP-8048</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (util)<br>
+ <b>Allow merging of Credentials</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8046">HADOOP-8046</a>.
+ Minor bug reported by Steve Loughran and fixed by Steve Loughran <br>
+ <b>Revert StaticMapping semantics to the existing ones, add DNS mapping diagnostics in progress</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8042">HADOOP-8042</a>.
+ Critical bug reported by Kevin J. Price and fixed by Daryn Sharp (fs)<br>
+ <b>When copying a file out of HDFS, modifying it, and uploading it back into HDFS, the put fails due to a CRC mismatch</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8036">HADOOP-8036</a>.
+ Major bug reported by Eli Collins and fixed by Colin Patrick McCabe (fs , test)<br>
+ <b>TestViewFsTrash assumes the user's home directory is 2 levels deep</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8035">HADOOP-8035</a>.
+ Minor bug reported by Andrew Bayer and fixed by Andrew Bayer (build)<br>
+ <b>Hadoop Maven site is inefficient and runs phases redundantly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8032">HADOOP-8032</a>.
+ Major wish reported by Ravi Prakash and fixed by Ravi Prakash (build , documentation)<br>
+ <b>mvn site:stage-deploy should be able to use the scp protocol to stage documents</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6502">HADOOP-6502</a>.
+ Critical bug reported by Hairong Kuang and fixed by Sharad Agarwal (util)<br>
+ <b>DistributedFileSystem#listStatus is very slow when listing a directory with a size of 1300</b><br>
+ <blockquote></blockquote></li>
+</ul>
+</body></html>
+<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
+<title>Hadoop 0.23.1 Release Notes</title>
+<STYLE type="text/css">
+ H1 {font-family: sans-serif}
+ H2 {font-family: sans-serif; margin-left: 7mm}
+ TABLE {margin-left: 7mm}
+</STYLE>
+</head>
+<body>
+<h1>Hadoop 0.23.1 Release Notes</h1>
+These release notes include new developer and user-facing incompatibilities, features, and major improvements.
+<a name="changes"/>
+<h2>Changes since Hadoop 0.23.0</h2>
+<ul>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3858">MAPREDUCE-3858</a>.
+ Critical bug reported by Tom White and fixed by Tom White (mrv2)<br>
+ <b>Task attempt failure during commit results in task never completing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3856">MAPREDUCE-3856</a>.
+ Critical bug reported by Eric Payne and fixed by Eric Payne (mrv2)<br>
+ <b>Instances of RunningJob class givs incorrect job tracking urls when mutiple jobs are submitted from same client jvm.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3854">MAPREDUCE-3854</a>.
+ Major test reported by Tom White and fixed by Tom White (mrv2)<br>
+ <b>Reinstate environment variable tests in TestMiniMRChildTask</b><br>
+ <blockquote>Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3846">MAPREDUCE-3846</a>.
+ Critical sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>Restarted+Recovered AM hangs in some corner cases</b><br>
+ <blockquote>Addressed MR AM hanging issues during AM restart and then the recovery.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3843">MAPREDUCE-3843</a>.
+ Critical bug reported by Anupam Seth and fixed by Anupam Seth (jobhistoryserver , mrv2)<br>
+ <b>Job summary log file found missing on the RM host</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3840">MAPREDUCE-3840</a>.
+ Blocker bug reported by Ravi Prakash and fixed by Ravi Prakash (mrv2)<br>
+ <b>JobEndNotifier doesn't use the proxyToUse during connecting</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3834">MAPREDUCE-3834</a>.
+ Critical bug reported by Siddharth Seth and fixed by Siddharth Seth (mr-am , mrv2)<br>
+ <b>If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table</b><br>
+ <blockquote>Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3833">MAPREDUCE-3833</a>.
+ Major bug reported by Jason Lowe and fixed by Jason Lowe (mrv2)<br>
+ <b>Capacity scheduler queue refresh doesn't recompute queue capacities properly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3828">MAPREDUCE-3828</a>.
+ Major bug reported by Ahmed Radwan and fixed by Siddharth Seth (mrv2)<br>
+ <b>Broken urls: AM tracking url and jobhistory url in a single node setup.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3827">MAPREDUCE-3827</a>.
+ Blocker sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2 , performance)<br>
+ <b>Counters aggregation slowed down significantly after MAPREDUCE-3749</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3826">MAPREDUCE-3826</a>.
+ Major bug reported by Arpit Gupta and fixed by Jonathan Eagles (mrv2)<br>
+ <b>RM UI when loaded throws a message stating Data Tables warning and then the column sorting stops working</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3823">MAPREDUCE-3823</a>.
+ Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2 , performance)<br>
+ <b>Counters are getting calculated twice at job-finish and delaying clients.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3822">MAPREDUCE-3822</a>.
+ Critical bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>TestJobCounters is failing intermittently on trunk and 0.23.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3817">MAPREDUCE-3817</a>.
+ Major bug reported by Arpit Gupta and fixed by Arpit Gupta (mrv2)<br>
+ <b>bin/mapred command cannot run distcp and archive jobs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3815">MAPREDUCE-3815</a>.
+ Critical sub-task reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>Data Locality suffers if the AM asks for containers using IPs instead of hostnames</b><br>
+ <blockquote>Fixed MR AM to always use hostnames and never IPs when requesting containers so that scheduler can give off data local containers correctly.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3814">MAPREDUCE-3814</a>.
+ Major bug reported by Arun C Murthy and fixed by Arun C Murthy (mrv1 , mrv2)<br>
+ <b>MR1 compile fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3813">MAPREDUCE-3813</a>.
+ Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2 , performance)<br>
+ <b>RackResolver should maintain a cache to avoid repetitive lookups.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3811">MAPREDUCE-3811</a>.
+ Critical task reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>Make the Client-AM IPC retry count configurable</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3810">MAPREDUCE-3810</a>.
+ Blocker sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2 , performance)<br>
+ <b>MR AM's ContainerAllocator is assigning the allocated containers very slowly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3809">MAPREDUCE-3809</a>.
+ Blocker sub-task reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>Tasks may take upto 3 seconds to exit after completion</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3808">MAPREDUCE-3808</a>.
+ Blocker bug reported by Siddharth Seth and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>NPE in FileOutputCommitter when running a 0 reduce job</b><br>
+ <blockquote>Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3804">MAPREDUCE-3804</a>.
+ Major bug reported by Dave Thompson and fixed by Dave Thompson (jobhistoryserver , mrv2 , resourcemanager)<br>
+ <b>yarn webapp interface vulnerable to cross scripting attacks</b><br>
+ <blockquote>fix cross scripting attacks vulnerability through webapp interface.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3803">MAPREDUCE-3803</a>.
+ Major test reported by Ravi Prakash and fixed by Ravi Prakash (build)<br>
+ <b>HDFS-2864 broke ant compilation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3802">MAPREDUCE-3802</a>.
+ Critical sub-task reported by Robert Joseph Evans and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2)<br>
+ <b>If an MR AM dies twice it looks like the process freezes</b><br>
+ <blockquote>Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3795">MAPREDUCE-3795</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>"job -status" command line output is malformed</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3794">MAPREDUCE-3794</a>.
+ Major bug reported by Tom White and fixed by Tom White (mrv2)<br>
+ <b>Support mapred.Task.Counter and mapred.JobInProgress.Counter enums for compatibility</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3791">MAPREDUCE-3791</a>.
+ Major bug reported by Roman Shaposhnik and fixed by Mahadev konar (documentation , mrv2)<br>
+ <b>can't build site in hadoop-yarn-server-common</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3787">MAPREDUCE-3787</a>.
+ Major improvement reported by Amar Kamat and fixed by Amar Kamat (contrib/gridmix)<br>
+ <b>[Gridmix] Improve STRESS mode</b><br>
+ <blockquote>JobMonitor can now deploy multiple threads for faster job-status polling. Use 'gridmix.job-monitor.thread-count' to set the number of threads. Stress mode now relies on the updates from the job monitor instead of polling for job status. Failures in job submission now get reported to the statistics module and ultimately reported to the user via summary.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3784">MAPREDUCE-3784</a>.
+ Major bug reported by Ramya Sunil and fixed by Arun C Murthy (mrv2)<br>
+ <b>maxActiveApplications(|PerUser) per queue is too low for small clusters</b><br>
+ <blockquote>Fixed CapacityScheduler so that maxActiveApplication and maxActiveApplicationsPerUser per queue are not too low for small clusters. </blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3780">MAPREDUCE-3780</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Hitesh Shah (mrv2)<br>
+ <b>RM assigns containers to killed applications</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3775">MAPREDUCE-3775</a>.
+ Minor bug reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>Change MiniYarnCluster to escape special chars in testname</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3774">MAPREDUCE-3774</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>yarn-default.xml should be moved to hadoop-yarn-common.</b><br>
+ <blockquote>MAPREDUCE-3774. Moved yarn-default.xml to hadoop-yarn-common from hadoop-server-common.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3771">MAPREDUCE-3771</a>.
+ Major improvement reported by Arun C Murthy and fixed by Arun C Murthy <br>
+ <b>Port MAPREDUCE-1735 to trunk/0.23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3770">MAPREDUCE-3770</a>.
+ Critical bug reported by Amar Kamat and fixed by Amar Kamat (tools/rumen)<br>
+ <b>[Rumen] Zombie.getJobConf() results into NPE</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3765">MAPREDUCE-3765</a>.
+ Minor bug reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3764">MAPREDUCE-3764</a>.
+ Critical bug reported by Siddharth Seth and fixed by Arun C Murthy (mrv2)<br>
+ <b>AllocatedGB etc metrics incorrect if min-allocation-mb isn't a multiple of 1GB</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3762">MAPREDUCE-3762</a>.
+ Critical bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>Resource Manager fails to come up with default capacity scheduler configs.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3760">MAPREDUCE-3760</a>.
+ Major bug reported by Ramya Sunil and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>Blacklisted NMs should not appear in Active nodes list</b><br>
+ <blockquote>Changed active nodes list to not contain unhealthy nodes on the webUI and metrics.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3759">MAPREDUCE-3759</a>.
+ Major bug reported by Ramya Sunil and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3756">MAPREDUCE-3756</a>.
+ Major improvement reported by Arun C Murthy and fixed by Hitesh Shah (mrv2)<br>
+ <b>Make single shuffle limit configurable</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3754">MAPREDUCE-3754</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2 , webapps)<br>
+ <b>RM webapp should have pages filtered based on App-state</b><br>
+ <blockquote>Modified RM UI to filter applications based on state of the applications.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3752">MAPREDUCE-3752</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Arun C Murthy (mrv2)<br>
+ <b>Headroom should be capped by queue max-cap</b><br>
+ <blockquote>Modified application limits to include queue max-capacities besides the usual user limits.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3749">MAPREDUCE-3749</a>.
+ Blocker bug reported by Tom White and fixed by Tom White (mrv2)<br>
+ <b>ConcurrentModificationException in counter groups</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3748">MAPREDUCE-3748</a>.
+ Minor bug reported by Ramya Sunil and fixed by Ramya Sunil (mrv2)<br>
+ <b>Move CS related nodeUpdate log messages to DEBUG</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3747">MAPREDUCE-3747</a>.
+ Major bug reported by Ramya Sunil and fixed by Arun C Murthy (mrv2)<br>
+ <b>Memory Total is not refreshed until an app is launched</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3744">MAPREDUCE-3744</a>.
+ Blocker bug reported by Jason Lowe and fixed by Jason Lowe (mrv2)<br>
+ <b>Unable to retrieve application logs via "yarn logs" or "mapred job -logs"</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3742">MAPREDUCE-3742</a>.
+ Blocker bug reported by Jason Lowe and fixed by Jason Lowe (mrv2)<br>
+ <b>"yarn logs" command fails with ClassNotFoundException</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3737">MAPREDUCE-3737</a>.
+ Critical bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>The Web Application Proxy's is not documented very well</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3735">MAPREDUCE-3735</a>.
+ Blocker bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>Add distcp jar to the distribution (tar)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3733">MAPREDUCE-3733</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar <br>
+ <b>Add Apache License Header to hadoop-distcp/pom.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3732">MAPREDUCE-3732</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Arun C Murthy (mrv2 , resourcemanager , scheduler)<br>
+ <b>CS should only use 'activeUsers with pending requests' for computing user-limits</b><br>
+ <blockquote>Modified CapacityScheduler to use only users with pending requests for computing user-limits.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3727">MAPREDUCE-3727</a>.
+ Critical bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)<br>
+ <b>jobtoken location property in jobconf refers to wrong jobtoken file</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3723">MAPREDUCE-3723</a>.
+ Major bug reported by Bhallamudi Venkata Siva Kamesh and fixed by Bhallamudi Venkata Siva Kamesh (mrv2 , test , webapps)<br>
+ <b>TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3721">MAPREDUCE-3721</a>.
+ Blocker bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>Race in shuffle can cause it to hang</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3720">MAPREDUCE-3720</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (client , mrv2)<br>
+ <b>Command line listJobs should not visit each AM</b><br>
+ <blockquote>Changed bin/mapred job -list to not print job-specific information not available at RM.
+
+Very minor incompatibility in cmd-line output, inevitable due to MRv2 architecture.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3718">MAPREDUCE-3718</a>.
+ Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Hitesh Shah (mrv2 , performance)<br>
+ <b>Default AM heartbeat interval should be one second</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3717">MAPREDUCE-3717</a>.
+ Blocker bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>JobClient test jar has missing files to run all the test programs.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3716">MAPREDUCE-3716</a>.
+ Blocker bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>java.io.File.createTempFile fails in map/reduce tasks</b><br>
+ <blockquote>Fixing YARN+MR to allow MR jobs to be able to use java.io.File.createTempFile to create temporary files as part of their tasks.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3714">MAPREDUCE-3714</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2 , task)<br>
+ <b>Reduce hangs in a corner case</b><br>
+ <blockquote>Fixed EventFetcher and Fetcher threads to shut-down properly so that reducers don't hang in corner cases.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3713">MAPREDUCE-3713</a>.
+ Blocker bug reported by Siddharth Seth and fixed by Arun C Murthy (mrv2 , resourcemanager)<br>
+ <b>Incorrect headroom reported to jobs</b><br>
+ <blockquote>Fixed the way head-room is allocated to applications by CapacityScheduler so that it deducts current-usage per user and not per-application.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3712">MAPREDUCE-3712</a>.
+ Blocker bug reported by Ravi Prakash and fixed by Mahadev konar (mrv2)<br>
+ <b>The mapreduce tar does not contain the hadoop-mapreduce-client-jobclient-tests.jar. </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3711">MAPREDUCE-3711</a>.
+ Blocker sub-task reported by Siddharth Seth and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>AppMaster recovery for Medium to large jobs take long time</b><br>
+ <blockquote>Fixed MR AM recovery so that only single selected task output is recovered and thus reduce the unnecessarily bloated recovery time.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3710">MAPREDUCE-3710</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv1 , mrv2)<br>
+ <b>last split generated by FileInputFormat.getSplits may not have the best locality</b><br>
+ <blockquote>Improved FileInputFormat to return better locality for the last split.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3709">MAPREDUCE-3709</a>.
+ Major bug reported by Eli Collins and fixed by Hitesh Shah (mrv2 , test)<br>
+ <b>TestDistributedShell is failing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3708">MAPREDUCE-3708</a>.
+ Major bug reported by Bhallamudi Venkata Siva Kamesh and fixed by Bhallamudi Venkata Siva Kamesh (mrv2)<br>
+ <b>Metrics: Incorrect Apps Submitted Count</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3705">MAPREDUCE-3705</a>.
+ Blocker bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>ant build fails on 0.23 branch </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3703">MAPREDUCE-3703</a>.
+ Critical bug reported by Eric Payne and fixed by Eric Payne (mrv2 , resourcemanager)<br>
+ <b>ResourceManager should provide node lists in JMX output</b><br>
+ <blockquote>New JMX Bean in ResourceManager to provide list of live node managers:
+
+Hadoop:service=ResourceManager,name=RMNMInfo LiveNodeManagers</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3702">MAPREDUCE-3702</a>.
+ Critical bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>internal server error trying access application master via proxy with filter enabled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3701">MAPREDUCE-3701</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>Delete HadoopYarnRPC from 0.23 branch.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3699">MAPREDUCE-3699</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Hitesh Shah (mrv2)<br>
+ <b>Default RPC handlers are very low for YARN servers</b><br>
+ <blockquote>Increased RPC handlers for all YARN servers to reasonable values for working at scale.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3698">MAPREDUCE-3698</a>.
+ Blocker sub-task reported by Siddharth Seth and fixed by Mahadev konar (mrv2)<br>
+ <b>Client cannot talk to the history server in secure mode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3697">MAPREDUCE-3697</a>.
+ Blocker bug reported by John George and fixed by Mahadev konar (mrv2)<br>
+ <b>Hadoop Counters API limits Oozie's working across different hadoop versions</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3696">MAPREDUCE-3696</a>.
+ Blocker bug reported by John George and fixed by John George (mrv2)<br>
+ <b>MR job via oozie does not work on hadoop 23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3693">MAPREDUCE-3693</a>.
+ Minor improvement reported by Roman Shaposhnik and fixed by Roman Shaposhnik (mrv2)<br>
+ <b>Add admin env to mapred-default.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3692">MAPREDUCE-3692</a>.
+ Blocker improvement reported by Eli Collins and fixed by Eli Collins (mrv2)<br>
+ <b>yarn-resourcemanager out and log files can get big</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3691">MAPREDUCE-3691</a>.
+ Critical bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>webservices add support to compress response</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3689">MAPREDUCE-3689</a>.
+ Blocker bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>RM web UI doesn't handle newline in job name</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3684">MAPREDUCE-3684</a>.
+ Major bug reported by Tom White and fixed by Tom White (client)<br>
+ <b>LocalDistributedCacheManager does not shut down its thread pool</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3683">MAPREDUCE-3683</a>.
+ Blocker bug reported by Thomas Graves and fixed by Arun C Murthy (mrv2)<br>
+ <b>Capacity scheduler LeafQueues maximum capacity calculation issues</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3681">MAPREDUCE-3681</a>.
+ Critical bug reported by Thomas Graves and fixed by Arun C Murthy (mrv2)<br>
+ <b>capacity scheduler LeafQueues calculate used capacity wrong</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3679">MAPREDUCE-3679</a>.
+ Major improvement reported by Mahadev konar and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>AM logs and others should not automatically refresh after every 1 second.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3669">MAPREDUCE-3669</a>.
+ Blocker bug reported by Thomas Graves and fixed by Mahadev konar (mrv2)<br>
+ <b>Getting a lot of PriviledgedActionException / SaslException when running a job</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3664">MAPREDUCE-3664</a>.
+ Minor bug reported by praveen sripati and fixed by Brandon Li (documentation)<br>
+ <b>HDFS Federation Documentation has incorrect configuration example</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3657">MAPREDUCE-3657</a>.
+ Minor bug reported by Jason Lowe and fixed by Jason Lowe (build , mrv2)<br>
+ <b>State machine visualize build fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3656">MAPREDUCE-3656</a>.
+ Blocker bug reported by Karam Singh and fixed by Siddharth Seth (applicationmaster , mrv2 , resourcemanager)<br>
+ <b>Sort job on 350 scale is consistently failing with latest MRV2 code </b><br>
+ <blockquote>Fixed a race condition in MR AM which is failing the sort benchmark consistently.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3652">MAPREDUCE-3652</a>.
+ Blocker bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>org.apache.hadoop.mapred.TestWebUIAuthorization.testWebUIAuthorization fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3651">MAPREDUCE-3651</a>.
+ Blocker bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>TestQueueManagerRefresh fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3649">MAPREDUCE-3649</a>.
+ Blocker bug reported by Mahadev konar and fixed by Ravi Prakash (mrv2)<br>
+ <b>Job End notification gives an error on calling back.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3648">MAPREDUCE-3648</a>.
+ Blocker bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>TestJobConf failing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3646">MAPREDUCE-3646</a>.
+ Major bug reported by Ramya Sunil and fixed by Jonathan Eagles (client , mrv2)<br>
+ <b>Remove redundant URL info from "mapred job" output</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3645">MAPREDUCE-3645</a>.
+ Blocker bug reported by Thomas Graves and fixed by Thomas Graves (mrv1)<br>
+ <b>TestJobHistory fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3641">MAPREDUCE-3641</a>.
+ Blocker sub-task reported by Arun C Murthy and fixed by Arun C Murthy (mrv2 , scheduler)<br>
+ <b>CapacityScheduler should be more conservative assigning off-switch requests</b><br>
+ <blockquote>Making CapacityScheduler more conservative so as to assign only one off-switch container in a single scheduling iteration.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3640">MAPREDUCE-3640</a>.
+ Blocker sub-task reported by Siddharth Seth and fixed by Arun C Murthy (mrv2)<br>
+ <b>AMRecovery should pick completed task form partial JobHistory files</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3639">MAPREDUCE-3639</a>.
+ Blocker bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>TokenCache likely broken for FileSystems which don't issue delegation tokens</b><br>
+ <blockquote>Fixed TokenCache to work with absent FileSystem canonical service-names.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3630">MAPREDUCE-3630</a>.
+ Critical task reported by Amol Kekre and fixed by Mahadev konar (mrv2)<br>
+ <b>NullPointerException running teragen</b><br>
+ <blockquote>Committed to trunk and branch-0.23. Thanks Mahadev.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3625">MAPREDUCE-3625</a>.
+ Critical bug reported by Arun C Murthy and fixed by Jason Lowe (mrv2)<br>
+ <b>CapacityScheduler web-ui display of queue's used capacity is broken</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3624">MAPREDUCE-3624</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>bin/yarn script adds jdk tools.jar to the classpath.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3618">MAPREDUCE-3618</a>.
+ Major sub-task reported by Siddharth Seth and fixed by Siddharth Seth (mrv2 , performance)<br>
+ <b>TaskHeartbeatHandler holds a global lock for all task-updates</b><br>
+ <blockquote>Fixed TaskHeartbeatHandler to not hold a global lock for all task-updates.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3617">MAPREDUCE-3617</a>.
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>Remove yarn default values for resource manager and nodemanager principal</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3616">MAPREDUCE-3616</a>.
+ Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mr-am , mrv2 , performance)<br>
+ <b>Thread pool for launching containers in MR AM not expanding as expected</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3610">MAPREDUCE-3610</a>.
+ Minor improvement reported by Sho Shimauchi and fixed by Sho Shimauchi <br>
+ <b>Some parts in MR use old property dfs.block.size</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3608">MAPREDUCE-3608</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>MAPREDUCE-3522 commit causes compilation to fail</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3604">MAPREDUCE-3604</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Arun C Murthy (contrib/streaming)<br>
+ <b>Streaming's check for local mode is broken</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3597">MAPREDUCE-3597</a>.
+ Major improvement reported by Ravi Gummadi and fixed by Ravi Gummadi (tools/rumen)<br>
+ <b>Provide a way to access other info of history file from Rumentool</b><br>
+ <blockquote>Rumen now provides {{Parsed*}} objects. These objects provide extra information that are not provided by {{Logged*}} objects.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3596">MAPREDUCE-3596</a>.
+ Blocker bug reported by Ravi Prakash and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2)<br>
+ <b>Sort benchmark got hang after completion of 99% map phase</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3595">MAPREDUCE-3595</a>.
+ Major test reported by Tom White and fixed by Tom White (test)<br>
+ <b>Add missing TestCounters#testCounterValue test from branch 1 to 0.23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3588">MAPREDUCE-3588</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Arun C Murthy <br>
+ <b>bin/yarn broken after MAPREDUCE-3366</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3586">MAPREDUCE-3586</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mr-am , mrv2)<br>
+ <b>Lots of AMs hanging around in PIG testing</b><br>
+ <blockquote>Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3582">MAPREDUCE-3582</a>.
+ Major bug reported by Ahmed Radwan and fixed by Ahmed Radwan (mrv2 , test)<br>
+ <b>Move successfully passing MR1 tests to MR2 maven tree.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3579">MAPREDUCE-3579</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (mrv2)<br>
+ <b>ConverterUtils should not include a port in a path for a URL with no port</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3572">MAPREDUCE-3572</a>.
+ Critical sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mr-am , mrv2 , performance)<br>
+ <b>MR AM's dispatcher is blocked by heartbeats to ResourceManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3569">MAPREDUCE-3569</a>.
+ Critical sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mr-am , mrv2 , performance)<br>
+ <b>TaskAttemptListener holds a global lock for all task-updates</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3568">MAPREDUCE-3568</a>.
+ Critical sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mr-am , mrv2 , performance)<br>
+ <b>Optimize Job's progress calculations in MR AM</b><br>
+ <blockquote>Optimized Job's progress calculations in MR AM.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3567">MAPREDUCE-3567</a>.
+ Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mr-am , mrv2 , performance)<br>
+ <b>Extraneous JobConf objects in AM heap</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3566">MAPREDUCE-3566</a>.
+ Critical sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mr-am , mrv2)<br>
+ <b>MR AM slows down due to repeatedly constructing ContainerLaunchContext</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3564">MAPREDUCE-3564</a>.
+ Blocker bug reported by Mahadev konar and fixed by Siddharth Seth (mrv2)<br>
+ <b>TestStagingCleanup and TestJobEndNotifier are failing on trunk.</b><br>
+ <blockquote>Fixed failures in TestStagingCleanup and TestJobEndNotifier tests.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3563">MAPREDUCE-3563</a>.
+ Major bug reported by Arun C Murthy and fixed by Arun C Murthy (mrv2)<br>
+ <b>LocalJobRunner doesn't handle Jobs using o.a.h.mapreduce.OutputCommitter</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3560">MAPREDUCE-3560</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Siddharth Seth (mrv2 , resourcemanager , test)<br>
+ <b>TestRMNodeTransitions is failing on trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3557">MAPREDUCE-3557</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>MR1 test fail to compile because of missing hadoop-archives dependency</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3553">MAPREDUCE-3553</a>.
+ Minor sub-task reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>Add support for data returned when exceptions thrown from web service apis to be in either xml or in JSON</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3549">MAPREDUCE-3549</a>.
+ Blocker bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>write api documentation for web service apis for RM, NM, mapreduce app master, and job history server</b><br>
+ <blockquote>new files added: A hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebServicesIntro.apt.vm
+A hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/NodeManagerRest.apt.vm
+A hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRest.apt.vm
+A hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/MapredAppMasterRest.apt.vm
+A hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/HistoryServerRest.apt.vm
+
+The hadoop-project/src/site/site.xml is split into separate patch.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3548">MAPREDUCE-3548</a>.
+ Critical sub-task reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>write unit tests for web services for mapreduce app master and job history server</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3547">MAPREDUCE-3547</a>.
+ Critical sub-task reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>finish unit tests for web services for RM and NM</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3544">MAPREDUCE-3544</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build , tools/rumen)<br>
+ <b>gridmix build is broken, requires hadoop-archives to be added as ivy dependency</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3542">MAPREDUCE-3542</a>.
+ Major bug reported by Tom White and fixed by Tom White <br>
+ <b>Support "FileSystemCounter" legacy counter group name for compatibility</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3541">MAPREDUCE-3541</a>.
+ Blocker bug reported by Ravi Prakash and fixed by Ravi Prakash (mrv2)<br>
+ <b>Fix broken TestJobQueueClient test</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3537">MAPREDUCE-3537</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Arun C Murthy <br>
+ <b>DefaultContainerExecutor has a race condn. with multiple concurrent containers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3534">MAPREDUCE-3534</a>.
+ Blocker sub-task reported by Vinay Kumar Thota and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>Compression benchmark run-time increased by 13% in 0.23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3532">MAPREDUCE-3532</a>.
+ Critical bug reported by Karam Singh and fixed by Bhallamudi Venkata Siva Kamesh (mrv2 , nodemanager)<br>
+ <b>When 0 is provided as port number in yarn.nodemanager.webapp.address, NMs webserver component picks up random port, NM keeps on Reporting 0 port to RM</b><br>
+ <blockquote>Modified NM to report correct http address when an ephemeral web port is configured.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3531">MAPREDUCE-3531</a>.
+ Blocker bug reported by Karam Singh and fixed by Robert Joseph Evans (mrv2 , resourcemanager , scheduler)<br>
+ <b>Sometimes java.lang.IllegalArgumentException: Invalid key to HMAC computation in NODE_UPDATE also causing RM to stop scheduling </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3530">MAPREDUCE-3530</a>.
+ Blocker bug reported by Karam Singh and fixed by Arun C Murthy (mrv2 , resourcemanager , scheduler)<br>
+ <b>Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop</b><br>
+ <blockquote>Fixed an NPE occuring during scheduling in the ResourceManager.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3529">MAPREDUCE-3529</a>.
+ Critical bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>TokenCache does not cache viewfs credentials correctly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3528">MAPREDUCE-3528</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mr-am , mrv2)<br>
+ <b>The task timeout check interval should be configurable independent of mapreduce.task.timeout</b><br>
+ <blockquote>Fixed TaskHeartBeatHandler to use a new configuration for the thread loop interval separate from task-timeout configuration property.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3527">MAPREDUCE-3527</a>.
+ Major bug reported by Tom White and fixed by Tom White <br>
+ <b>Fix minor API incompatibilities between 1.0 and 0.23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3525">MAPREDUCE-3525</a>.
+ Blocker sub-task reported by Karam Singh and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>Shuffle benchmark is nearly 1.5x slower in 0.23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3522">MAPREDUCE-3522</a>.
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>Capacity Scheduler ACLs not inherited by default</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3521">MAPREDUCE-3521</a>.
+ Minor bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>Hadoop Streaming ignores unknown parameters</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3519">MAPREDUCE-3519</a>.
+ Blocker sub-task reported by Ravi Gummadi and fixed by Ravi Gummadi (mrv2 , nodemanager)<br>
+ <b>Deadlock in LocalDirsHandlerService and ShuffleHandler</b><br>
+ <blockquote>Fixed a deadlock in NodeManager LocalDirectories's handling service.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3518">MAPREDUCE-3518</a>.
+ Critical bug reported by Jonathan Eagles and fixed by Jonathan Eagles (client , mrv2)<br>
+ <b>mapred queue -info <queue> -showJobs throws NPE</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3513">MAPREDUCE-3513</a>.
+ Trivial bug reported by Mahadev konar and fixed by chackaravarthy (mrv2)<br>
+ <b>Capacity Scheduler web UI has a spelling mistake for Memory.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3512">MAPREDUCE-3512</a>.
+ Blocker sub-task reported by Siddharth Seth and fixed by Siddharth Seth (mr-am , mrv2)<br>
+ <b>Batch jobHistory disk flushes</b><br>
+ <blockquote>Batching JobHistory flushing to DFS so that we don't flush for every event slowing down AM.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3511">MAPREDUCE-3511</a>.
+ Blocker sub-task reported by Siddharth Seth and fixed by Vinod Kumar Vavilapalli (mr-am , mrv2)<br>
+ <b>Counters occupy a good part of AM heap</b><br>
+ <blockquote>Removed a multitude of cloned/duplicate counters in the AM thereby reducing the AM heap size and preventing full GCs.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3510">MAPREDUCE-3510</a>.
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles (capacity-sched , mrv2)<br>
+ <b>Capacity Scheduler inherited ACLs not displayed by mapred queue -showacls</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3505">MAPREDUCE-3505</a>.
+ Major bug reported by Bruno Mahé and fixed by Ahmed Radwan (mrv2)<br>
+ <b>yarn APPLICATION_CLASSPATH needs to be overridable</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3500">MAPREDUCE-3500</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (mrv2)<br>
+ <b>MRJobConfig creates an LD_LIBRARY_PATH using the platform ARCH</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3499">MAPREDUCE-3499</a>.
+ Blocker bug reported by Alejandro Abdelnur and fixed by John George (mrv2 , test)<br>
+ <b>New MiniMR does not setup proxyuser configuration correctly, thus tests using doAs do not work</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3496">MAPREDUCE-3496</a>.
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>Yarn initializes ACL operations from capacity scheduler config in a non-deterministic order</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3490">MAPREDUCE-3490</a>.
+ Blocker bug reported by Siddharth Seth and fixed by Sharad Agarwal (mr-am , mrv2)<br>
+ <b>RMContainerAllocator counts failed maps towards Reduce ramp up</b><br>
+ <blockquote>Fixed MapReduce AM to count failed maps also towards Reduce ramp up.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3488">MAPREDUCE-3488</a>.
+ Blocker bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>Streaming jobs are failing because the main class isnt set in the pom files.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3487">MAPREDUCE-3487</a>.
+ Critical bug reported by Thomas Graves and fixed by Jason Lowe (mrv2)<br>
+ <b>jobhistory web ui task counters no longer links to singletakecounter page</b><br>
+ <blockquote>Fixed JobHistory web-UI to display links to single task's counters' page.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3485">MAPREDUCE-3485</a>.
+ Major sub-task reported by Hitesh Shah and fixed by Ravi Gummadi (mrv2)<br>
+ <b>DISKS_FAILED -101 error code should be defined in same location as ABORTED_CONTAINER_EXIT_STATUS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3484">MAPREDUCE-3484</a>.
+ Major bug reported by Ravi Prakash and fixed by Ravi Prakash (mr-am , mrv2)<br>
+ <b>JobEndNotifier is getting interrupted before completing all its retries.</b><br>
+ <blockquote>Fixed JobEndNotifier to not get interrupted before completing all its retries.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3481">MAPREDUCE-3481</a>.
+ Major improvement reported by Amar Kamat and fixed by Amar Kamat (contrib/gridmix)<br>
+ <b>[Gridmix] Improve STRESS mode locking</b><br>
+ <blockquote>Modified Gridmix STRESS mode locking structure. The submitted thread and the polling thread now run simultaneously without blocking each other. </blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3479">MAPREDUCE-3479</a>.
+ Major bug reported by Tom White and fixed by Tom White (client)<br>
+ <b>JobClient#getJob cannot find local jobs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3478">MAPREDUCE-3478</a>.
+ Minor bug reported by Andrew Bayer and fixed by Tom White (mrv2)<br>
+ <b>Cannot build against ZooKeeper 3.4.0</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3477">MAPREDUCE-3477</a>.
+ Major bug reported by Bruno Mahé and fixed by Jonathan Eagles (documentation , mrv2)<br>
+ <b>Hadoop site documentation cannot be built anymore on trunk and branch-0.23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3468">MAPREDUCE-3468</a>.
+ Major task reported by Siddharth Seth and fixed by Siddharth Seth <br>
+ <b>Change version to 0.23.1 for ant builds on the 23 branch</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3465">MAPREDUCE-3465</a>.
+ Minor bug reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin fails on 0.23 </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3464">MAPREDUCE-3464</a>.
+ Trivial bug reported by Dave Vronay and fixed by Dave Vronay <br>
+ <b>mapreduce jsp pages missing DOCTYPE [post-split branches]</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3463">MAPREDUCE-3463</a>.
+ Blocker bug reported by Karam Singh and fixed by Siddharth Seth (applicationmaster , mrv2)<br>
+ <b>Second AM fails to recover properly when first AM is killed with java.lang.IllegalArgumentException causing lost job</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3462">MAPREDUCE-3462</a>.
+ Blocker bug reported by Amar Kamat and fixed by Ravi Prakash (mrv2 , test)<br>
+ <b>Job submission failing in JUnit tests</b><br>
+ <blockquote>Fixed failing JUnit tests in Gridmix.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3460">MAPREDUCE-3460</a>.
+ Blocker bug reported by Siddharth Seth and fixed by Robert Joseph Evans (mr-am , mrv2)<br>
+ <b>MR AM can hang if containers are allocated on a node blacklisted by the AM</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3458">MAPREDUCE-3458</a>.
+ Major bug reported by Arun C Murthy and fixed by Devaraj K (mrv2)<br>
+ <b>Fix findbugs warnings in hadoop-examples</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3456">MAPREDUCE-3456</a>.
+ Blocker bug reported by Eric Payne and fixed by Eric Payne (mrv2)<br>
+ <b>$HADOOP_PREFIX/bin/yarn should set defaults for $HADOOP_*_HOME</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3454">MAPREDUCE-3454</a>.
+ Major bug reported by Amar Kamat and fixed by Hitesh Shah (contrib/gridmix)<br>
+ <b>[Gridmix] TestDistCacheEmulation is broken</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3453">MAPREDUCE-3453</a>.
+ Major bug reported by Thomas Graves and fixed by Jonathan Eagles (mrv2)<br>
+ <b>RM web ui application details page shows RM cluster about information</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3452">MAPREDUCE-3452</a>.
+ Major bug reported by Thomas Graves and fixed by Jonathan Eagles (mrv2)<br>
+ <b>fifoscheduler web ui page always shows 0% used for the queue</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3450">MAPREDUCE-3450</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mr-am , mrv2)<br>
+ <b>NM port info no longer available in JobHistory</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3448">MAPREDUCE-3448</a>.
+ Minor bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>TestCombineOutputCollector javac unchecked warning on mocked generics</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3447">MAPREDUCE-3447</a>.
+ Blocker bug reported by Thomas Graves and fixed by Mahadev konar (mrv2)<br>
+ <b>mapreduce examples not working</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3444">MAPREDUCE-3444</a>.
+ Blocker bug reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>trunk/0.23 builds broken </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3443">MAPREDUCE-3443</a>.
+ Blocker bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>Oozie jobs are running as oozie user even though they create the jobclient as doAs.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3437">MAPREDUCE-3437</a>.
+ Blocker bug reported by Jonathan Eagles and fixed by Jonathan Eagles (build , mrv2)<br>
+ <b>Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3436">MAPREDUCE-3436</a>.
+ Major bug reported by Bruno Mahé and fixed by Ahmed Radwan (mrv2 , webapps)<br>
+ <b>JobHistory webapp address should use the host from the jobhistory address</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3434">MAPREDUCE-3434</a>.
+ Blocker bug reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>Nightly build broken </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3433">MAPREDUCE-3433</a>.
+ Major sub-task reported by Tom White and fixed by Tom White (client , mrv2)<br>
+ <b>Finding counters by legacy group name returns empty counters</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3427">MAPREDUCE-3427</a>.
+ Blocker bug reported by Alejandro Abdelnur and fixed by Hitesh Shah (contrib/streaming , mrv2)<br>
+ <b>streaming tests fail with MR2</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3426">MAPREDUCE-3426</a>.
+ Blocker sub-task reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>uber-jobs tried to write outputs into wrong dir</b><br>
+ <blockquote>Fixed MR AM in uber mode to write map intermediate outputs in the correct directory to work properly in secure mode.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3422">MAPREDUCE-3422</a>.
+ Major bug reported by Tom White and fixed by Jonathan Eagles (mrv2)<br>
+ <b>Counter display names are not being picked up</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3420">MAPREDUCE-3420</a>.
+ Major bug reported by Hitesh Shah and fixed by (mrv2)<br>
+ <b>[Umbrella ticket] Make uber jobs functional</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3417">MAPREDUCE-3417</a>.
+ Blocker bug reported by Thomas Graves and fixed by Jonathan Eagles (mrv2)<br>
+ <b>job access controls not working app master and job history UI's</b><br>
+ <blockquote>Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3415">MAPREDUCE-3415</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (mrv2)<br>
+ <b>improve MiniMRYarnCluster & DistributedShell JAR resolution </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3413">MAPREDUCE-3413</a>.
+ Minor bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>RM web ui applications not sorted in any order by default</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3412">MAPREDUCE-3412</a>.
+ Major bug reported by Amar Kamat and fixed by Amar Kamat <br>
+ <b>'ant docs' is broken</b><br>
+ <blockquote>Fixes 'ant docs' by removing stale references to capacity-scheduler docs.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3411">MAPREDUCE-3411</a>.
+ Minor improvement reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>Performance Upgrade for jQuery</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3408">MAPREDUCE-3408</a>.
+ Major bug reported by Bruno Mahé and fixed by Bruno Mahé (mrv2 , nodemanager , resourcemanager)<br>
+ <b>yarn-daemon.sh unconditionnaly sets yarn.root.logger</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3407">MAPREDUCE-3407</a>.
+ Minor bug reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>Wrong jar getting used in TestMR*Jobs* for MiniMRYarnCluster</b><br>
+ <blockquote>Fixed pom files to refer to the correct MR app-jar needed by the integration tests.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3404">MAPREDUCE-3404</a>.
+ Critical bug reported by patrick white and fixed by Eric Payne (job submission , mrv2)<br>
+ <b>Speculative Execution: speculative map tasks launched even if -Dmapreduce.map.speculative=false</b><br>
+ <blockquote>Corrected MR AM to honor speculative configuration and enable speculating either maps or reduces.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3402">MAPREDUCE-3402</a>.
+ Blocker sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2)<br>
+ <b>AMScalability test of Sleep job with 100K 1-sec maps regressed into running very slowly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3399">MAPREDUCE-3399</a>.
+ Blocker sub-task reported by Siddharth Seth and fixed by Siddharth Seth (mrv2 , nodemanager)<br>
+ <b>ContainerLocalizer should request new resources after completing the current one</b><br>
+ <blockquote>Modified ContainerLocalizer to send a heartbeat to NM immediately after downloading a resource instead of always waiting for a second.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3398">MAPREDUCE-3398</a>.
+ Blocker bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2 , nodemanager)<br>
+ <b>Log Aggregation broken in Secure Mode</b><br>
+ <blockquote>Fixed log aggregation to work correctly in secure mode. Contributed by Siddharth Seth.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3392">MAPREDUCE-3392</a>.
+ Blocker sub-task reported by John George and fixed by John George <br>
+ <b>Cluster.getDelegationToken() throws NPE if client.getDelegationToken() returns null.</b><br>
+ <blockquote>Fixed Cluster's getDelegationToken's API to return null when there isn't a supported token.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3391">MAPREDUCE-3391</a>.
+ Minor bug reported by Subroto Sanyal and fixed by Subroto Sanyal (applicationmaster)<br>
+ <b>Connecting to CM is logged as Connecting to RM</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3389">MAPREDUCE-3389</a>.
+ Critical bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (mrv2)<br>
+ <b>MRApps loads the 'mrapp-generated-classpath' file with classpath from the build machine</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3387">MAPREDUCE-3387</a>.
+ Critical bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>A tracking URL of N/A before the app master is launched breaks oozie</b><br>
+ <blockquote>Fixed AM's tracking URL to always go through the proxy, even before the job started, so that it works properly with oozie throughout the job execution.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3382">MAPREDUCE-3382</a>.
+ Critical bug reported by Vinod Kumar Vavilapalli and fixed by Ravi Prakash (applicationmaster , mrv2)<br>
+ <b>Network ACLs can prevent AMs to ping the Job-end notification URL</b><br>
+ <blockquote>Enhanced MR AM to use a proxy to ping the job-end notification URL.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3380">MAPREDUCE-3380</a>.
+ Blocker sub-task reported by Alejandro Abdelnur and fixed by Mahadev konar (mr-am , mrv2)<br>
+ <b>Token infrastructure for running clients which are not kerberos authenticated</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3379">MAPREDUCE-3379</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2 , nodemanager)<br>
+ <b>LocalResourceTracker should not tracking deleted cache entries</b><br>
+ <blockquote>Fixed LocalResourceTracker in NodeManager to remove deleted cache entries correctly.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3376">MAPREDUCE-3376</a>.
+ Major bug reported by Robert Joseph Evans and fixed by Subroto Sanyal (mrv1 , mrv2)<br>
+ <b>Old mapred API combiner uses NULL reporter</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3375">MAPREDUCE-3375</a>.
+ Major task reported by Vinay Kumar Thota and fixed by Vinay Kumar Thota <br>
+ <b>Memory Emulation system tests.</b><br>
+ <blockquote>Added system tests to test the memory emulation feature in Gridmix.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3372">MAPREDUCE-3372</a>.
+ Major bug reported by Bruno Mahé and fixed by Bruno Mahé <br>
+ <b>HADOOP_PREFIX cannot be overriden</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3371">MAPREDUCE-3371</a>.
+ Minor improvement reported by Ravi Prakash and fixed by Ravi Prakash (documentation , mrv2)<br>
+ <b>Review and improve the yarn-api javadocs.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3370">MAPREDUCE-3370</a>.
+ Major bug reported by Ahmed Radwan and fixed by Ahmed Radwan (mrv2 , test)<br>
+ <b>MiniMRYarnCluster uses a hard coded path location for the MapReduce application jar</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3369">MAPREDUCE-3369</a>.
+ Major improvement reported by Ahmed Radwan and fixed by Ahmed Radwan (mrv1 , mrv2 , test)<br>
+ <b>Migrate MR1 tests to run on MR2 using the new interfaces introduced in MAPREDUCE-3169</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3368">MAPREDUCE-3368</a>.
+ Critical bug reported by Ramya Sunil and fixed by Hitesh Shah (build , mrv2)<br>
+ <b>compile-mapred-test fails</b><br>
+ <blockquote>Fixed ant test compilation.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3366">MAPREDUCE-3366</a>.
+ Major bug reported by Eric Yang and fixed by Eric Yang (mrv2)<br>
+ <b>Mapreduce component should use consistent directory structure layout as HDFS/common</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3360">MAPREDUCE-3360</a>.
+ Critical improvement reported by Bhallamudi Venkata Siva Kamesh and fixed by Bhallamudi Venkata Siva Kamesh (mrv2)<br>
+ <b>Provide information about lost nodes in the UI.</b><br>
+ <blockquote>Added information about lost/rebooted/decommissioned nodes on the webapps.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3355">MAPREDUCE-3355</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2)<br>
+ <b>AM scheduling hangs frequently with sort job on 350 nodes</b><br>
+ <blockquote>Fixed MR AM's ContainerLauncher to handle node-command timeouts correctly.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3354">MAPREDUCE-3354</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Jonathan Eagles (jobhistoryserver , mrv2)<br>
+ <b>JobHistoryServer should be started by bin/mapred and not by bin/yarn</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3349">MAPREDUCE-3349</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Amar Kamat (mrv2)<br>
+ <b>No rack-name logged in JobHistory for unsuccessful tasks</b><br>
+ <blockquote>Unsuccessful tasks now log hostname and rackname to job history. </blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3346">MAPREDUCE-3346</a>.
+ Blocker bug reported by Karam Singh and fixed by Amar Kamat (tools/rumen)<br>
+ <b>Rumen LoggedTaskAttempt getHostName call returns hostname as null</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3345">MAPREDUCE-3345</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Hitesh Shah (mrv2 , resourcemanager)<br>
+ <b>Race condition in ResourceManager causing TestContainerManagerSecurity to fail sometimes</b><br>
+ <blockquote>Fixed a race condition in ResourceManager that was causing TestContainerManagerSecurity to fail sometimes.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3344">MAPREDUCE-3344</a>.
+ Major bug reported by Brock Noland and fixed by Brock Noland <br>
+ <b>o.a.h.mapreduce.Reducer since 0.21 blindly casts to ReduceContext.ValueIterator</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3342">MAPREDUCE-3342</a>.
+ Critical bug reported by Thomas Graves and fixed by Jonathan Eagles (jobhistoryserver , mrv2)<br>
+ <b>JobHistoryServer doesn't show job queue</b><br>
+ <blockquote>Fixed JobHistoryServer to also show the job's queue name.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3341">MAPREDUCE-3341</a>.
+ Major improvement reported by Anupam Seth and fixed by Anupam Seth (mrv2)<br>
+ <b>Enhance logging of initalized queue limit values</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3339">MAPREDUCE-3339</a>.
+ Blocker bug reported by Ramgopal N and fixed by Siddharth Seth (mrv2)<br>
+ <b>Job is getting hanged indefinitely,if the child processes are killed on the NM. KILL_CONTAINER eventtype is continuosly sent to the containers that are not existing</b><br>
+ <blockquote>Fixed MR AM to stop considering node blacklisting after the number of nodes blacklisted crosses a threshold.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3336">MAPREDUCE-3336</a>.
+ Critical bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>com.google.inject.internal.Preconditions not public api - shouldn't be using it</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3333">MAPREDUCE-3333</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2)<br>
+ <b>MR AM for sort-job going out of memory</b><br>
+ <blockquote>Fixed bugs in ContainerLauncher of MR AppMaster due to which per-container connections to NodeManager were lingering long enough to hit the ulimits on number of processes.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3331">MAPREDUCE-3331</a>.
+ Minor improvement reported by Anupam Seth and fixed by Anupam Seth (mrv2)<br>
+ <b>Improvement to single node cluster setup documentation for 0.23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3329">MAPREDUCE-3329</a>.
+ Blocker bug reported by Thomas Graves and fixed by Arun C Murthy (mrv2)<br>
+ <b>capacity schedule maximum-capacity allowed to be less then capacity</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3328">MAPREDUCE-3328</a>.
+ Critical bug reported by Thomas Graves and fixed by Ravi Prakash (mrv2)<br>
+ <b>mapred queue -list output inconsistent and missing child queues</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3327">MAPREDUCE-3327</a>.
+ Critical bug reported by Thomas Graves and fixed by Anupam Seth (mrv2)<br>
+ <b>RM web ui scheduler link doesn't show correct max value for queues</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3326">MAPREDUCE-3326</a>.
+ Critical bug reported by Thomas Graves and fixed by Jason Lowe (mrv2)<br>
+ <b>RM web UI scheduler link not as useful as should be</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3325">MAPREDUCE-3325</a>.
+ Major improvement reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>Improvements to CapacityScheduler doc</b><br>
+ <blockquote>document changes only.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3324">MAPREDUCE-3324</a>.
+ Critical bug reported by Jonathan Eagles and fixed by Jonathan Eagles (jobhistoryserver , mrv2 , nodemanager)<br>
+ <b>Not All HttpServer tools links (stacks,logs,config,metrics) are accessible through all UI servers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3312">MAPREDUCE-3312</a>.
+ Major bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>Make MR AM not send a stopContainer w/o corresponding start container</b><br>
+ <blockquote>Modified MR AM to not send a stop-container request for a container that isn't launched at all.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3299">MAPREDUCE-3299</a>.
+ Minor improvement reported by Siddharth Seth and fixed by Jonathan Eagles (mrv2)<br>
+ <b>Add AMInfo table to the AM job page</b><br>
+ <blockquote>Added AMInfo table to the MR AM job pages to list all the job-attempts when AM restarts and recovers.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3297">MAPREDUCE-3297</a>.
+ Major task reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>Move Log Related components from yarn-server-nodemanager to yarn-common</b><br>
+ <blockquote>Moved log related components into yarn-common so that HistoryServer and clients can use them without depending on the yarn-server-nodemanager module.
+</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3291">MAPREDUCE-3291</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>App fail to launch due to delegation token not found in cache</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3280">MAPREDUCE-3280</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2)<br>
+ <b>MR AM should not read the username from configuration</b><br>
+ <blockquote>Removed the unnecessary job user-name configuration in mapred-site.xml.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3265">MAPREDUCE-3265</a>.
+ Blocker improvement reported by Todd Lipcon and fixed by Arun C Murthy (mrv2)<br>
+ <b>Reduce log level on MR2 IPC construction, etc</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3251">MAPREDUCE-3251</a>.
+ Critical task reported by Anupam Seth and fixed by Anupam Seth (mrv2)<br>
+ <b>Network ACLs can prevent some clients to talk to MR ApplicationMaster</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3243">MAPREDUCE-3243</a>.
+ Major bug reported by Ramya Sunil and fixed by Jonathan Eagles (contrib/streaming , mrv2)<br>
+ <b>Invalid tracking URL for streaming jobs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3238">MAPREDUCE-3238</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon (mrv2)<br>
+ <b>Small cleanup in SchedulerApp</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3221">MAPREDUCE-3221</a>.
+ Minor sub-task reported by Hitesh Shah and fixed by Devaraj K (mrv2 , test)<br>
+ <b>ant test TestSubmitJob failing on trunk</b><br>
+ <blockquote>Fixed a bug in TestSubmitJob.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3219">MAPREDUCE-3219</a>.
+ Minor sub-task reported by Hitesh Shah and fixed by Hitesh Shah (mrv2 , test)<br>
+ <b>ant test TestDelegationToken failing on trunk</b><br>
+ <blockquote>Reenabled and fixed bugs in the failing test TestDelegationToken.
+</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3217">MAPREDUCE-3217</a>.
+ Minor sub-task reported by Hitesh Shah and fixed by Devaraj K (mrv2 , test)<br>
+ <b>ant test TestAuditLogger fails on trunk</b><br>
+ <blockquote>Reenabled and fixed bugs in the failing ant test TestAuditLogger.
+</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3215">MAPREDUCE-3215</a>.
+ Minor sub-task reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk</b><br>
+ <blockquote>Reneabled and fixed bugs in the failing test TestNoJobSetupCleanup.
+</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3194">MAPREDUCE-3194</a>.
+ Major bug reported by Siddharth Seth and fixed by Jason Lowe (mrv2)<br>
+ <b>"mapred mradmin" command is broken in mrv2</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3169">MAPREDUCE-3169</a>.
+ Major improvement reported by Todd Lipcon and fixed by Ahmed Radwan (mrv1 , mrv2 , test)<br>
+ <b>Create a new MiniMRCluster equivalent which only provides client APIs cross MR1 and MR2</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3147">MAPREDUCE-3147</a>.
+ Major improvement reported by Ravi Prakash and fixed by Ravi Prakash (mrv2)<br>
+ <b>Handle leaf queues with the same name properly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3121">MAPREDUCE-3121</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Ravi Gummadi (mrv2 , nodemanager)<br>
+ <b>DFIP aka 'NodeManager should handle Disk-Failures In Place'</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3102">MAPREDUCE-3102</a>.
+ Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Hitesh Shah (mrv2 , security)<br>
+ <b>NodeManager should fail fast with wrong configuration or permissions for LinuxContainerExecutor</b><br>
+ <blockquote>Changed NodeManager to fail fast when LinuxContainerExecutor has wrong configuration or permissions.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3045">MAPREDUCE-3045</a>.
+ Minor bug reported by Ramya Sunil and fixed by Jonathan Eagles (jobhistoryserver , mrv2)<br>
+ <b>Elapsed time filter on jobhistory server displays incorrect table entries</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2950">MAPREDUCE-2950</a>.
+ Major bug reported by Amar Kamat and fixed by Ravi Gummadi (contrib/gridmix)<br>
+ <b>[Gridmix] TestUserResolve fails in trunk</b><br>
+ <blockquote>Fixes bug in TestUserResolve.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2863">MAPREDUCE-2863</a>.
+ Blocker improvement reported by Arun C Murthy and fixed by Thomas Graves (mrv2 , nodemanager , resourcemanager)<br>
+ <b>Support web-services for RM & NM</b><br>
+ <blockquote>Support for web-services in YARN and MR components.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2784">MAPREDUCE-2784</a>.
+ Major bug reported by Amar Kamat and fixed by Amar Kamat (contrib/gridmix)<br>
+ <b>[Gridmix] TestGridmixSummary fails with NPE when run in DEBUG mode.</b><br>
+ <blockquote>Fixed bugs in ExecutionSummarizer and ResourceUsageMatcher.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2765">MAPREDUCE-2765</a>.
+ Major new feature reported by Mithun Radhakrishnan and fixed by Mithun Radhakrishnan (distcp , mrv2)<br>
+ <b>DistCp Rewrite</b><br>
+ <blockquote>DistCpV2 added to hadoop-tools.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2733">MAPREDUCE-2733</a>.
+ Major task reported by Vinay Kumar Thota and fixed by Vinay Kumar Thota <br>
+ <b>Gridmix v3 cpu emulation system tests.</b><br>
+ <blockquote>Adds system tests for the CPU emulation feature in Gridmix3.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2450">MAPREDUCE-2450</a>.
+ Major bug reported by Matei Zaharia and fixed by Rajesh Balamohan <br>
+ <b>Calls from running tasks to TaskTracker methods sometimes fail and incur a 60s timeout</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1744">MAPREDUCE-1744</a>.
+ Major bug reported by Dick King and fixed by Dick King <br>
+ <b>DistributedCache creates its own FileSytem instance when adding a file/archive to the path</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-778">MAPREDUCE-778</a>.
+ Major new feature reported by Hong Tang and fixed by Amar Kamat (tools/rumen)<br>
+ <b>[Rumen] Need a standalone JobHistory log anonymizer</b><br>
+ <blockquote>Added an anonymizer tool to Rumen. Anonymizer takes a Rumen trace file and/or topology as input. It supports persistence and plugins to override the default behavior.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2923">HDFS-2923</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Namenode IPC handler count uses the wrong configuration key</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2893">HDFS-2893</a>.
+ Minor bug reported by Eli Collins and fixed by Eli Collins <br>
+ <b>The start/stop scripts don't start/stop the 2NN when using the default configuration</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2889">HDFS-2889</a>.
+ Major bug reported by Gregory Chanan and fixed by Gregory Chanan (hdfs client)<br>
+ <b>getNumCurrentReplicas is package private but should be public on 0.23 (see HDFS-2408)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2879">HDFS-2879</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Change FSDataset to package private</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2869">HDFS-2869</a>.
+ Minor bug reported by Harsh J and fixed by Harsh J (webhdfs)<br>
+ <b>Error in Webhdfs documentation for mkdir</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2868">HDFS-2868</a>.
+ Minor improvement reported by Harsh J and fixed by Harsh J (data-node)<br>
+ <b>Add number of active transfer threads to the DataNode status</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2864">HDFS-2864</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Remove redundant methods and a constant from FSDataset</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2840">HDFS-2840</a>.
+ Major bug reported by Eli Collins and fixed by Alejandro Abdelnur (test)<br>
+ <b>TestHostnameFilter should work with localhost or localhost.localdomain </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2837">HDFS-2837</a>.
+ Major bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans <br>
+ <b>mvn javadoc:javadoc not seeing LimitedPrivate class </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2836">HDFS-2836</a>.
+ Major bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans <br>
+ <b>HttpFSServer still has 2 javadoc warnings in trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2835">HDFS-2835</a>.
+ Major bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (tools)<br>
+ <b>Fix org.apache.hadoop.hdfs.tools.GetConf$Command Findbug issue</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2827">HDFS-2827</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (name-node)<br>
+ <b>Cannot save namespace after renaming a directory above a file with an open lease</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2826">HDFS-2826</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (name-node , test)<br>
+ <b>Test case for HDFS-1476 (safemode can initialize repl queues before exiting)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2825">HDFS-2825</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Add test hook to turn off the writer preferring its local DN</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2822">HDFS-2822</a>.
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon (ha , name-node)<br>
+ <b>processMisReplicatedBlock incorrectly identifies under-construction blocks as under-replicated</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2818">HDFS-2818</a>.
+ Trivial bug reported by Todd Lipcon and fixed by Devaraj K (name-node)<br>
+ <b>dfshealth.jsp missing space between role and node name</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2817">HDFS-2817</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (test)<br>
+ <b>Combine the two TestSafeMode test suites</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2816">HDFS-2816</a>.
+ Trivial bug reported by Hitesh Shah and fixed by Hitesh Shah <br>
+ <b>Fix missing license header in hadoop-hdfs-project/hadoop-hdfs-httpfs/dev-support/findbugsExcludeFile.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2814">HDFS-2814</a>.
+ Minor improvement reported by Hitesh Shah and fixed by Hitesh Shah <br>
+ <b>NamenodeMXBean does not account for svn revision in the version information</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2810">HDFS-2810</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (hdfs client)<br>
+ <b>Leases not properly getting renewed by clients</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2803">HDFS-2803</a>.
+ Minor improvement reported by Jimmy Xiang and fixed by Jimmy Xiang (name-node)<br>
+ <b>Adding logging to LeaseRenewer for better lease expiration triage.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2791">HDFS-2791</a>.
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon (data-node , name-node)<br>
+ <b>If block report races with closing of file, replica is incorrectly marked corrupt</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2790">HDFS-2790</a>.
+ Minor bug reported by Arpit Gupta and fixed by Arpit Gupta <br>
+ <b>FSNamesystem.setTimes throws exception with wrong configuration name in the message</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2788">HDFS-2788</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins (data-node)<br>
+ <b>HdfsServerConstants#DN_KEEPALIVE_TIMEOUT is dead code</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2786">HDFS-2786</a>.
+ Major sub-task reported by Daryn Sharp and fixed by Kihwal Lee (name-node , security)<br>
+ <b>Fix host-based token incompatibilities in DFSUtil</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2785">HDFS-2785</a>.
+ Major sub-task reported by Daryn Sharp and fixed by Robert Joseph Evans (webhdfs)<br>
+ <b>Update webhdfs and httpfs for host-based token support</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2784">HDFS-2784</a>.
+ Major sub-task reported by Daryn Sharp and fixed by Kihwal Lee (hdfs client , name-node , security)<br>
+ <b>Update hftp and hdfs for host-based token support</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2761">HDFS-2761</a>.
+ Major improvement reported by Roman Shaposhnik and fixed by Roman Shaposhnik (build , hdfs client , scripts)<br>
+ <b>Improve Hadoop subcomponent integration in Hadoop 0.23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2751">HDFS-2751</a>.
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>Datanode drops OS cache behind reads even for short reads</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2729">HDFS-2729</a>.
+ Minor improvement reported by Harsh J and fixed by Harsh J (name-node)<br>
+ <b>Update BlockManager's comments regarding the invalid block set</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2726">HDFS-2726</a>.
+ Major improvement reported by Michael Bieniosek and fixed by Harsh J <br>
+ <b>"Exception in createBlockOutputStream" shouldn't delete exception stack trace</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2722">HDFS-2722</a>.
+ Major bug reported by Harsh J and fixed by Harsh J (hdfs client)<br>
+ <b>HttpFs shouldn't be using an int for block size</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2710">HDFS-2710</a>.
+ Critical bug reported by Siddharth Seth and fixed by <br>
+ <b>HDFS part of MAPREDUCE-3529, HADOOP-7933</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2707">HDFS-2707</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)<br>
+ <b>HttpFS should read the hadoop-auth secret from a file instead inline from the configuration</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2706">HDFS-2706</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Use configuration for blockInvalidateLimit if it is set</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2705">HDFS-2705</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur <br>
+ <b>HttpFS server should check that upload requests have correct content-type</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2675">HDFS-2675</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Reduce verbosity when double-closing edit logs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2658">HDFS-2658</a>.
+ Major bug reported by Eli Collins and fixed by Alejandro Abdelnur <br>
+ <b>HttpFS introduced 70 javadoc warnings</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2657">HDFS-2657</a>.
+ Major bug reported by Eli Collins and fixed by Alejandro Abdelnur <br>
+ <b>TestHttpFSServer and TestServerWebApp are failing on trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2654">HDFS-2654</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins (data-node)<br>
+ <b>Make BlockReaderLocal not extend RemoteBlockReader2</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2653">HDFS-2653</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins (data-node)<br>
+ <b>DFSClient should cache whether addrs are non-local when short-circuiting is enabled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2649">HDFS-2649</a>.
+ Major bug reported by Jason Lowe and fixed by Jason Lowe (build)<br>
+ <b>eclipse:eclipse build fails for hadoop-hdfs-httpfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2646">HDFS-2646</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Alejandro Abdelnur <br>
+ <b>Hadoop HttpFS introduced 4 findbug warnings.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2640">HDFS-2640</a>.
+ Major bug reported by Tom White and fixed by Tom White <br>
+ <b>Javadoc generation hangs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2614">HDFS-2614</a>.
+ Major bug reported by Bruno Mahé and fixed by Alejandro Abdelnur (build)<br>
+ <b>hadoop dist tarball is missing hdfs headers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2606">HDFS-2606</a>.
+ Critical bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (webhdfs)<br>
+ <b>webhdfs client filesystem impl must set the content-type header for create/append</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2604">HDFS-2604</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>Add a log message to show if WebHDFS is enabled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2596">HDFS-2596</a>.
+ Major bug reported by Eli Collins and fixed by Eli Collins (data-node , test)<br>
+ <b>TestDirectoryScanner doesn't test parallel scans</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2590">HDFS-2590</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>Some links in WebHDFS forrest doc do not work</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2588">HDFS-2588</a>.
+ Trivial bug reported by Dave Vronay and fixed by Dave Vronay (scripts)<br>
+ <b>hdfs jsp pages missing DOCTYPE [post-split branches]</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2587">HDFS-2587</a>.
+ Major task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>Add WebHDFS apt doc</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2575">HDFS-2575</a>.
+ Minor bug reported by Todd Lipcon and fixed by Todd Lipcon (test)<br>
+ <b>DFSTestUtil may create empty files</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2574">HDFS-2574</a>.
+ Trivial task reported by Joe Crobak and fixed by Joe Crobak (documentation)<br>
+ <b>remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2572">HDFS-2572</a>.
+ Trivial improvement reported by Harsh J and fixed by Harsh J (data-node)<br>
+ <b>Unnecessary double-check in DN#getHostName</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2570">HDFS-2570</a>.
+ Trivial improvement reported by Eli Collins and fixed by Eli Collins (documentation)<br>
+ <b>Add descriptions for dfs.*.https.address in hdfs-default.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2568">HDFS-2568</a>.
+ Trivial improvement reported by Harsh J and fixed by Harsh J (data-node)<br>
+ <b>Use a set to manage child sockets in XceiverServer</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2567">HDFS-2567</a>.
+ Major bug reported by Harsh J and fixed by Harsh J (name-node)<br>
+ <b>When 0 DNs are available, show a proper error when trying to browse DFS via web UI</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2566">HDFS-2566</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>Move BPOfferService to be a non-inner class</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2563">HDFS-2563</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>Some cleanup in BPOfferService</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2562">HDFS-2562</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>Refactor DN configuration variables out of DataNode class</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2560">HDFS-2560</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>Refactor BPOfferService to be a static inner class</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2553">HDFS-2553</a>.
+ Critical bug reported by Todd Lipcon and fixed by Uma Maheswara Rao G (data-node)<br>
+ <b>BlockPoolSliceScanner spinning in loop</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2552">HDFS-2552</a>.
+ Major task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>Add WebHdfs Forrest doc</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2545">HDFS-2545</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>Webhdfs: Support multiple namenodes in federation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2544">HDFS-2544</a>.
+ Major bug reported by Bruno Mahé and fixed by Bruno Mahé (scripts)<br>
+ <b>Hadoop scripts unconditionally source "$bin"/../libexec/hadoop-config.sh.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2543">HDFS-2543</a>.
+ Major bug reported by Bruno Mahé and fixed by Bruno Mahé (scripts)<br>
+ <b>HADOOP_PREFIX cannot be overriden</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2541">HDFS-2541</a>.
+ Major bug reported by Harsh J and fixed by Harsh J (data-node)<br>
+ <b>For a sufficiently large value of blocks, the DN Scanner may request a random number with a negative seed value.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2536">HDFS-2536</a>.
+ Trivial improvement reported by Aaron T. Myers and fixed by Harsh J (name-node)<br>
+ <b>Remove unused imports</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2533">HDFS-2533</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node , performance)<br>
+ <b>Remove needless synchronization on FSDataSet.getBlockFile</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2511">HDFS-2511</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Alejandro Abdelnur (build)<br>
+ <b>Add dev script to generate HDFS protobufs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2502">HDFS-2502</a>.
+ Minor improvement reported by Eli Collins and fixed by Harsh J (documentation)<br>
+ <b>hdfs-default.xml should include dfs.name.dir.restore</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2454">HDFS-2454</a>.
+ Minor improvement reported by Uma Maheswara Rao G and fixed by Harsh J (data-node)<br>
+ <b>Move maxXceiverCount check to before starting the thread in dataXceiver</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2397">HDFS-2397</a>.
+ Major improvement reported by Todd Lipcon and fixed by Eli Collins (name-node)<br>
+ <b>Undeprecate SecondaryNameNode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2349">HDFS-2349</a>.
+ Trivial improvement reported by Harsh J and fixed by Harsh J (data-node)<br>
+ <b>DN should log a WARN, not an INFO when it detects a corruption during block transfer</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2335">HDFS-2335</a>.
+ Major improvement reported by Eli Collins and fixed by Uma Maheswara Rao G (data-node , name-node)<br>
+ <b>DataNodeCluster and NNStorage always pull fresh entropy</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2246">HDFS-2246</a>.
+ Major improvement reported by Sanjay Radia and fixed by Jitendra Nath Pandey <br>
+ <b>Shortcut a local client reads to a Datanodes files directly</b><br>
+ <blockquote>1. New configurations
+a. dfs.block.local-path-access.user is the key in datanode configuration to specify the user allowed to do short circuit read.
+b. dfs.client.read.shortcircuit is the key to enable short circuit read at the client side configuration.
+c. dfs.client.read.shortcircuit.skip.checksum is the key to bypass checksum check at the client side.
+2. By default none of the above are enabled and short circuit read will not kick in.
+3. If security is on, the feature can be used only for user that has kerberos credentials at the client, therefore map reduce tasks cannot benefit from it in general.
+</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2178">HDFS-2178</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur <br>
+ <b>HttpFS - a read/write Hadoop file system proxy</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2130">HDFS-2130</a>.
+ Major sub-task reported by Todd Lipcon and fixed by Todd Lipcon (hdfs client)<br>
+ <b>Switch default checksum to CRC32C</b><br>
+ <blockquote>The default checksum algorithm used on HDFS is now CRC32C. Data from previous versions of Hadoop can still be read backwards-compatibly.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2129">HDFS-2129</a>.
+ Major sub-task reported by Todd Lipcon and fixed by Todd Lipcon (hdfs client , performance)<br>
+ <b>Simplify BlockReader to not inherit from FSInputChecker</b><br>
+ <blockquote>BlockReader has been reimplemented to use direct byte buffers. If you use a custom socket factory, it must generate sockets that have associated Channels.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2080">HDFS-2080</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (hdfs client , performance)<br>
+ <b>Speed up DFS read path by lessening checksum overhead</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1314">HDFS-1314</a>.
+ Minor bug reported by Karim Saadah and fixed by Sho Shimauchi <br>
+ <b>dfs.blocksize accepts only absolute value</b><br>
+ <blockquote>The default blocksize property 'dfs.blocksize' now accepts unit symbols to be used instead of byte length. Values such as "10k", "128m", "1g" are now OK to provide instead of just no. of bytes as was before.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-554">HDFS-554</a>.
+ Minor improvement reported by Steve Loughran and fixed by Harsh J (name-node)<br>
+ <b>BlockInfo.ensureCapacity may get a speedup from System.arraycopy()</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-442">HDFS-442</a>.
+ Minor bug reported by Ramya Sunil and fixed by Harsh J (test)<br>
+ <b>dfsthroughput in test.jar throws NPE</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-362">HDFS-362</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Uma Maheswara Rao G (name-node)<br>
+ <b>FSEditLog should not writes long and short as UTF8 and should not use ArrayWritable for writing non-array items</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-69">HDFS-69</a>.
+ Minor bug reported by Ravi Phulari and fixed by Harsh J <br>
+ <b>Improve dfsadmin command line help </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8055">HADOOP-8055</a>.
+ Major bug reported by Eric Charles and fixed by Harsh J (build)<br>
+ <b>Distribution tar.gz does not contain etc/hadoop/core-site.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8054">HADOOP-8054</a>.
+ Critical bug reported by Amareshwari Sriramadasu and fixed by Daryn Sharp (fs)<br>
+ <b>NPE with FilterFileSystem</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8052">HADOOP-8052</a>.
+ Major bug reported by Varun Kapoor and fixed by Varun Kapoor (metrics)<br>
+ <b>Hadoop Metrics2 should emit Float.MAX_VALUE (instead of Double.MAX_VALUE) to avoid making Ganglia's gmetad core</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8027">HADOOP-8027</a>.
+ Minor improvement reported by Harsh J and fixed by Aaron T. Myers (metrics)<br>
+ <b>Visiting /jmx on the daemon web interfaces may print unnecessary error in logs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8018">HADOOP-8018</a>.
+ Major bug reported by Matt Foley and fixed by Jonathan Eagles (build , test)<br>
+ <b>Hudson auto test for HDFS has started throwing javadoc: warning - Error fetching URL: http://java.sun.com/javase/6/docs/api/package-list</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8015">HADOOP-8015</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>ChRootFileSystem should extend FilterFileSystem</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8013">HADOOP-8013</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>ViewFileSystem does not honor setVerifyChecksum</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8012">HADOOP-8012</a>.
+ Minor bug reported by Roman Shaposhnik and fixed by Roman Shaposhnik (scripts)<br>
+ <b>hadoop-daemon.sh and yarn-daemon.sh are trying to mkdir and chow log/pid dirs which can fail</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8009">HADOOP-8009</a>.
+ Critical improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Create hadoop-client and hadoop-minicluster artifacts for downstream projects </b><br>
+ <blockquote>Generate integration artifacts "org.apache.hadoop:hadoop-client" and "org.apache.hadoop:hadoop-minicluster" containing all the jars needed to use Hadoop client APIs, and to run Hadoop MiniClusters, respectively. Push these artifacts to the maven repository when mvn-deploy, along with existing artifacts. </blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8006">HADOOP-8006</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Daryn Sharp (fs)<br>
+ <b>TestFSInputChecker is failing in trunk.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8002">HADOOP-8002</a>.
+ Major bug reported by Arpit Gupta and fixed by Arpit Gupta <br>
+ <b>SecurityUtil acquired token message should be a debug rather than info</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8001">HADOOP-8001</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>ChecksumFileSystem's rename doesn't correctly handle checksum files</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8000">HADOOP-8000</a>.
+ Critical bug reported by Arpit Gupta and fixed by Arpit Gupta <br>
+ <b>fetchdt command not available in bin/hadoop</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7999">HADOOP-7999</a>.
+ Critical bug reported by Jason Lowe and fixed by Jason Lowe (scripts)<br>
+ <b>"hadoop archive" fails with ClassNotFoundException</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7998">HADOOP-7998</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>CheckFileSystem does not correctly honor setVerifyChecksum</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7993">HADOOP-7993</a>.
+ Major bug reported by Anupam Seth and fixed by Anupam Seth (conf)<br>
+ <b>Hadoop ignores old-style config options for enabling compressed output</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7988">HADOOP-7988</a>.
+ Major bug reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Upper case in hostname part of the principals doesn't work with kerberos.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7987">HADOOP-7987</a>.
+ Major improvement reported by Devaraj Das and fixed by Jitendra Nath Pandey (security)<br>
+ <b>Support setting the run-as user in unsecure mode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7986">HADOOP-7986</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar <br>
+ <b>Add config for History Server protocol in hadoop-policy for service level authorization.</b><br>
+ <blockquote>Adding config for MapReduce History Server protocol in hadoop-policy.xml for service level authorization.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7982">HADOOP-7982</a>.
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon (security)<br>
+ <b>UserGroupInformation fails to login if thread's context classloader can't load HadoopLoginModule</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7981">HADOOP-7981</a>.
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles (io)<br>
+ <b>Improve documentation for org.apache.hadoop.io.compress.Decompressor.getRemaining</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7975">HADOOP-7975</a>.
+ Minor bug reported by Harsh J and fixed by Harsh J <br>
+ <b>Add entry to XML defaults for new LZ4 codec</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7974">HADOOP-7974</a>.
+ Major bug reported by Eli Collins and fixed by Harsh J (fs , test)<br>
+ <b>TestViewFsTrash incorrectly determines the user's home directory</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7971">HADOOP-7971</a>.
+ Blocker bug reported by Thomas Graves and fixed by Prashant Sharma <br>
+ <b>hadoop <job/queue/pipes> removed - should be added back, but deprecated</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7964">HADOOP-7964</a>.
+ Blocker bug reported by Kihwal Lee and fixed by Daryn Sharp (security , util)<br>
+ <b>Deadlock in class init.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7963">HADOOP-7963</a>.
+ Blocker bug reported by Thomas Graves and fixed by Siddharth Seth <br>
+ <b>test failures: TestViewFileSystemWithAuthorityLocalFileSystem and TestViewFileSystemLocalFileSystem</b><br>
+ <blockquote>Fix ViewFS to catch a null canonical service-name and pass tests TestViewFileSystem*</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7949">HADOOP-7949</a>.
+ Trivial bug reported by Eli Collins and fixed by Eli Collins (ipc)<br>
+ <b>Updated maxIdleTime default in the code to match core-default.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7948">HADOOP-7948</a>.
+ Minor bug reported by Michajlo Matijkiw and fixed by Michajlo Matijkiw (build)<br>
+ <b>Shell scripts created by hadoop-dist/pom.xml to build tar do not properly propagate failure</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7939">HADOOP-7939</a>.
+ Major improvement reported by Roman Shaposhnik and fixed by Roman Shaposhnik (build , conf , documentation , scripts)<br>
+ <b>Improve Hadoop subcomponent integration in Hadoop 0.23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7936">HADOOP-7936</a>.
+ Major bug reported by Eli Collins and fixed by Alejandro Abdelnur (build)<br>
+ <b>There's a Hoop README in the root dir of the tarball</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7934">HADOOP-7934</a>.
+ Critical improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Normalize dependencies versions across all modules</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7933">HADOOP-7933</a>.
+ Critical bug reported by Siddharth Seth and fixed by Siddharth Seth (viewfs)<br>
+ <b>Viewfs changes for MAPREDUCE-3529</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7919">HADOOP-7919</a>.
+ Trivial improvement reported by Harsh J and fixed by Harsh J (documentation)<br>
+ <b>[Doc] Remove hadoop.logfile.* properties.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7917">HADOOP-7917</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>compilation of protobuf files fails in windows/cygwin</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7914">HADOOP-7914</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (build)<br>
+ <b>duplicate declaration of hadoop-hdfs test-jar</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7912">HADOOP-7912</a>.
+ Major bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (build)<br>
+ <b>test-patch should run eclipse:eclipse to verify that it does not break again</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7910">HADOOP-7910</a>.
+ Minor improvement reported by Sho Shimauchi and fixed by Sho Shimauchi (conf)<br>
+ <b>add configuration methods to handle human readable size values</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7907">HADOOP-7907</a>.
+ Blocker bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>hadoop-tools JARs are not part of the distro</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7902">HADOOP-7902</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Alejandro Abdelnur <br>
+ <b>skipping name rules setting (if already set) should be done on UGI initialization only </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7898">HADOOP-7898</a>.
+ Minor bug reported by Suresh Srinivas and fixed by Suresh Srinivas (security)<br>
+ <b>Fix javadoc warnings in AuthenticationToken.java</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7890">HADOOP-7890</a>.
+ Trivial improvement reported by Koji Noguchi and fixed by Koji Noguchi (scripts)<br>
+ <b>Redirect hadoop script's deprecation message to stderr</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7887">HADOOP-7887</a>.
+ Critical bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)<br>
+ <b>KerberosAuthenticatorHandler is not setting KerberosName name rules from configuration</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7878">HADOOP-7878</a>.
+ Minor bug reported by Steve Loughran and fixed by Steve Loughran (util)<br>
+ <b>Regression HADOOP-7777 switch changes break HDFS tests when the isSingleSwitch() predicate is used</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7877">HADOOP-7877</a>.
+ Major task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (documentation)<br>
+ <b>Federation: update Balancer documentation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7874">HADOOP-7874</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>native libs should be under lib/native/ dir</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7870">HADOOP-7870</a>.
+ Major bug reported by Jonathan Hsieh and fixed by Jonathan Hsieh <br>
+ <b>fix SequenceFile#createWriter with boolean createParent arg to respect createParent.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7864">HADOOP-7864</a>.
+ Major bug reported by Andrew Bayer and fixed by Andrew Bayer (build)<br>
+ <b>Building mvn site with Maven < 3.0.2 causes OOM errors</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7859">HADOOP-7859</a>.
+ Major bug reported by Eli Collins and fixed by Eli Collins (fs)<br>
+ <b>TestViewFsHdfs.testgetFileLinkStatus is failing an assert</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7858">HADOOP-7858</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Drop some info logging to DEBUG level in IPC, metrics, and HTTP</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7854">HADOOP-7854</a>.
+ Critical bug reported by Daryn Sharp and fixed by Daryn Sharp (security)<br>
+ <b>UGI getCurrentUser is not synchronized</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7853">HADOOP-7853</a>.
+ Blocker bug reported by Daryn Sharp and fixed by Daryn Sharp (security)<br>
+ <b>multiple javax security configurations cause conflicts</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7851">HADOOP-7851</a>.
+ Major bug reported by Amar Kamat and fixed by Uma Maheswara Rao G (conf)<br>
+ <b>Configuration.getClasses() never returns the default value.</b><br>
+ <blockquote>Fixed Configuration.getClasses() API to return the default value if the key is not set.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7843">HADOOP-7843</a>.
+ Major bug reported by John George and fixed by John George <br>
+ <b>compilation failing because workDir not initialized in RunJar.java</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7841">HADOOP-7841</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon (build)<br>
+ <b>Run tests with non-secure random</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7837">HADOOP-7837</a>.
+ Major bug reported by Steve Loughran and fixed by Eli Collins (conf)<br>
+ <b>no NullAppender in the log4j config</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7813">HADOOP-7813</a>.
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles (build , test)<br>
+ <b>test-patch +1 patches that introduce javadoc and findbugs warnings in some cases</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7811">HADOOP-7811</a>.
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles (security , test)<br>
+ <b>TestUserGroupInformation#testGetServerSideGroups test fails in chroot</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7810">HADOOP-7810</a>.
+ Blocker bug reported by John George and fixed by John George <br>
+ <b>move hadoop archive to core from tools</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7808">HADOOP-7808</a>.
+ Major new feature reported by Daryn Sharp and fixed by Daryn Sharp (fs , security)<br>
+ <b>Port token service changes from 205</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7804">HADOOP-7804</a>.
+ Major improvement reported by Arpit Gupta and fixed by Arpit Gupta (conf)<br>
+ <b>enable hadoop config generator to set dfs.block.local-path-access.user to enable short circuit read</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7802">HADOOP-7802</a>.
+ Major bug reported by Bruno Mahé and fixed by Bruno Mahé <br>
+ <b>Hadoop scripts unconditionally source "$bin"/../libexec/hadoop-config.sh.</b><br>
+ <blockquote>Here is a patch to enable this behavior
+</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7801">HADOOP-7801</a>.
+ Major bug reported by Bruno Mahé and fixed by Bruno Mahé (build)<br>
+ <b>HADOOP_PREFIX cannot be overriden</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7787">HADOOP-7787</a>.
+ Major bug reported by Bruno Mahé and fixed by Bruno Mahé (build)<br>
+ <b>Make source tarball use conventional name.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7761">HADOOP-7761</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (io , performance , util)<br>
+ <b>Improve performance of raw comparisons</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7758">HADOOP-7758</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (fs)<br>
+ <b>Make GlobFilter class public</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7736">HADOOP-7736</a>.
+ Trivial improvement reported by Harsh J and fixed by Harsh J (fs)<br>
+ <b>Remove duplicate call of Path#normalizePath during initialization.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7657">HADOOP-7657</a>.
+ Major improvement reported by Bert Sanders and fixed by Binglin Chang <br>
+ <b>Add support for LZ4 compression</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7590">HADOOP-7590</a>.
+ Major sub-task reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Mavenize streaming and MR examples</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7574">HADOOP-7574</a>.
+ Trivial improvement reported by XieXianshan and fixed by XieXianshan (fs)<br>
+ <b>Improvement for FSshell -stat</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7504">HADOOP-7504</a>.
+ Trivial improvement reported by Eli Collins and fixed by Harsh J (metrics)<br>
+ <b>hadoop-metrics.properties missing some Ganglia31 options </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7470">HADOOP-7470</a>.
+ Minor improvement reported by Steve Loughran and fixed by Enis Soztutar (util)<br>
+ <b>move up to Jackson 1.8.8</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7424">HADOOP-7424</a>.
+ Major improvement reported by Eli Collins and fixed by Uma Maheswara Rao G <br>
+ <b>Log an error if the topology script doesn't handle multiple args</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7348">HADOOP-7348</a>.
+ Major improvement reported by XieXianshan and fixed by XieXianshan (fs)<br>
+ <b>Modify the option of FsShell getmerge from [addnl] to [-nl] for consistency</b><br>
+ <blockquote>The 'fs -getmerge' tool now uses a -nl flag to determine if adding a newline at end of each file is required, in favor of the 'addnl' boolean flag that was used earlier.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6886">HADOOP-6886</a>.
+ Minor improvement reported by Nicolas Spiegelberg and fixed by Nicolas Spiegelberg (fs)<br>
+ <b>LocalFileSystem Needs createNonRecursive API</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6840">HADOOP-6840</a>.
+ Minor improvement reported by Nicolas Spiegelberg and fixed by Nicolas Spiegelberg (fs , io)<br>
+ <b>Support non-recursive create() in FileSystem & SequenceFile.Writer</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6614">HADOOP-6614</a>.
+ Minor improvement reported by Steve Loughran and fixed by Jonathan Hsieh (util)<br>
+ <b>RunJar should provide more diags when it can't create a temp file</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6490">HADOOP-6490</a>.
+ Minor bug reported by Zheng Shao and fixed by Uma Maheswara Rao G (fs)<br>
+ <b>Path.normalize should use StringUtils.replace in favor of String.replace</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-4515">HADOOP-4515</a>.
+ Minor improvement reported by Abhijit Bagri and fixed by Sho Shimauchi <br>
+ <b>conf.getBoolean must be case insensitive</b><br>
+ <blockquote></blockquote></li>
+</ul>
+</body></html>
+<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
+<title>Hadoop 0.23.0 Release Notes</title>
+<STYLE type="text/css">
+ H1 {font-family: sans-serif}
+ H2 {font-family: sans-serif; margin-left: 7mm}
+ TABLE {margin-left: 7mm}
+</STYLE>
+</head>
+<body>
+<h1>Hadoop 0.23.0 Release Notes</h1>
+These release notes include new developer and user-facing incompatibilities, features, and major improvements.
+<a name="changes"/>
+<h2>Changes since Hadoop 1.0.0</h2>
+<ul>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3332">MAPREDUCE-3332</a>.
+ Trivial bug reported by Hitesh Shah and fixed by Hitesh Shah (contrib/raid)<br>
+ <b>contrib/raid compile breaks due to changes in hdfs/protocol/datatransfer/Sender#writeBlock related to checksum handling </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3322">MAPREDUCE-3322</a>.
+ Major improvement reported by Arun C Murthy and fixed by Arun C Murthy (documentation , mrv2)<br>
+ <b>Create a better index.html for maven docs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3321">MAPREDUCE-3321</a>.
+ Minor bug reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>Disable some failing legacy tests for MRv2 builds to go through</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3317">MAPREDUCE-3317</a>.
+ Major bug reported by Ravi Gummadi and fixed by Ravi Gummadi (tools/rumen)<br>
+ <b>Rumen TraceBuilder is emiting null as hostname</b><br>
+ <blockquote>Fixes Rumen to get correct hostName that includes rackName in attempt info.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3316">MAPREDUCE-3316</a>.
+ Major bug reported by Bhallamudi Venkata Siva Kamesh and fixed by Bhallamudi Venkata Siva Kamesh (resourcemanager)<br>
+ <b>Rebooted link is not working properly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3313">MAPREDUCE-3313</a>.
+ Blocker bug reported by Ravi Gummadi and fixed by Hitesh Shah (mrv2 , test)<br>
+ <b>TestResourceTrackerService failing in trunk some times</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3306">MAPREDUCE-3306</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2 , nodemanager)<br>
+ <b>Cannot run apps after MAPREDUCE-2989</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3304">MAPREDUCE-3304</a>.
+ Major bug reported by Ravi Prakash and fixed by Ravi Prakash (mrv2 , test)<br>
+ <b>TestRMContainerAllocator#testBlackListedNodes fails intermittently</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3296">MAPREDUCE-3296</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (build)<br>
+ <b>Pending(9) findBugs warnings</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3295">MAPREDUCE-3295</a>.
+ Critical bug reported by Mahadev konar and fixed by <br>
+ <b>TestAMAuthorization failing on branch 0.23.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3292">MAPREDUCE-3292</a>.
+ Critical bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>In secure mode job submission fails with Provider org.apache.hadoop.mapreduce.security.token.JobTokenIndentifier$Renewer not found.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3290">MAPREDUCE-3290</a>.
+ Major bug reported by Ramya Sunil and fixed by Arun C Murthy (mrv2)<br>
+ <b>list-active-trackers throws NPE</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3288">MAPREDUCE-3288</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Mahadev konar (mrv2)<br>
+ <b>Mapreduce 23 builds failing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3285">MAPREDUCE-3285</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Siddharth Seth (mrv2)<br>
+ <b>Tests on branch-0.23 failing </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3284">MAPREDUCE-3284</a>.
+ Major bug reported by Ramya Sunil and fixed by Arun C Murthy (mrv2)<br>
+ <b>bin/mapred queue fails with JobQueueClient ClassNotFoundException</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3282">MAPREDUCE-3282</a>.
+ Critical bug reported by Ramya Sunil and fixed by Arun C Murthy (mrv2)<br>
+ <b>bin/mapred job -list throws exception</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3281">MAPREDUCE-3281</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (test)<br>
+ <b>TestLinuxContainerExecutorWithMocks failing on trunk.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3279">MAPREDUCE-3279</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>TestJobHistoryParsing broken</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3275">MAPREDUCE-3275</a>.
+ Critical improvement reported by Robert Joseph Evans and fixed by Robert Joseph Evans (documentation , mrv2)<br>
+ <b>Add docs for WebAppProxy</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3274">MAPREDUCE-3274</a>.
+ Blocker bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (applicationmaster , mrv2)<br>
+ <b>Race condition in MR App Master Preemtion can cause a dead lock</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3269">MAPREDUCE-3269</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Mahadev konar (mrv2)<br>
+ <b>Jobsummary logs not being moved to a separate file</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3264">MAPREDUCE-3264</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Arun C Murthy (mrv2)<br>
+ <b>mapreduce.job.user.name needs to be set automatically</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3263">MAPREDUCE-3263</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Hitesh Shah (build , mrv2)<br>
+ <b>compile-mapred-test target fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3262">MAPREDUCE-3262</a>.
+ Critical bug reported by Hitesh Shah and fixed by Hitesh Shah (mrv2 , nodemanager)<br>
+ <b>A few events are not handled by the NodeManager in failure scenarios</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3261">MAPREDUCE-3261</a>.
+ Major bug reported by Chris Riccomini and fixed by (applicationmaster)<br>
+ <b>AM unable to release containers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3259">MAPREDUCE-3259</a>.
+ Blocker bug reported by Kihwal Lee and fixed by Kihwal Lee (mrv2 , nodemanager)<br>
+ <b>ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3258">MAPREDUCE-3258</a>.
+ Blocker bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>Job counters missing from AM and history UI</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3257">MAPREDUCE-3257</a>.
+ Blocker sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2 , resourcemanager , security)<br>
+ <b>Authorization checks needed for AM->RM protocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3256">MAPREDUCE-3256</a>.
+ Blocker sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2 , nodemanager , security)<br>
+ <b>Authorization checks needed for AM->NM protocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3254">MAPREDUCE-3254</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Arun C Murthy (contrib/streaming , mrv2)<br>
+ <b>Streaming jobs failing with PipeMapRunner ClassNotFoundException</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3253">MAPREDUCE-3253</a>.
+ Blocker bug reported by Daniel Dai and fixed by Arun C Murthy (mrv2)<br>
+ <b>ContextFactory throw NoSuchFieldException</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3252">MAPREDUCE-3252</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (mrv2 , task)<br>
+ <b>MR2: Map tasks rewrite data once even if output fits in sort buffer</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3250">MAPREDUCE-3250</a>.
+ Blocker sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2)<br>
+ <b>When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3249">MAPREDUCE-3249</a>.
+ Blocker sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2)<br>
+ <b>Recovery of MR AMs with reduces fails the subsequent generation of the job</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3248">MAPREDUCE-3248</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Vinod Kumar Vavilapalli (test)<br>
+ <b>Log4j logs from unit tests are lost</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3242">MAPREDUCE-3242</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>Trunk compilation broken with bad interaction from MAPREDUCE-3070 and MAPREDUCE-3239.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3241">MAPREDUCE-3241</a>.
+ Major bug reported by Devaraj K and fixed by Amar Kamat <br>
+ <b>(Rumen)TraceBuilder throws IllegalArgumentException</b><br>
+ <blockquote>Rumen is fixed to ignore the AMRestartedEvent.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3240">MAPREDUCE-3240</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Hitesh Shah (mrv2 , nodemanager)<br>
+ <b>NM should send a SIGKILL for completed containers also</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3239">MAPREDUCE-3239</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (mrv2)<br>
+ <b>Use new createSocketAddr API in MRv2 to give better error messages on misconfig</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3237">MAPREDUCE-3237</a>.
+ Major improvement reported by Tom White and fixed by Tom White (client)<br>
+ <b>Move LocalJobRunner to hadoop-mapreduce-client-core module</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3233">MAPREDUCE-3233</a>.
+ Blocker sub-task reported by Karam Singh and fixed by Mahadev konar (mrv2)<br>
+ <b>AM fails to restart when first AM is killed</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3228">MAPREDUCE-3228</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2)<br>
+ <b>MR AM hangs when one node goes bad</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3226">MAPREDUCE-3226</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2 , task)<br>
+ <b>Few reduce tasks hanging in a gridmix-run</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3220">MAPREDUCE-3220</a>.
+ Minor sub-task reported by Hitesh Shah and fixed by Devaraj K (mrv2 , test)<br>
+ <b>ant test TestCombineOutputCollector failing on trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3212">MAPREDUCE-3212</a>.
+ Minor bug reported by Bhallamudi Venkata Siva Kamesh and fixed by Bhallamudi Venkata Siva Kamesh (mrv2)<br>
+ <b>Message displays while executing yarn command should be proper</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3209">MAPREDUCE-3209</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (build , mrv2)<br>
+ <b>Jenkins reports 160 FindBugs warnings</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3208">MAPREDUCE-3208</a>.
+ Minor bug reported by liangzhaowang and fixed by liangzhaowang (mrv2)<br>
+ <b>NPE while flushing TaskLogAppender</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3205">MAPREDUCE-3205</a>.
+ Blocker improvement reported by Todd Lipcon and fixed by Todd Lipcon (mrv2 , nodemanager)<br>
+ <b>MR2 memory limits should be pmem, not vmem</b><br>
+ <blockquote>Resource limits are now expressed and enforced in terms of physical memory, rather than virtual memory. The virtual memory limit is set as a configurable multiple of the physical limit. The NodeManager's memory usage is now configured in units of MB rather than GB.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3204">MAPREDUCE-3204</a>.
+ Major bug reported by Suresh Srinivas and fixed by Alejandro Abdelnur (build)<br>
+ <b>mvn site:site fails on MapReduce</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3203">MAPREDUCE-3203</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>Fix some javac warnings in MRAppMaster.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3199">MAPREDUCE-3199</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2 , test)<br>
+ <b>TestJobMonitorAndPrint is broken on trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3198">MAPREDUCE-3198</a>.
+ Trivial bug reported by Hitesh Shah and fixed by Arun C Murthy (mrv2)<br>
+ <b>Change mode for hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/mock-container-executor to 755 </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3197">MAPREDUCE-3197</a>.
+ Major bug reported by Anupam Seth and fixed by Mahadev konar (mrv2)<br>
+ <b>TestMRClientService failing on building clean checkout of branch 0.23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3196">MAPREDUCE-3196</a>.
+ Major bug reported by Arun C Murthy and fixed by Arun C Murthy (mrv2)<br>
+ <b>TestLinuxContainerExecutorWithMocks fails on Mac OSX</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3192">MAPREDUCE-3192</a>.
+ Major bug reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Fix Javadoc warning in JobClient.java and Cluster.java</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3190">MAPREDUCE-3190</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (mrv2)<br>
+ <b>bin/yarn should barf early if HADOOP_COMMON_HOME or HADOOP_HDFS_HOME are not set</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3189">MAPREDUCE-3189</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (mrv2)<br>
+ <b>Add link decoration back to MR2's CSS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3188">MAPREDUCE-3188</a>.
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon (mrv2)<br>
+ <b>Lots of errors in logs when daemon startup fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3187">MAPREDUCE-3187</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (mrv2)<br>
+ <b>Add names for various unnamed threads in MR2</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3186">MAPREDUCE-3186</a>.
+ Blocker bug reported by Ramgopal N and fixed by Eric Payne (mrv2)<br>
+ <b>User jobs are getting hanged if the Resource manager process goes down and comes up while job is getting executed.</b><br>
+ <blockquote>New Yarn configuration property:
+
+Name: yarn.app.mapreduce.am.scheduler.connection.retries
+Description: Number of times AM should retry to contact RM if connection is lost.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3185">MAPREDUCE-3185</a>.
+ Critical bug reported by Mahadev konar and fixed by Jonathan Eagles (mrv2)<br>
+ <b>RM Web UI does not sort the columns in some cases.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3183">MAPREDUCE-3183</a>.
+ Trivial bug reported by Hitesh Shah and fixed by Hitesh Shah (build)<br>
+ <b>hadoop-assemblies/src/main/resources/assemblies/hadoop-mapreduce-dist.xml missing license header</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3181">MAPREDUCE-3181</a>.
+ Blocker bug reported by Anupam Seth and fixed by Arun C Murthy (mrv2)<br>
+ <b>Terasort fails with Kerberos exception on secure cluster</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3179">MAPREDUCE-3179</a>.
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2 , test)<br>
+ <b>Incorrect exit code for hadoop-mapreduce-test tests when exception thrown</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3176">MAPREDUCE-3176</a>.
+ Blocker bug reported by Ravi Prakash and fixed by Hitesh Shah (mrv2 , test)<br>
+ <b>ant mapreduce tests are timing out</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3175">MAPREDUCE-3175</a>.
+ Blocker sub-task reported by Thomas Graves and fixed by Jonathan Eagles (mrv2)<br>
+ <b>Yarn httpservers not created with access Control lists</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3171">MAPREDUCE-3171</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>normalize nodemanager native code compilation with common/hdfs native</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3170">MAPREDUCE-3170</a>.
+ Critical bug reported by Mahadev konar and fixed by Hitesh Shah (build , mrv1 , mrv2)<br>
+ <b>Trunk nightly commit builds are failing.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3167">MAPREDUCE-3167</a>.
+ Minor bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>container-executor is not being packaged with the assembly target.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3166">MAPREDUCE-3166</a>.
+ Major bug reported by Ravi Gummadi and fixed by Ravi Gummadi (tools/rumen)<br>
+ <b>Make Rumen use job history api instead of relying on current history file name format</b><br>
+ <blockquote>Makes Rumen use job history api instead of relying on current history file name format.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3165">MAPREDUCE-3165</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Todd Lipcon (applicationmaster , mrv2)<br>
+ <b>Ensure logging option is set on child command line</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3163">MAPREDUCE-3163</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Mahadev konar (job submission , mrv2)<br>
+ <b>JobClient spews errors when killing MR2 job</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3162">MAPREDUCE-3162</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (mrv2 , nodemanager)<br>
+ <b>Separate application-init and container-init event types in NM's ApplicationImpl FSM</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3161">MAPREDUCE-3161</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (mrv2)<br>
+ <b>Improve javadoc and fix some typos in MR2 code</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3159">MAPREDUCE-3159</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Todd Lipcon (mrv2)<br>
+ <b>DefaultContainerExecutor removes appcache dir on every localization</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3158">MAPREDUCE-3158</a>.
+ Major bug reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>Fix trunk build failures</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3157">MAPREDUCE-3157</a>.
+ Major bug reported by Ravi Gummadi and fixed by Ravi Gummadi (tools/rumen)<br>
+ <b>Rumen TraceBuilder is skipping analyzing 0.20 history files</b><br>
+ <blockquote>Fixes TraceBuilder to handle 0.20 history file names also.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3154">MAPREDUCE-3154</a>.
+ Major improvement reported by Abhijit Suresh Shingate and fixed by Abhijit Suresh Shingate (client , mrv2)<br>
+ <b>Validate the Jobs Output Specification as the first statement in JobSubmitter.submitJobInternal(Job, Cluster) method</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3153">MAPREDUCE-3153</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Mahadev konar (mrv2 , test)<br>
+ <b>TestFileOutputCommitter.testFailAbort() is failing on trunk on Jenkins</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3148">MAPREDUCE-3148</a>.
+ Blocker sub-task reported by Arun C Murthy and fixed by Arun C Murthy (mrv2)<br>
+ <b>Port MAPREDUCE-2702 to old mapred api</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3146">MAPREDUCE-3146</a>.
+ Critical sub-task reported by Vinod Kumar Vavilapalli and fixed by Siddharth Seth (mrv2 , nodemanager)<br>
+ <b>Add a MR specific command line to dump logs for a given TaskAttemptID</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3144">MAPREDUCE-3144</a>.
+ Critical sub-task reported by Vinod Kumar Vavilapalli and fixed by Siddharth Seth (mrv2)<br>
+ <b>Augment JobHistory to include information needed for serving aggregated logs.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3143">MAPREDUCE-3143</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by (mrv2 , nodemanager)<br>
+ <b>Complete aggregation of user-logs spit out by containers onto DFS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3141">MAPREDUCE-3141</a>.
+ Blocker sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2 , security)<br>
+ <b>Yarn+MR secure mode is broken, uncovered after MAPREDUCE-3056</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3140">MAPREDUCE-3140</a>.
+ Major bug reported by Bhallamudi Venkata Siva Kamesh and fixed by Subroto Sanyal (mrv2)<br>
+ <b>Invalid JobHistory URL for failed applications</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3138">MAPREDUCE-3138</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Owen O'Malley (client , mrv2)<br>
+ <b>Allow for applications to deal with MAPREDUCE-954</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3137">MAPREDUCE-3137</a>.
+ Trivial sub-task reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>Fix broken merge of MR-2719 to 0.23 branch for the distributed shell test case </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3136">MAPREDUCE-3136</a>.
+ Blocker sub-task reported by Arun C Murthy and fixed by Arun C Murthy (documentation , mrv2)<br>
+ <b>Add docs for setting up real-world MRv2 clusters</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3134">MAPREDUCE-3134</a>.
+ Blocker sub-task reported by Arun C Murthy and fixed by Arun C Murthy (documentation , mrv2 , scheduler)<br>
+ <b>Add documentation for CapacityScheduler</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3133">MAPREDUCE-3133</a>.
+ Major improvement reported by Jonathan Eagles and fixed by Jonathan Eagles (build)<br>
+ <b>Running a set of methods in a Single Test Class</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3127">MAPREDUCE-3127</a>.
+ Blocker sub-task reported by Amol Kekre and fixed by Arun C Murthy (mrv2 , resourcemanager)<br>
+ <b>Unable to restrict users based on resourcemanager.admin.acls value set</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3126">MAPREDUCE-3126</a>.
+ Blocker bug reported by Thomas Graves and fixed by Arun C Murthy (mrv2)<br>
+ <b>mr job stuck because reducers using all slots and mapper isn't scheduled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3125">MAPREDUCE-3125</a>.
+ Critical bug reported by Thomas Graves and fixed by Hitesh Shah (mrv2)<br>
+ <b>app master web UI shows reduce task progress 100% even though reducers not complete and state running/scheduled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3124">MAPREDUCE-3124</a>.
+ Blocker bug reported by Thomas Graves and fixed by John George (mrv2)<br>
+ <b>mapper failed with failed to load native libs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3123">MAPREDUCE-3123</a>.
+ Blocker bug reported by Thomas Graves and fixed by Hitesh Shah (mrv2)<br>
+ <b>Symbolic links with special chars causing container/task.sh to fail</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3114">MAPREDUCE-3114</a>.
+ Major bug reported by Subroto Sanyal and fixed by Subroto Sanyal (mrv2)<br>
+ <b>Invalid ApplicationMaster URL in Applications Page</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3113">MAPREDUCE-3113</a>.
+ Minor improvement reported by XieXianshan and fixed by XieXianshan (mrv2)<br>
+ <b>the scripts yarn-daemon.sh and yarn are not working properly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3112">MAPREDUCE-3112</a>.
+ Major bug reported by Eric Yang and fixed by Eric Yang (contrib/streaming)<br>
+ <b>Calling hadoop cli inside mapreduce job leads to errors</b><br>
+ <blockquote>Removed inheritance of certain server environment variables (HADOOP_OPTS and HADOOP_ROOT_LOGGER) in task attempt process.
+</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3110">MAPREDUCE-3110</a>.
+ Major bug reported by Devaraj K and fixed by Vinod Kumar Vavilapalli (mrv2 , test)<br>
+ <b>TestRPC.testUnknownCall() is failing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3104">MAPREDUCE-3104</a>.
+ Blocker sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2 , resourcemanager , security)<br>
+ <b>Implement Application ACLs, Queue ACLs and their interaction</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3099">MAPREDUCE-3099</a>.
+ Major sub-task reported by Mahadev konar and fixed by Mahadev konar <br>
+ <b>Add docs for setting up a single node MRv2 cluster.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3098">MAPREDUCE-3098</a>.
+ Blocker sub-task reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>Report Application status as well as ApplicationMaster status in GetApplicationReportResponse </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3095">MAPREDUCE-3095</a>.
+ Major bug reported by John George and fixed by John George (mrv2)<br>
+ <b>fairscheduler ivy including wrong version for hdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3092">MAPREDUCE-3092</a>.
+ Minor bug reported by Devaraj K and fixed by Devaraj K (mrv2)<br>
+ <b>Remove JOB_ID_COMPARATOR usage in JobHistory.java</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3090">MAPREDUCE-3090</a>.
+ Major improvement reported by Arun C Murthy and fixed by Arun C Murthy (applicationmaster , mrv2)<br>
+ <b>Change MR AM to use ApplicationAttemptId rather than <applicationId, startCount> everywhere</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3087">MAPREDUCE-3087</a>.
+ Critical bug reported by Ravi Prakash and fixed by Ravi Prakash (mrv2)<br>
+ <b>CLASSPATH not the same after MAPREDUCE-2880</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3081">MAPREDUCE-3081</a>.
+ Major bug reported by vitthal (Suhas) Gogate and fixed by (contrib/vaidya)<br>
+ <b>Change the name format for hadoop core and vaidya jar to be hadoop-{core/vaidya}-{version}.jar in vaidya.sh</b><br>
+ <blockquote>contrib/vaidya/bin/vaidya.sh script fixed to use appropriate jars and classpath </blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3078">MAPREDUCE-3078</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2 , resourcemanager)<br>
+ <b>Application's progress isn't updated from AM to RM.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3073">MAPREDUCE-3073</a>.
+ Blocker bug reported by Mahadev konar and fixed by Mahadev konar <br>
+ <b>Build failure for MRv1 caused due to changes to MRConstants.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3071">MAPREDUCE-3071</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>app master configuration web UI link under the Job menu opens up application menu</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3070">MAPREDUCE-3070</a>.
+ Blocker bug reported by Ravi Teja Ch N V and fixed by Devaraj K (mrv2 , nodemanager)<br>
+ <b>NM not able to register with RM after NM restart</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3068">MAPREDUCE-3068</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Chris Riccomini (mrv2)<br>
+ <b>Should set MALLOC_ARENA_MAX for all YARN daemons and AMs/Containers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3067">MAPREDUCE-3067</a>.
+ Blocker bug reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>Container exit status not set properly to launched process's exit code on successful completion of process</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3066">MAPREDUCE-3066</a>.
+ Major bug reported by Chris Riccomini and fixed by Chris Riccomini (mrv2 , nodemanager)<br>
+ <b>YARN NM fails to start</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3064">MAPREDUCE-3064</a>.
+ Blocker bug reported by Thomas Graves and fixed by Venu Gopala Rao <br>
+ <b>27 unit test failures with Invalid "mapreduce.jobtracker.address" configuration value for JobTracker: "local"</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3062">MAPREDUCE-3062</a>.
+ Major bug reported by Chris Riccomini and fixed by Chris Riccomini (mrv2 , nodemanager , resourcemanager)<br>
+ <b>YARN NM/RM fail to start</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3059">MAPREDUCE-3059</a>.
+ Blocker bug reported by Karam Singh and fixed by Devaraj K (mrv2)<br>
+ <b>QueueMetrics do not have metrics for aggregate containers-allocated and aggregate containers-released</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3058">MAPREDUCE-3058</a>.
+ Critical bug reported by Karam Singh and fixed by Vinod Kumar Vavilapalli (contrib/gridmix , mrv2)<br>
+ <b>Sometimes task keeps on running while its Syslog says that it is shutdown</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3057">MAPREDUCE-3057</a>.
+ Blocker bug reported by Karam Singh and fixed by Eric Payne (jobhistoryserver , mrv2)<br>
+ <b>Job History Server goes of OutOfMemory with 1200 Jobs and Heap Size set to 10 GB</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3056">MAPREDUCE-3056</a>.
+ Blocker bug reported by Devaraj K and fixed by Devaraj K (applicationmaster , mrv2)<br>
+ <b>Jobs are failing when those are submitted by other users</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3055">MAPREDUCE-3055</a>.
+ Minor bug reported by Hitesh Shah and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>Simplify parameter passing to Application Master from Client. SImplify approach to pass info such appId, ClusterTimestamp and failcount required by App Master.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3054">MAPREDUCE-3054</a>.
+ Blocker bug reported by Siddharth Seth and fixed by Mahadev konar (mrv2)<br>
+ <b>Unable to kill submitted jobs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3053">MAPREDUCE-3053</a>.
+ Major bug reported by Chris Riccomini and fixed by Vinod Kumar Vavilapalli (mrv2 , resourcemanager)<br>
+ <b>YARN Protobuf RPC Failures in RM</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3050">MAPREDUCE-3050</a>.
+ Blocker bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2 , resourcemanager)<br>
+ <b>YarnScheduler needs to expose Resource Usage Information</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3048">MAPREDUCE-3048</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (build)<br>
+ <b>Fix test-patch to run tests via "mvn clean install test"</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3044">MAPREDUCE-3044</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Mahadev konar (mrv2)<br>
+ <b>Pipes jobs stuck without making progress</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3042">MAPREDUCE-3042</a>.
+ Major bug reported by Chris Riccomini and fixed by Chris Riccomini (mrv2 , resourcemanager)<br>
+ <b>YARN RM fails to start</b><br>
+ <blockquote>Simple typo fix to allow ResourceManager to start instead of fail</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3041">MAPREDUCE-3041</a>.
+ Blocker bug reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3040">MAPREDUCE-3040</a>.
+ Major bug reported by Thomas Graves and fixed by Arun C Murthy (mrv2)<br>
+ <b>TestMRJobs, TestMRJobsWithHistoryService, TestMROldApiJobs fail</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3038">MAPREDUCE-3038</a>.
+ Blocker bug reported by Thomas Graves and fixed by Jeffrey Naisbitt (mrv2)<br>
+ <b>job history server not starting because conf() missing HsController</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3036">MAPREDUCE-3036</a>.
+ Blocker bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>Some of the Resource Manager memory metrics go negative.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3035">MAPREDUCE-3035</a>.
+ Critical bug reported by Karam Singh and fixed by chackaravarthy (mrv2)<br>
+ <b>MR V2 jobhistory does not contain rack information</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3033">MAPREDUCE-3033</a>.
+ Blocker bug reported by Karam Singh and fixed by Hitesh Shah (job submission , mrv2)<br>
+ <b>JobClient requires mapreduce.jobtracker.address config even when mapreduce.framework.name is set to yarn</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3032">MAPREDUCE-3032</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Devaraj K (applicationmaster , mrv2)<br>
+ <b>JobHistory doesn't have error information from failed tasks</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3031">MAPREDUCE-3031</a>.
+ Blocker bug reported by Karam Singh and fixed by Siddharth Seth (mrv2)<br>
+ <b>Job Client goes into infinite loop when we kill AM</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3030">MAPREDUCE-3030</a>.
+ Blocker bug reported by Devaraj K and fixed by Devaraj K (mrv2 , resourcemanager)<br>
+ <b>RM is not processing heartbeat and continuously giving the message 'Node not found rebooting'</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3028">MAPREDUCE-3028</a>.
+ Blocker bug reported by Mohammad Kamrul Islam and fixed by Ravi Prakash (mrv2)<br>
+ <b>Support job end notification in .next /0.23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3023">MAPREDUCE-3023</a>.
+ Major bug reported by Ravi Prakash and fixed by Ravi Prakash (mrv2)<br>
+ <b>Queue state is not being translated properly (is always assumed to be running)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3021">MAPREDUCE-3021</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>all yarn webapps use same base name of "yarn/"</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3020">MAPREDUCE-3020</a>.
+ Major bug reported by chackaravarthy and fixed by chackaravarthy (jobhistoryserver)<br>
+ <b>Node link in reduce task attempt page is not working [Job History Page]</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3018">MAPREDUCE-3018</a>.
+ Blocker bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>Streaming jobs with -file option fail to run.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3017">MAPREDUCE-3017</a>.
+ Blocker bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>The Web UI shows FINISHED for killed/successful/failed jobs.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3014">MAPREDUCE-3014</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Rename and invert logic of '-cbuild' profile to 'native' and off by default</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3013">MAPREDUCE-3013</a>.
+ Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2 , security)<br>
+ <b>Remove YarnConfiguration.YARN_SECURITY_INFO</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3007">MAPREDUCE-3007</a>.
+ Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (jobhistoryserver , mrv2)<br>
+ <b>JobClient cannot talk to JobHistory server in secure mode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3006">MAPREDUCE-3006</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2)<br>
+ <b>MapReduce AM exits prematurely before completely writing and closing the JobHistory file</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3005">MAPREDUCE-3005</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Arun C Murthy (mrv2)<br>
+ <b>MR app hangs because of a NPE in ResourceManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3004">MAPREDUCE-3004</a>.
+ Minor bug reported by Hitesh Shah and fixed by Hitesh Shah (mrv2)<br>
+ <b>sort example fails in shuffle/reduce stage as it assumes a local job by default </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3003">MAPREDUCE-3003</a>.
+ Major bug reported by Tom White and fixed by Alejandro Abdelnur (build)<br>
+ <b>Publish MR JARs to Maven snapshot repository</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3001">MAPREDUCE-3001</a>.
+ Blocker improvement reported by Robert Joseph Evans and fixed by Robert Joseph Evans (jobhistoryserver , mrv2)<br>
+ <b>Map Reduce JobHistory and AppMaster UI should have ability to display task specific counters.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2999">MAPREDUCE-2999</a>.
+ Critical bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>hadoop.http.filter.initializers not working properly on yarn UI</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2998">MAPREDUCE-2998</a>.
+ Critical bug reported by Jeffrey Naisbitt and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>Failing to contact Am/History for jobs: java.io.EOFException in DataInputStream</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2997">MAPREDUCE-2997</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (applicationmaster , mrv2)<br>
+ <b>MR task fails before launch itself with an NPE in ContainerLauncher</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2996">MAPREDUCE-2996</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Jonathan Eagles (jobhistoryserver , mrv2)<br>
+ <b>Log uberized information into JobHistory and use the same via CompletedJob</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2995">MAPREDUCE-2995</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>MR AM crashes when a container-launch hangs on a faulty NM</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2994">MAPREDUCE-2994</a>.
+ Major bug reported by Devaraj K and fixed by Devaraj K (mrv2 , resourcemanager)<br>
+ <b>Parse Error is coming for App ID when we click application link on the RM UI</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2991">MAPREDUCE-2991</a>.
+ Major bug reported by Priyo Mustafi and fixed by Priyo Mustafi (scheduler)<br>
+ <b>queueinfo.jsp fails to show queue status if any Capacity scheduler queue name has dash/hiphen in it.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2990">MAPREDUCE-2990</a>.
+ Blocker improvement reported by Mahadev konar and fixed by Subroto Sanyal (mrv2)<br>
+ <b>Health Report on Resource Manager UI is null if the NM's are all healthy.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2989">MAPREDUCE-2989</a>.
+ Critical sub-task reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>JobHistory should link to task logs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2988">MAPREDUCE-2988</a>.
+ Critical sub-task reported by Eric Payne and fixed by Robert Joseph Evans (mrv2 , security , test)<br>
+ <b>Reenable TestLinuxContainerExecutor reflecting the current NM code. </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2987">MAPREDUCE-2987</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>RM UI display logged in user as null</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2986">MAPREDUCE-2986</a>.
+ Critical task reported by Anupam Seth and fixed by Anupam Seth (mrv2 , test)<br>
+ <b>Multiple node managers support for the MiniYARNCluster</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2985">MAPREDUCE-2985</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>findbugs error in ResourceLocalizationService.handle(LocalizationEvent)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2984">MAPREDUCE-2984</a>.
+ Minor bug reported by Devaraj K and fixed by Devaraj K (mrv2 , nodemanager)<br>
+ <b>Throwing NullPointerException when we open the container page</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2979">MAPREDUCE-2979</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>Remove ClientProtocolProvider configuration under mapreduce-client-core</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2977">MAPREDUCE-2977</a>.
+ Blocker sub-task reported by Owen O'Malley and fixed by Arun C Murthy (mrv2 , resourcemanager , security)<br>
+ <b>ResourceManager needs to renew and cancel tokens associated with a job</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2975">MAPREDUCE-2975</a>.
+ Blocker bug reported by Mahadev konar and fixed by Mahadev konar <br>
+ <b>ResourceManager Delegate is not getting initialized with yarn-site.xml as default configuration.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2971">MAPREDUCE-2971</a>.
+ Blocker bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>ant build mapreduce fails protected access jc.displayJobList(jobs);</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2970">MAPREDUCE-2970</a>.
+ Major bug reported by Venu Gopala Rao and fixed by Venu Gopala Rao (job submission , mrv2)<br>
+ <b>Null Pointer Exception while submitting a Job, If mapreduce.framework.name property is not set.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2966">MAPREDUCE-2966</a>.
+ Major improvement reported by Abhijit Suresh Shingate and fixed by Abhijit Suresh Shingate (applicationmaster , jobhistoryserver , nodemanager , resourcemanager)<br>
+ <b>Add ShutDown hooks for MRV2 processes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2965">MAPREDUCE-2965</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Siddharth Seth (mrv2)<br>
+ <b>Streamline hashCode(), equals(), compareTo() and toString() for all IDs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2963">MAPREDUCE-2963</a>.
+ Critical bug reported by Mahadev konar and fixed by Siddharth Seth <br>
+ <b>TestMRJobs hangs waiting to connect to history server.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2961">MAPREDUCE-2961</a>.
+ Blocker improvement reported by Mahadev konar and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>Increase the default threadpool size for container launching in the application master.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2958">MAPREDUCE-2958</a>.
+ Critical bug reported by Thomas Graves and fixed by Arun C Murthy (mrv2)<br>
+ <b>mapred-default.xml not merged from mr279</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2954">MAPREDUCE-2954</a>.
+ Critical bug reported by Vinod Kumar Vavilapalli and fixed by Siddharth Seth (mrv2)<br>
+ <b>Deadlock in NM with threads racing for ApplicationAttemptId</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2953">MAPREDUCE-2953</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Thomas Graves (mrv2 , resourcemanager)<br>
+ <b>JobClient fails due to a race in RM, removes staged files and in turn crashes MR AM</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2952">MAPREDUCE-2952</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Arun C Murthy (mrv2 , resourcemanager)<br>
+ <b>Application failure diagnostics are not consumed in a couple of cases</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2949">MAPREDUCE-2949</a>.
+ Major bug reported by Ravi Teja Ch N V and fixed by Ravi Teja Ch N V (mrv2 , nodemanager)<br>
+ <b>NodeManager in a inconsistent state if a service startup fails.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2948">MAPREDUCE-2948</a>.
+ Major bug reported by Milind Bhandarkar and fixed by Mahadev konar (contrib/streaming)<br>
+ <b>Hadoop streaming test failure, post MR-2767</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2947">MAPREDUCE-2947</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>Sort fails on YARN+MR with lots of task failures</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2938">MAPREDUCE-2938</a>.
+ Trivial bug reported by Arun C Murthy and fixed by Arun C Murthy (mrv2 , scheduler)<br>
+ <b>Missing log stmt for app submission fail CS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2937">MAPREDUCE-2937</a>.
+ Critical bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>Errors in Application failures are not shown in the client trace.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2936">MAPREDUCE-2936</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Vinod Kumar Vavilapalli <br>
+ <b>Contrib Raid compilation broken after HDFS-1620</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2933">MAPREDUCE-2933</a>.
+ Blocker sub-task reported by Arun C Murthy and fixed by Arun C Murthy (applicationmaster , mrv2 , nodemanager , resourcemanager)<br>
+ <b>Change allocate call to return ContainerStatus for completed containers rather than Container </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2930">MAPREDUCE-2930</a>.
+ Major improvement reported by Sharad Agarwal and fixed by Binglin Chang (mrv2)<br>
+ <b>Generate state graph from the State Machine Definition</b><br>
+ <blockquote>Generate state graph from State Machine Definition</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2925">MAPREDUCE-2925</a>.
+ Major bug reported by Devaraj K and fixed by Devaraj K (mrv2)<br>
+ <b>job -status <JOB_ID> is giving continuously info message for completed jobs on the console</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2917">MAPREDUCE-2917</a>.
+ Major bug reported by Arun C Murthy and fixed by Arun C Murthy (mrv2 , resourcemanager)<br>
+ <b>Corner case in container reservations</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2916">MAPREDUCE-2916</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar <br>
+ <b>Ivy build for MRv1 fails with bad organization for common daemon.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2913">MAPREDUCE-2913</a>.
+ Critical bug reported by Robert Joseph Evans and fixed by Jonathan Eagles (mrv2 , test)<br>
+ <b>TestMRJobs.testFailingMapper does not assert the correct thing.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2909">MAPREDUCE-2909</a>.
+ Major sub-task reported by Arun C Murthy and fixed by Arun C Murthy (documentation , mrv2)<br>
+ <b>Docs for remaining records in yarn-api</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2908">MAPREDUCE-2908</a>.
+ Critical bug reported by Mahadev konar and fixed by Vinod Kumar Vavilapalli (mrv2)<br>
+ <b>Fix findbugs warnings in Map Reduce.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2907">MAPREDUCE-2907</a>.
+ Major bug reported by Ravi Prakash and fixed by Ravi Prakash (mrv2 , resourcemanager)<br>
+ <b>ResourceManager logs filled with [INFO] debug messages from org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2904">MAPREDUCE-2904</a>.
+ Major bug reported by Sharad Agarwal and fixed by Sharad Agarwal <br>
+ <b>HDFS jars added incorrectly to yarn classpath</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2899">MAPREDUCE-2899</a>.
+ Major sub-task reported by Arun C Murthy and fixed by Arun C Murthy (mrv2 , resourcemanager)<br>
+ <b>Replace major parts of ApplicationSubmissionContext with a ContainerLaunchContext</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2898">MAPREDUCE-2898</a>.
+ Major sub-task reported by Arun C Murthy and fixed by Arun C Murthy (documentation , mrv2)<br>
+ <b>Docs for core protocols in yarn-api - ContainerManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2897">MAPREDUCE-2897</a>.
+ Major sub-task reported by Arun C Murthy and fixed by Arun C Murthy (documentation , mrv2)<br>
+ <b>Docs for core protocols in yarn-api - ClientRMProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2896">MAPREDUCE-2896</a>.
+ Major sub-task reported by Arun C Murthy and fixed by Arun C Murthy (mrv2)<br>
+ <b>Remove all apis other than getters and setters in all org/apache/hadoop/yarn/api/records/*</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2894">MAPREDUCE-2894</a>.
+ Blocker improvement reported by Arun C Murthy and fixed by (mrv2)<br>
+ <b>Improvements to YARN apis</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2893">MAPREDUCE-2893</a>.
+ Trivial improvement reported by Liang-Chi Hsieh and fixed by Liang-Chi Hsieh (client)<br>
+ <b>Removing duplicate service provider in hadoop-mapreduce-client-jobclient</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2891">MAPREDUCE-2891</a>.
+ Major sub-task reported by Arun C Murthy and fixed by Arun C Murthy (documentation , mrv2)<br>
+ <b>Docs for core protocols in yarn-api - AMRMProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2889">MAPREDUCE-2889</a>.
+ Critical sub-task reported by Arun C Murthy and fixed by Hitesh Shah (documentation , mrv2)<br>
+ <b>Add docs for writing new application frameworks</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2886">MAPREDUCE-2886</a>.
+ Critical bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>Fix Javadoc warnings in MapReduce.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2885">MAPREDUCE-2885</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Arun C Murthy <br>
+ <b>mapred-config.sh doesn't look for $HADOOP_COMMON_HOME/libexec/hadoop-config.sh</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2882">MAPREDUCE-2882</a>.
+ Minor bug reported by Todd Lipcon and fixed by Todd Lipcon (test)<br>
+ <b>TestLineRecordReader depends on ant jars</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2881">MAPREDUCE-2881</a>.
+ Major bug reported by Giridharan Kesavan and fixed by Giridharan Kesavan (build)<br>
+ <b>mapreduce ant compilation fails "java.lang.IllegalStateException: impossible to get artifacts"</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2880">MAPREDUCE-2880</a>.
+ Blocker improvement reported by Luke Lu and fixed by Arun C Murthy (mrv2)<br>
+ <b>Fix classpath construction for MRv2</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2879">MAPREDUCE-2879</a>.
+ Major bug reported by Arun C Murthy and fixed by Arun C Murthy <br>
+ <b>Change mrv2 version to be 0.23.0-SNAPSHOT</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2877">MAPREDUCE-2877</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar <br>
+ <b>Add missing Apache license header in some files in MR and also add the rat plugin to the poms.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2876">MAPREDUCE-2876</a>.
+ Critical bug reported by Robert Joseph Evans and fixed by Anupam Seth (mrv2)<br>
+ <b>ContainerAllocationExpirer appears to use the incorrect configs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2874">MAPREDUCE-2874</a>.
+ Major bug reported by Thomas Graves and fixed by Eric Payne (mrv2)<br>
+ <b>ApplicationId printed in 2 different formats and has 2 different toString routines that are used</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2868">MAPREDUCE-2868</a>.
+ Major bug reported by Thomas Graves and fixed by Mahadev konar (build)<br>
+ <b>ant build broken in hadoop-mapreduce dir</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2867">MAPREDUCE-2867</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar <br>
+ <b>Remove Unused TestApplicaitonCleanup in resourcemanager/applicationsmanager.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2864">MAPREDUCE-2864</a>.
+ Major improvement reported by Robert Joseph Evans and fixed by Robert Joseph Evans (jobhistoryserver , mrv2 , nodemanager , resourcemanager)<br>
+ <b>Renaming of configuration property names in yarn</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2860">MAPREDUCE-2860</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar (mrv2)<br>
+ <b>Fix log4j logging in the maven test cases.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2859">MAPREDUCE-2859</a>.
+ Major bug reported by Giridharan Kesavan and fixed by Giridharan Kesavan <br>
+ <b>mapreduce trunk is broken with eclipse plugin contrib</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2858">MAPREDUCE-2858</a>.
+ Blocker sub-task reported by Luke Lu and fixed by Robert Joseph Evans (applicationmaster , mrv2 , security)<br>
+ <b>MRv2 WebApp Security</b><br>
+ <blockquote>A new server has been added to yarn. It is a web proxy that sits in front of the AM web UI. The server is controlled by the yarn.web-proxy.address config. If that config is set, and it points to an address that is different then the RM web interface then a separate proxy server needs to be launched.
+
+This can be done by running
+
+yarn-daemon.sh start proxyserver
+
+If a separate proxy server is needed other configs also may need to be set, if security is enabled.
+yarn.web-proxy.principal
+yarn.web-proxy.keytab
+
+The proxy server is stateless and should be able to support a VIP or other load balancing sitting in front of multiple instances of this server.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2854">MAPREDUCE-2854</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves <br>
+ <b>update INSTALL with config necessary run mapred on yarn</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2848">MAPREDUCE-2848</a>.
+ Major improvement reported by Luke Lu and fixed by Luke Lu <br>
+ <b>Upgrade avro to 1.5.2</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2846">MAPREDUCE-2846</a>.
+ Blocker bug reported by Allen Wittenauer and fixed by Owen O'Malley (task , task-controller , tasktracker)<br>
+ <b>a small % of all tasks fail with DefaultTaskController</b><br>
+ <blockquote>Fixed a race condition in writing the log index file that caused tasks to 'fail'.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2844">MAPREDUCE-2844</a>.
+ Trivial bug reported by Ramya Sunil and fixed by Ravi Teja Ch N V (mrv2)<br>
+ <b>[MR-279] Incorrect node ID info </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2843">MAPREDUCE-2843</a>.
+ Major bug reported by Ramya Sunil and fixed by Abhijit Suresh Shingate (mrv2)<br>
+ <b>[MR-279] Node entries on the RM UI are not sortable</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2840">MAPREDUCE-2840</a>.
+ Minor bug reported by Thomas Graves and fixed by Jonathan Eagles (mrv2)<br>
+ <b>mr279 TestUberAM.testSleepJob test fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2839">MAPREDUCE-2839</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth <br>
+ <b>MR Jobs fail on a secure cluster with viewfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2821">MAPREDUCE-2821</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Mahadev konar (mrv2)<br>
+ <b>[MR-279] Missing fields in job summary logs </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2808">MAPREDUCE-2808</a>.
+ Minor bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>pull MAPREDUCE-2797 into mr279 branch</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2807">MAPREDUCE-2807</a>.
+ Major sub-task reported by Sharad Agarwal and fixed by Sharad Agarwal (applicationmaster , mrv2 , resourcemanager)<br>
+ <b>MR-279: AM restart does not work after RM refactor</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2805">MAPREDUCE-2805</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (contrib/raid)<br>
+ <b>Update RAID for HDFS-2241</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2802">MAPREDUCE-2802</a>.
+ Critical improvement reported by Ramya Sunil and fixed by Jonathan Eagles (mrv2)<br>
+ <b>[MR-279] Jobhistory filenames should have jobID to help in better parsing </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2800">MAPREDUCE-2800</a>.
+ Major bug reported by Ramya Sunil and fixed by Siddharth Seth (mrv2)<br>
+ <b>clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 in jhist files</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2797">MAPREDUCE-2797</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (contrib/raid , test)<br>
+ <b>Some java files cannot be compiled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2796">MAPREDUCE-2796</a>.
+ Major bug reported by Ramya Sunil and fixed by Devaraj K (mrv2)<br>
+ <b>[MR-279] Start time for all the apps is set to 0</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2794">MAPREDUCE-2794</a>.
+ Blocker bug reported by Ramya Sunil and fixed by John George (mrv2)<br>
+ <b>[MR-279] Incorrect metrics value for AvailableGB per queue per user</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2792">MAPREDUCE-2792</a>.
+ Blocker sub-task reported by Ramya Sunil and fixed by Vinod Kumar Vavilapalli (mrv2 , security)<br>
+ <b>[MR-279] Replace IP addresses with hostnames</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2791">MAPREDUCE-2791</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Devaraj K (mrv2)<br>
+ <b>[MR-279] Missing/incorrect info on job -status CLI </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2789">MAPREDUCE-2789</a>.
+ Major bug reported by Ramya Sunil and fixed by Eric Payne (mrv2)<br>
+ <b>[MR:279] Update the scheduling info on CLI</b><br>
+ <blockquote>"mapred/job -list" now contains map/reduce, container, and resource information.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2788">MAPREDUCE-2788</a>.
+ Major bug reported by Ahmed Radwan and fixed by Ahmed Radwan (mrv2)<br>
+ <b>Normalize requests in FifoScheduler.allocate to prevent NPEs later</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2783">MAPREDUCE-2783</a>.
+ Critical bug reported by Thomas Graves and fixed by Eric Payne (mrv2)<br>
+ <b>mr279 job history handling after killing application</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2782">MAPREDUCE-2782</a>.
+ Major test reported by Arun C Murthy and fixed by Arun C Murthy (mrv2)<br>
+ <b>MR-279: Unit (mockito) tests for CS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2781">MAPREDUCE-2781</a>.
+ Minor bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>mr279 RM application finishtime not set</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2779">MAPREDUCE-2779</a>.
+ Major bug reported by Ming Ma and fixed by Ming Ma (job submission)<br>
+ <b>JobSplitWriter.java can't handle large job.split file</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2776">MAPREDUCE-2776</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: Fix some of the yarn findbug warnings</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2775">MAPREDUCE-2775</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Devaraj K (mrv2)<br>
+ <b>[MR-279] Decommissioned node does not shutdown</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2774">MAPREDUCE-2774</a>.
+ Minor bug reported by Ramya Sunil and fixed by Venu Gopala Rao (mrv2)<br>
+ <b>[MR-279] Add a startup msg while starting RM/NM</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2773">MAPREDUCE-2773</a>.
+ Minor bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>[MR-279] server.api.records.NodeHealthStatus renamed but not updated in client NodeHealthStatus.java</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2772">MAPREDUCE-2772</a>.
+ Major bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>MR-279: mrv2 no longer compiles against trunk after common mavenization.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2767">MAPREDUCE-2767</a>.
+ Blocker bug reported by Milind Bhandarkar and fixed by Milind Bhandarkar (security)<br>
+ <b>Remove Linux task-controller from 0.22 branch</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2766">MAPREDUCE-2766</a>.
+ Blocker sub-task reported by Ramya Sunil and fixed by Hitesh Shah (mrv2)<br>
+ <b>[MR-279] Set correct permissions for files in dist cache</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2764">MAPREDUCE-2764</a>.
+ Major bug reported by Daryn Sharp and fixed by Owen O'Malley <br>
+ <b>Fix renewal of dfs delegation tokens</b><br>
+ <blockquote>Generalizes token renewal and canceling to a common interface and provides a plugin interface for adding renewers for new kinds of tokens. Hftp changed to store the tokens as HFTP and renew them over http.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2763">MAPREDUCE-2763</a>.
+ Major bug reported by Ramya Sunil and fixed by (mrv2)<br>
+ <b>IllegalArgumentException while using the dist cache</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2762">MAPREDUCE-2762</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Mahadev konar (mrv2)<br>
+ <b>[MR-279] - Cleanup staging dir after job completion</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2760">MAPREDUCE-2760</a>.
+ Minor bug reported by Todd Lipcon and fixed by Todd Lipcon (documentation)<br>
+ <b>mapreduce.jobtracker.split.metainfo.maxsize typoed in mapred-default.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2756">MAPREDUCE-2756</a>.
+ Minor bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (client , mrv2)<br>
+ <b>JobControl can drop jobs if an error occurs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2754">MAPREDUCE-2754</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Ravi Teja Ch N V (mrv2)<br>
+ <b>MR-279: AM logs are incorrectly going to stderr and error messages going incorrectly to stdout</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2751">MAPREDUCE-2751</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Siddharth Seth (mrv2)<br>
+ <b>[MR-279] Lot of local files left on NM after the app finish.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2749">MAPREDUCE-2749</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Thomas Graves (mrv2)<br>
+ <b>[MR-279] NM registers with RM even before it starts various servers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2747">MAPREDUCE-2747</a>.
+ Blocker sub-task reported by Vinod Kumar Vavilapalli and fixed by Robert Joseph Evans (mrv2 , nodemanager , security)<br>
+ <b>[MR-279] [Security] Cleanup LinuxContainerExecutor binary sources</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2746">MAPREDUCE-2746</a>.
+ Blocker sub-task reported by Vinod Kumar Vavilapalli and fixed by Arun C Murthy (mrv2 , security)<br>
+ <b>[MR-279] [Security] Yarn servers can't communicate with each other with hadoop.security.authorization set to true</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2741">MAPREDUCE-2741</a>.
+ Major task reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Make ant build system work with hadoop-common JAR generated by Maven</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2740">MAPREDUCE-2740</a>.
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>MultipleOutputs in new API creates needless TaskAttemptContexts</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2738">MAPREDUCE-2738</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>Missing cluster level stats on the RM UI</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2737">MAPREDUCE-2737</a>.
+ Major bug reported by Ramya Sunil and fixed by Siddharth Seth (mrv2)<br>
+ <b>Update the progress of jobs on client side</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2736">MAPREDUCE-2736</a>.
+ Major task reported by Eli Collins and fixed by Eli Collins (jobtracker , tasktracker)<br>
+ <b>Remove unused contrib components dependent on MR1</b><br>
+ <blockquote>The pre-MR2 MapReduce implementation (JobTracker, TaskTracer, etc) and contrib components are no longer supported. This implementation is currently supported in the 0.20.20x releases.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2735">MAPREDUCE-2735</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>MR279: finished applications should be added to an application summary log</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2732">MAPREDUCE-2732</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (test)<br>
+ <b>Some tests using FSNamesystem.LOG cannot be compiled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2727">MAPREDUCE-2727</a>.
+ Major bug reported by Jeffrey Naisbitt and fixed by Jeffrey Naisbitt (mrv2)<br>
+ <b>MR-279: SleepJob throws divide by zero exception when count = 0</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2726">MAPREDUCE-2726</a>.
+ Blocker improvement reported by Jeffrey Naisbitt and fixed by Jeffrey Naisbitt (mrv2)<br>
+ <b>MR-279: Add the jobFile to the web UI</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2719">MAPREDUCE-2719</a>.
+ Major new feature reported by Sharad Agarwal and fixed by Hitesh Shah (mrv2)<br>
+ <b>MR-279: Write a shell command application</b><br>
+ <blockquote>Adding a simple, DistributedShell application as an alternate framework to MapReduce and to act as an illustrative example for porting applications to YARN.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2716">MAPREDUCE-2716</a>.
+ Major bug reported by Jeffrey Naisbitt and fixed by Jeffrey Naisbitt (mrv2)<br>
+ <b>MR279: MRReliabilityTest job fails because of missing job-file.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2711">MAPREDUCE-2711</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (contrib/raid)<br>
+ <b>TestBlockPlacementPolicyRaid cannot be compiled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2710">MAPREDUCE-2710</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (client)<br>
+ <b>Update DFSClient.stringifyToken(..) in JobSubmitter.printTokens(..) for HDFS-2161</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2708">MAPREDUCE-2708</a>.
+ Blocker sub-task reported by Sharad Agarwal and fixed by Sharad Agarwal (applicationmaster , mrv2)<br>
+ <b>[MR-279] Design and implement MR Application Master recovery</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2707">MAPREDUCE-2707</a>.
+ Major improvement reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>ProtoOverHadoopRpcEngine without using TunnelProtocol over WritableRpc</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2706">MAPREDUCE-2706</a>.
+ Major bug reported by Jeffrey Naisbitt and fixed by Jeffrey Naisbitt (mrv2)<br>
+ <b>MR-279: Submit jobs beyond the max jobs per queue limit no longer gets logged</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2705">MAPREDUCE-2705</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves (tasktracker)<br>
+ <b>tasks localized and launched serially by TaskLauncher - causing other tasks to be delayed</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2702">MAPREDUCE-2702</a>.
+ Blocker sub-task reported by Sharad Agarwal and fixed by Sharad Agarwal (applicationmaster , mrv2)<br>
+ <b>[MR-279] OutputCommitter changes for MR Application Master recovery</b><br>
+ <blockquote>Enhance OutputCommitter and FileOutputCommitter to allow for recover of tasks across job restart.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2701">MAPREDUCE-2701</a>.
+ Major improvement reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>MR-279: app/Job.java needs UGI for the user that launched it</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2697">MAPREDUCE-2697</a>.
+ Major bug reported by Arun C Murthy and fixed by Arun C Murthy (mrv2)<br>
+ <b>Enhance CS to cap concurrently running jobs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2696">MAPREDUCE-2696</a>.
+ Major sub-task reported by Arun C Murthy and fixed by Siddharth Seth (mrv2 , nodemanager)<br>
+ <b>Container logs aren't getting cleaned up when LogAggregation is disabled</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2693">MAPREDUCE-2693</a>.
+ Critical bug reported by Amol Kekre and fixed by Hitesh Shah (mrv2)<br>
+ <b>NPE in AM causes it to lose containers which are never returned back to RM</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2692">MAPREDUCE-2692</a>.
+ Major new feature reported by Amol Kekre and fixed by Sharad Agarwal (mrv2)<br>
+ <b>Ensure AM Restart and Recovery-on-restart is complete</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2691">MAPREDUCE-2691</a>.
+ Major improvement reported by Amol Kekre and fixed by Siddharth Seth (mrv2)<br>
+ <b>Finish up the cleanup of distributed cache file resources and related tests.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2690">MAPREDUCE-2690</a>.
+ Major bug reported by Ramya Sunil and fixed by Eric Payne (mrv2)<br>
+ <b>Construct the web page for default scheduler</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2689">MAPREDUCE-2689</a>.
+ Major bug reported by Ramya Sunil and fixed by (mrv2)<br>
+ <b>InvalidStateTransisiton when AM is not assigned to a job</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2687">MAPREDUCE-2687</a>.
+ Blocker bug reported by Ramya Sunil and fixed by Mahadev konar (mrv2)<br>
+ <b>Non superusers unable to launch apps in both secure and non-secure cluster</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2682">MAPREDUCE-2682</a>.
+ Trivial improvement reported by Arun C Murthy and fixed by Vinod Kumar Vavilapalli <br>
+ <b>Add a -classpath option to bin/mapred</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2680">MAPREDUCE-2680</a>.
+ Minor improvement reported by Arun C Murthy and fixed by Arun C Murthy <br>
+ <b>Enhance job-client cli to show queue information for running jobs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2679">MAPREDUCE-2679</a>.
+ Trivial improvement reported by Arun C Murthy and fixed by Arun C Murthy <br>
+ <b>MR-279: Merge MR-279 related minor patches into trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2678">MAPREDUCE-2678</a>.
+ Major bug reported by Jeffrey Naisbitt and fixed by Jeffrey Naisbitt (capacity-sched)<br>
+ <b>MR-279: minimum-user-limit-percent no longer honored</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2677">MAPREDUCE-2677</a>.
+ Major bug reported by Ramya Sunil and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>MR-279: 404 error while accessing pages from history server</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2676">MAPREDUCE-2676</a>.
+ Major improvement reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>MR-279: JobHistory Job page needs reformatted</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2675">MAPREDUCE-2675</a>.
+ Major improvement reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>MR-279: JobHistory Server main page needs to be reformatted</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2672">MAPREDUCE-2672</a>.
+ Major improvement reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>MR-279: JobHistory Server needs Analysis this job</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2670">MAPREDUCE-2670</a>.
+ Trivial bug reported by Eli Collins and fixed by Eli Collins <br>
+ <b>Fixing spelling mistake in FairSchedulerServlet.java</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2668">MAPREDUCE-2668</a>.
+ Blocker bug reported by Robert Joseph Evans and fixed by Thomas Graves (mrv2)<br>
+ <b>MR-279: APPLICATION_STOP is never sent to AuxServices</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2667">MAPREDUCE-2667</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>MR279: mapred job -kill leaves application in RUNNING state</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2666">MAPREDUCE-2666</a>.
+ Blocker sub-task reported by Robert Joseph Evans and fixed by Jonathan Eagles (mrv2)<br>
+ <b>MR-279: Need to retrieve shuffle port number on ApplicationMaster restart</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2664">MAPREDUCE-2664</a>.
+ Major improvement reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: Implement JobCounters for MRv2 + Fix for Map Data Locality</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2663">MAPREDUCE-2663</a>.
+ Minor bug reported by Ahmed Radwan and fixed by Ahmed Radwan (mrv2)<br>
+ <b>MR-279: Refactoring StateMachineFactory inner classes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2661">MAPREDUCE-2661</a>.
+ Minor bug reported by Ahmed Radwan and fixed by Ahmed Radwan (mrv2)<br>
+ <b>MR-279: Accessing MapTaskImpl from TaskImpl</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2655">MAPREDUCE-2655</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>MR279: Audit logs for YARN </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2649">MAPREDUCE-2649</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves (mrv2)<br>
+ <b>MR279: Fate of finished Applications on RM</b><br>
+ <blockquote>New config added:
+
+ // the maximum number of completed applications the RM keeps <name>yarn.server.resourcemanager.expire.applications.completed.max</name></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2646">MAPREDUCE-2646</a>.
+ Critical bug reported by Sharad Agarwal and fixed by Sharad Agarwal (applicationmaster , mrv2)<br>
+ <b>MR-279: AM with same sized maps and reduces hangs in presence of failing maps</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2644">MAPREDUCE-2644</a>.
+ Major bug reported by Josh Wills and fixed by Josh Wills (mrv2)<br>
+ <b>NodeManager fails to create containers when NM_LOG_DIR is not explicitly set in the Configuration</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2641">MAPREDUCE-2641</a>.
+ Minor sub-task reported by Josh Wills and fixed by Josh Wills (mrv2)<br>
+ <b>Fix the ExponentiallySmoothedTaskRuntimeEstimator and its unit test</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2630">MAPREDUCE-2630</a>.
+ Minor bug reported by Josh Wills and fixed by Josh Wills (mrv2)<br>
+ <b>MR-279: refreshQueues leads to NPEs when used w/FifoScheduler</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2629">MAPREDUCE-2629</a>.
+ Minor improvement reported by Eric Caspole and fixed by Eric Caspole (task)<br>
+ <b>Class loading quirk prevents inner class method compilation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2628">MAPREDUCE-2628</a>.
+ Minor bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>MR-279: Add compiled on date to NM and RM info/about page</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2625">MAPREDUCE-2625</a>.
+ Minor bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>MR-279: Add Node Manager Version to NM info page</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2624">MAPREDUCE-2624</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (contrib/raid)<br>
+ <b>Update RAID for HDFS-2107</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2623">MAPREDUCE-2623</a>.
+ Minor improvement reported by Jim Plush and fixed by Harsh J (test)<br>
+ <b>Update ClusterMapReduceTestCase to use MiniDFSCluster.Builder</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2622">MAPREDUCE-2622</a>.
+ Minor task reported by Harsh J and fixed by Harsh J (test)<br>
+ <b>Remove the last remaining reference to "io.sort.mb"</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2620">MAPREDUCE-2620</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (contrib/raid)<br>
+ <b>Update RAID for HDFS-2087</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2618">MAPREDUCE-2618</a>.
+ Major bug reported by Jeffrey Naisbitt and fixed by Jeffrey Naisbitt (mrv2)<br>
+ <b>MR-279: 0 map, 0 reduce job fails with Null Pointer Exception</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2615">MAPREDUCE-2615</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: KillJob should go through AM whenever possible</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2611">MAPREDUCE-2611</a>.
+ Major improvement reported by Siddharth Seth and fixed by (mrv2)<br>
+ <b>MR 279: Metrics, finishTimes, etc in JobHistory</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2606">MAPREDUCE-2606</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur <br>
+ <b>Remove IsolationRunner</b><br>
+ <blockquote>IsolationRunner is no longer maintained. See MAPREDUCE-2637 for its replacement. </blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2603">MAPREDUCE-2603</a>.
+ Major bug reported by Vinay Kumar Thota and fixed by Vinay Kumar Thota (contrib/gridmix)<br>
+ <b>Gridmix system tests are failing due to high ram emulation enable by default for normal mr jobs in the trace which exceeds the solt capacity.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2602">MAPREDUCE-2602</a>.
+ Major improvement reported by Ahmed Radwan and fixed by Ahmed Radwan <br>
+ <b>Allow setting of end-of-record delimiter for TextInputFormat (for the old API)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2598">MAPREDUCE-2598</a>.
+ Minor bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: miscellaneous UI, NPE fixes for JobHistory, UI</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2596">MAPREDUCE-2596</a>.
+ Major improvement reported by Arun C Murthy and fixed by Amar Kamat (benchmarks , contrib/gridmix)<br>
+ <b>Gridmix should notify job failures</b><br>
+ <blockquote>Gridmix now prints a summary information after every run. It summarizes the runs w.r.t input trace details, input data statistics, cli arguments, data-gen runtime, simulation runtimes etc and also the cluster w.r.t map slots, reduce slots, jobtracker-address, hdfs-address etc.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2595">MAPREDUCE-2595</a>.
+ Minor bug reported by Thomas Graves and fixed by Thomas Graves <br>
+ <b>MR279: update yarn INSTALL doc</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2588">MAPREDUCE-2588</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (contrib/raid)<br>
+ <b>Raid is not compile after DataTransferProtocol refactoring</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2587">MAPREDUCE-2587</a>.
+ Minor bug reported by Thomas Graves and fixed by Thomas Graves <br>
+ <b>MR279: Fix RM version in the cluster->about page </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2582">MAPREDUCE-2582</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: Cleanup JobHistory event generation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2581">MAPREDUCE-2581</a>.
+ Trivial bug reported by Dave Syer and fixed by Tim Sell <br>
+ <b>Spelling errors in log messages (MapTask)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2580">MAPREDUCE-2580</a>.
+ Minor improvement reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: RM UI should redirect finished jobs to History UI</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2576">MAPREDUCE-2576</a>.
+ Trivial bug reported by Sherry Chen and fixed by Tim Sell <br>
+ <b>Typo in comment in SimulatorLaunchTaskAction.java</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2575">MAPREDUCE-2575</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves (test)<br>
+ <b>TestMiniMRDFSCaching fails if test.build.dir is set to something other than build/test</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2573">MAPREDUCE-2573</a>.
+ Major bug reported by Todd Lipcon and fixed by Robert Joseph Evans <br>
+ <b>New findbugs warning after MAPREDUCE-2494</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2569">MAPREDUCE-2569</a>.
+ Minor bug reported by Jonathan Eagles and fixed by Jonathan Eagles (mrv2)<br>
+ <b>MR-279: Restarting resource manager with root capacity not equal to 100 percent should result in error</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2566">MAPREDUCE-2566</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: YarnConfiguration should reloadConfiguration if instantiated with a non YarnConfiguration object</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2563">MAPREDUCE-2563</a>.
+ Major task reported by Vinay Kumar Thota and fixed by Vinay Kumar Thota (contrib/gridmix)<br>
+ <b>Gridmix high ram jobs emulation system tests.</b><br>
+ <blockquote>Adds system tests to test the High-Ram feature in Gridmix.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2559">MAPREDUCE-2559</a>.
+ Major bug reported by Eric Yang and fixed by Eric Yang (build)<br>
+ <b>ant binary fails due to missing c++ lib dir</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2556">MAPREDUCE-2556</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: NodeStatus.getNodeHealthStatus().setBlah broken</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2554">MAPREDUCE-2554</a>.
+ Major task reported by Vinay Kumar Thota and fixed by Vinay Kumar Thota (contrib/gridmix)<br>
+ <b>Gridmix distributed cache emulation system tests.</b><br>
+ <blockquote>Adds distributed cache related system tests to Gridmix.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2552">MAPREDUCE-2552</a>.
+ Minor bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: NPE when requesting attemptids for completed jobs </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2551">MAPREDUCE-2551</a>.
+ Major improvement reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: Implement JobSummaryLog</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2550">MAPREDUCE-2550</a>.
+ Blocker bug reported by Eric Yang and fixed by Eric Yang (build)<br>
+ <b>bin/mapred no longer works from a source checkout</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2544">MAPREDUCE-2544</a>.
+ Major task reported by Vinay Kumar Thota and fixed by Vinay Kumar Thota (contrib/gridmix)<br>
+ <b>Gridmix compression emulation system tests.</b><br>
+ <blockquote>Adds system tests for testing the compression emulation feature of Gridmix.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2543">MAPREDUCE-2543</a>.
+ Major new feature reported by Amar Kamat and fixed by Amar Kamat (contrib/gridmix)<br>
+ <b>[Gridmix] Add support for HighRam jobs</b><br>
+ <blockquote>Adds High-Ram feature emulation in Gridmix.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2541">MAPREDUCE-2541</a>.
+ Critical bug reported by Binglin Chang and fixed by Binglin Chang (tasktracker)<br>
+ <b>Race Condition in IndexCache(readIndexFileToCache,removeMap) causes value of totalMemoryUsed corrupt, which may cause TaskTracker continue throw Exception</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2537">MAPREDUCE-2537</a>.
+ Minor bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)<br>
+ <b>MR-279: The RM writes its log to yarn-mapred-resourcemanager-<RM_Host>.out</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2536">MAPREDUCE-2536</a>.
+ Minor test reported by Daryn Sharp and fixed by Daryn Sharp (test)<br>
+ <b>TestMRCLI broke due to change in usage output</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2534">MAPREDUCE-2534</a>.
+ Major bug reported by Luke Lu and fixed by Luke Lu (mrv2)<br>
+ <b>MR-279: Fix CI breaking hard coded version in jobclient pom</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2533">MAPREDUCE-2533</a>.
+ Major new feature reported by Luke Lu and fixed by Luke Lu (mrv2)<br>
+ <b>MR-279: Metrics for reserved resource in ResourceManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2532">MAPREDUCE-2532</a>.
+ Major new feature reported by Luke Lu and fixed by Luke Lu (mrv2)<br>
+ <b>MR-279: Metrics for NodeManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2531">MAPREDUCE-2531</a>.
+ Blocker bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (client)<br>
+ <b>org.apache.hadoop.mapred.jobcontrol.getAssignedJobID throw class cast exception </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2529">MAPREDUCE-2529</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves (tasktracker)<br>
+ <b>Recognize Jetty bug 1342 and handle it</b><br>
+ <blockquote>Added 2 new config parameters:
+
+mapreduce.reduce.shuffle.catch.exception.stack.regex
+mapreduce.reduce.shuffle.catch.exception.message.regex</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2527">MAPREDUCE-2527</a>.
+ Major new feature reported by Luke Lu and fixed by Luke Lu (mrv2)<br>
+ <b>MR-279: Metrics for MRAppMaster</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2522">MAPREDUCE-2522</a>.
+ Major sub-task reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: Security for JobHistory service</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2521">MAPREDUCE-2521</a>.
+ Major new feature reported by Eric Yang and fixed by Eric Yang (build)<br>
+ <b>Mapreduce RPM integration project</b><br>
+ <blockquote>Created rpm and debian packages for MapReduce. </blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2518">MAPREDUCE-2518</a>.
+ Major bug reported by Wei Yongjun and fixed by Wei Yongjun (distcp)<br>
+ <b>missing t flag in distcp help message '-p[rbugp]'</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2517">MAPREDUCE-2517</a>.
+ Major task reported by Vinay Kumar Thota and fixed by Vinay Kumar Thota (contrib/gridmix)<br>
+ <b>Porting Gridmix v3 system tests into trunk branch.</b><br>
+ <blockquote>Adds system tests to Gridmix. These system tests cover various features like job types (load and sleep), user resolvers (round-robin, submitter-user, echo) and submission modes (stress, replay and serial).</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2514">MAPREDUCE-2514</a>.
+ Trivial bug reported by Jonathan Eagles and fixed by Jonathan Eagles (tasktracker)<br>
+ <b>ReinitTrackerAction class name misspelled RenitTrackerAction in task tracker log</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2509">MAPREDUCE-2509</a>.
+ Major bug reported by Luke Lu and fixed by Luke Lu (mrv2)<br>
+ <b>MR-279: Fix NPE in UI for pending attempts</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2504">MAPREDUCE-2504</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: race in JobHistoryEventHandler stop </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2501">MAPREDUCE-2501</a>.
+ Major improvement reported by Luke Lu and fixed by Luke Lu (mrv2)<br>
+ <b>MR-279: Attach sources in builds</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2500">MAPREDUCE-2500</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: PB factories are not thread safe</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2497">MAPREDUCE-2497</a>.
+ Trivial bug reported by Robert Henry and fixed by Eli Collins <br>
+ <b>missing spaces in error messages</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2495">MAPREDUCE-2495</a>.
+ Minor improvement reported by Robert Joseph Evans and fixed by Robert Joseph Evans (distributed-cache)<br>
+ <b>The distributed cache cleanup thread has no monitoring to check to see if it has died for some reason</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2494">MAPREDUCE-2494</a>.
+ Major improvement reported by Robert Joseph Evans and fixed by Robert Joseph Evans (distributed-cache)<br>
+ <b>Make the distributed cache delete entires using LRU priority</b><br>
+ <blockquote>Added config option mapreduce.tasktracker.cache.local.keep.pct to the TaskTracker. It is the target percentage of the local distributed cache that should be kept in between garbage collection runs. In practice it will delete unused distributed cache entries in LRU order until the size of the cache is less than mapreduce.tasktracker.cache.local.keep.pct of the maximum cache size. This is a floating point value between 0.0 and 1.0. The default is 0.95.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2492">MAPREDUCE-2492</a>.
+ Major improvement reported by Amar Kamat and fixed by Amar Kamat (task)<br>
+ <b>[MAPREDUCE] The new MapReduce API should make available task's progress to the task</b><br>
+ <blockquote>Map and Reduce task can access the attempt's overall progress via TaskAttemptContext.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2490">MAPREDUCE-2490</a>.
+ Trivial improvement reported by Jonathan Eagles and fixed by Jonathan Eagles (jobtracker)<br>
+ <b>Log blacklist debug count</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2489">MAPREDUCE-2489</a>.
+ Major bug reported by Jeffrey Naisbitt and fixed by Jeffrey Naisbitt (jobtracker)<br>
+ <b>Jobsplits with random hostnames can make the queue unusable</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2483">MAPREDUCE-2483</a>.
+ Major bug reported by Eric Yang and fixed by Eric Yang (build)<br>
+ <b>Clean up duplication of dependent jar files</b><br>
+ <blockquote>Removed duplicated hadoop-common library dependencies.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2480">MAPREDUCE-2480</a>.
+ Major bug reported by Luke Lu and fixed by Luke Lu (mrv2)<br>
+ <b>MR-279: mr app should not depend on hard-coded version of shuffle</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2478">MAPREDUCE-2478</a>.
+ Major improvement reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: Improve history server</b><br>
+ <blockquote>Looks great. I just committed this. Thanks Siddharth!</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2475">MAPREDUCE-2475</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas (test)<br>
+ <b>Disable IPV6 for junit tests</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2474">MAPREDUCE-2474</a>.
+ Minor improvement reported by Harsh J and fixed by Harsh J (documentation)<br>
+ <b>Add docs to the new API Partitioner on how to access Job Configuration data</b><br>
+ <blockquote>Improve the Partitioner interface's docs to help fetch Job Configuration objects.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2473">MAPREDUCE-2473</a>.
+ Major new feature reported by Aaron T. Myers and fixed by Aaron T. Myers (jobtracker)<br>
+ <b>MR portion of HADOOP-7214 - Hadoop /usr/bin/groups equivalent</b><br>
+ <blockquote>Introduces a new command, "mapred groups", which displays what groups are associated with a user as seen by the JobTracker.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2470">MAPREDUCE-2470</a>.
+ Major bug reported by Aaron Baff and fixed by Robert Joseph Evans (client)<br>
+ <b>Receiving NPE occasionally on RunningJob.getCounters() call</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2469">MAPREDUCE-2469</a>.
+ Major improvement reported by Amar Kamat and fixed by Amar Kamat (task)<br>
+ <b>Task counters should also report the total heap usage of the task</b><br>
+ <blockquote>Task attempt's total heap usage gets recorded and published via counters as COMMITTED_HEAP_BYTES.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2467">MAPREDUCE-2467</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas (contrib/raid)<br>
+ <b>HDFS-1052 changes break the raid contrib module in MapReduce</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2466">MAPREDUCE-2466</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>TestFileInputFormat.testLocality failing after federation merge</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2463">MAPREDUCE-2463</a>.
+ Major bug reported by Devaraj K and fixed by Devaraj K (jobtracker)<br>
+ <b>Job History files are not moving to done folder when job history location is hdfs location</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2462">MAPREDUCE-2462</a>.
+ Minor improvement reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR 279: Write job conf along with JobHistory, other minor improvements</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2460">MAPREDUCE-2460</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>TestFairSchedulerSystem failing on Hudson</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2459">MAPREDUCE-2459</a>.
+ Major improvement reported by Mac Yang and fixed by Mac Yang (harchive)<br>
+ <b>Cache HAR filesystem metadata</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2458">MAPREDUCE-2458</a>.
+ Major bug reported by Luke Lu and fixed by Luke Lu (mrv2)<br>
+ <b>MR-279: Rename sanitized pom.xml in build directory to work around IDE bug</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2456">MAPREDUCE-2456</a>.
+ Trivial improvement reported by Jeffrey Naisbitt and fixed by Jeffrey Naisbitt (jobtracker)<br>
+ <b>Show the reducer taskid and map/reduce tasktrackers for "Failed fetch notification #_ for task attempt..." log messages</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2455">MAPREDUCE-2455</a>.
+ Major sub-task reported by Tom White and fixed by Tom White (build , client)<br>
+ <b>Remove deprecated JobTracker.State in favour of JobTrackerStatus</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2451">MAPREDUCE-2451</a>.
+ Trivial bug reported by Thomas Graves and fixed by Thomas Graves (jobtracker)<br>
+ <b>Log the reason string of healthcheck script</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2449">MAPREDUCE-2449</a>.
+ Minor improvement reported by Jeff Zemerick and fixed by Jeff Zemerick (contrib/eclipse-plugin)<br>
+ <b>Allow for command line arguments when performing "Run on Hadoop" action.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2440">MAPREDUCE-2440</a>.
+ Major bug reported by Luke Lu and fixed by Luke Lu (mrv2)<br>
+ <b>MR-279: Name clashes in TypeConverter</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2439">MAPREDUCE-2439</a>.
+ Major bug reported by Mahadev konar and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR-279: Fix YarnRemoteException to give more details.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2438">MAPREDUCE-2438</a>.
+ Major new feature reported by Mahadev konar and fixed by Krishna Ramachandran (mrv2)<br>
+ <b>MR-279: WebApp for Job History</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2434">MAPREDUCE-2434</a>.
+ Major new feature reported by Luke Lu and fixed by Luke Lu (mrv2)<br>
+ <b>MR-279: ResourceManager metrics</b><br>
+ <blockquote>I just committed this. Thanks Luke!</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2433">MAPREDUCE-2433</a>.
+ Blocker bug reported by Luke Lu and fixed by Mahadev konar (mrv2)<br>
+ <b>MR-279: YARNApplicationConstants hard code app master jar version</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2432">MAPREDUCE-2432</a>.
+ Major improvement reported by Luke Lu and fixed by Luke Lu (mrv2)<br>
+ <b>MR-279: Install sanitized poms for downstream sanity</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2430">MAPREDUCE-2430</a>.
+ Major task reported by Nigel Daley and fixed by Nigel Daley <br>
+ <b>Remove mrunit contrib</b><br>
+ <blockquote>MRUnit is now available as a separate Apache project.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2429">MAPREDUCE-2429</a>.
+ Major bug reported by Arun C Murthy and fixed by Siddharth Seth (tasktracker)<br>
+ <b>Check jvmid during task status report</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2428">MAPREDUCE-2428</a>.
+ Blocker bug reported by Tom White and fixed by Tom White <br>
+ <b>start-mapred.sh script fails if HADOOP_HOME is not set</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2426">MAPREDUCE-2426</a>.
+ Trivial test reported by Todd Lipcon and fixed by Todd Lipcon (contrib/fair-share)<br>
+ <b>Make TestFairSchedulerSystem fail with more verbose output</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2424">MAPREDUCE-2424</a>.
+ Major improvement reported by Greg Roelofs and fixed by Greg Roelofs (mrv2)<br>
+ <b>MR-279: counters/UI/etc. for uber-AppMaster (in-cluster LocalJobRunner for MRv2)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2422">MAPREDUCE-2422</a>.
+ Major sub-task reported by Tom White and fixed by Tom White (client)<br>
+ <b>Removed unused internal methods from DistributedCache</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2417">MAPREDUCE-2417</a>.
+ Major bug reported by Ravi Gummadi and fixed by Ravi Gummadi (contrib/gridmix)<br>
+ <b>In Gridmix, in RoundRobinUserResolver mode, the testing/proxy users are not associated with unique users in a trace</b><br>
+ <blockquote>Fixes Gridmix in RoundRobinUserResolver mode to map testing/proxy users to unique users in a trace.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2416">MAPREDUCE-2416</a>.
+ Major bug reported by Ravi Gummadi and fixed by Ravi Gummadi (contrib/gridmix)<br>
+ <b>In Gridmix, in RoundRobinUserResolver, the list of groups for a user obtained from users-list-file is incorrect</b><br>
+ <blockquote>Removes the restriction of specifying group names in users-list file for Gridmix in RoundRobinUserResolver mode.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2414">MAPREDUCE-2414</a>.
+ Major improvement reported by Arun C Murthy and fixed by Siddharth Seth (mrv2)<br>
+ <b>MR-279: Use generic interfaces for protocols</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2409">MAPREDUCE-2409</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth (distributed-cache)<br>
+ <b>Distributed Cache does not differentiate between file /archive for files with the same path</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2408">MAPREDUCE-2408</a>.
+ Major new feature reported by Ravi Gummadi and fixed by Amar Kamat (contrib/gridmix)<br>
+ <b>Make Gridmix emulate usage of data compression</b><br>
+ <blockquote>Emulates the MapReduce compression feature in Gridmix. By default, compression emulation is turned on. Compression emulation can be disabled by setting 'gridmix.compression-emulation.enable' to 'false'. Use 'gridmix.compression-emulation.map-input.decompression-ratio', 'gridmix.compression-emulation.map-output.compression-ratio' and 'gridmix.compression-emulation.reduce-output.compression-ratio' to configure the compression ratios at map input, map output and reduce output side respectively. Currently, compression ratios in the range [0.07, 0.68] are supported. Gridmix auto detects whether map-input, map output and reduce output should emulate compression based on original job's compression related configuration parameters.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2405">MAPREDUCE-2405</a>.
+ Major improvement reported by Mahadev konar and fixed by Greg Roelofs (mrv2)<br>
+ <b>MR-279: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2)</b><br>
+ <blockquote>An efficient implementation of small jobs by running all tasks in the MR ApplicationMaster JVM, there-by affecting lower latency.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2403">MAPREDUCE-2403</a>.
+ Major improvement reported by Mahadev konar and fixed by Krishna Ramachandran (mrv2)<br>
+ <b>MR-279: Improve job history event handling in AM to log to HDFS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2399">MAPREDUCE-2399</a>.
+ Major improvement reported by Arun C Murthy and fixed by Luke Lu <br>
+ <b>The embedded web framework for MAPREDUCE-279</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2395">MAPREDUCE-2395</a>.
+ Critical bug reported by Todd Lipcon and fixed by Ramkumar Vadali (contrib/raid)<br>
+ <b>TestBlockFixer timing out on trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2381">MAPREDUCE-2381</a>.
+ Major improvement reported by Philip Zeyliger and fixed by Philip Zeyliger <br>
+ <b>JobTracker instrumentation not consistent about error handling</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2379">MAPREDUCE-2379</a>.
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon (distributed-cache , documentation)<br>
+ <b>Distributed cache sizing configurations are missing from mapred-default.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2367">MAPREDUCE-2367</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Allow using a file to exclude certain tests from build</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2365">MAPREDUCE-2365</a>.
+ Major bug reported by Owen O'Malley and fixed by Siddharth Seth <br>
+ <b>Add counters for FileInputFormat (BYTES_READ) and FileOutputFormat (BYTES_WRITTEN)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2351">MAPREDUCE-2351</a>.
+ Major improvement reported by Tom White and fixed by Tom White <br>
+ <b>mapred.job.tracker.history.completed.location should support an arbitrary filesystem URI</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2331">MAPREDUCE-2331</a>.
+ Major test reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Add coverage of task graph servlet to fair scheduler system test</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2326">MAPREDUCE-2326</a>.
+ Major improvement reported by Arun C Murthy and fixed by <br>
+ <b>Port gridmix changes from hadoop-0.20.100 to trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2323">MAPREDUCE-2323</a>.
+ Major new feature reported by Todd Lipcon and fixed by Todd Lipcon (contrib/fair-share)<br>
+ <b>Add metrics to the fair scheduler</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2317">MAPREDUCE-2317</a>.
+ Minor bug reported by Devaraj K and fixed by Devaraj K (harchive)<br>
+ <b>HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2311">MAPREDUCE-2311</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Scott Chen (contrib/fair-share)<br>
+ <b>TestFairScheduler failing on trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2307">MAPREDUCE-2307</a>.
+ Minor bug reported by Devaraj K and fixed by Devaraj K (contrib/fair-share)<br>
+ <b>Exception thrown in Jobtracker logs, when the Scheduler configured is FairScheduler.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2302">MAPREDUCE-2302</a>.
+ Major improvement reported by Scott Chen and fixed by Scott Chen (contrib/raid)<br>
+ <b>Add static factory methods in GaloisField</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2290">MAPREDUCE-2290</a>.
+ Major bug reported by Eli Collins and fixed by Eli Collins (test)<br>
+ <b>TestTaskCommit missing getProtocolSignature override</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2271">MAPREDUCE-2271</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Liyin Liang (jobtracker)<br>
+ <b>TestSetupTaskScheduling failing in trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2263">MAPREDUCE-2263</a>.
+ Major improvement reported by Hairong Kuang and fixed by Hairong Kuang <br>
+ <b>MapReduce side of HADOOP-6904</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2260">MAPREDUCE-2260</a>.
+ Major improvement reported by Roman Shaposhnik and fixed by Roman Shaposhnik (build)<br>
+ <b>Remove auto-generated native build files</b><br>
+ <blockquote>The native build run when from trunk now requires autotools, libtool and openssl dev libraries.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2258">MAPREDUCE-2258</a>.
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon (task)<br>
+ <b>IFile reader closes stream and compressor in wrong order</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2254">MAPREDUCE-2254</a>.
+ Major improvement reported by Ahmed Radwan and fixed by Ahmed Radwan <br>
+ <b>Allow setting of end-of-record delimiter for TextInputFormat</b><br>
+ <blockquote>TextInputFormat may now split lines with delimiters other than newline, by specifying a configuration parameter "textinputformat.record.delimiter"</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2250">MAPREDUCE-2250</a>.
+ Trivial improvement reported by Ramkumar Vadali and fixed by Ramkumar Vadali (contrib/raid)<br>
+ <b>Fix logging in raid code.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2249">MAPREDUCE-2249</a>.
+ Major improvement reported by Bhallamudi Venkata Siva Kamesh and fixed by Devaraj K <br>
+ <b>Better to check the reflexive property of the object while overriding equals method of it</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2248">MAPREDUCE-2248</a>.
+ Major improvement reported by Ramkumar Vadali and fixed by Ramkumar Vadali <br>
+ <b>DistributedRaidFileSystem should unraid only the corrupt block</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2243">MAPREDUCE-2243</a>.
+ Minor improvement reported by Bhallamudi Venkata Siva Kamesh and fixed by Devaraj K (jobtracker , tasktracker)<br>
+ <b>Close all the file streams propely in a finally block to avoid their leakage.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2239">MAPREDUCE-2239</a>.
+ Major improvement reported by Scott Chen and fixed by Scott Chen (contrib/raid)<br>
+ <b>BlockPlacementPolicyRaid should call getBlockLocations only when necessary</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2225">MAPREDUCE-2225</a>.
+ Blocker improvement reported by Harsh J and fixed by Harsh J (job submission)<br>
+ <b>MultipleOutputs should not require the use of 'Writable'</b><br>
+ <blockquote>MultipleOutputs should not require the use/check of 'Writable' interfaces in key and value classes.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2215">MAPREDUCE-2215</a>.
+ Major bug reported by Patrick Kling and fixed by Patrick Kling (contrib/raid)<br>
+ <b>A more elegant FileSystem#listCorruptFileBlocks API (RAID changes)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2207">MAPREDUCE-2207</a>.
+ Major improvement reported by Scott Chen and fixed by Liyin Liang (jobtracker)<br>
+ <b>Task-cleanup task should not be scheduled on the node that the task just failed</b><br>
+ <blockquote>Task-cleanup task should not be scheduled on the node that the task just failed</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2206">MAPREDUCE-2206</a>.
+ Major improvement reported by Scott Chen and fixed by Scott Chen (jobtracker)<br>
+ <b>The task-cleanup tasks should be optional</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2203">MAPREDUCE-2203</a>.
+ Trivial improvement reported by Jingguo Yao and fixed by Jingguo Yao <br>
+ <b>Wong javadoc for TaskRunner's appendJobJarClasspaths method</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2202">MAPREDUCE-2202</a>.
+ Major improvement reported by Konstantin Boudnik and fixed by Konstantin Boudnik <br>
+ <b>Generalize CLITest structure and interfaces to facilitate upstream adoption (e.g. for web or system testing)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2199">MAPREDUCE-2199</a>.
+ Major bug reported by Konstantin Boudnik and fixed by Konstantin Boudnik (build)<br>
+ <b>build is broken 0.22 branch creation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2185">MAPREDUCE-2185</a>.
+ Major bug reported by Hairong Kuang and fixed by Ramkumar Vadali (job submission)<br>
+ <b>Infinite loop at creating splits using CombineFileInputFormat</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2172">MAPREDUCE-2172</a>.
+ Major bug reported by Patrick Kling and fixed by Nigel Daley <br>
+ <b>test-patch.properties contains incorrect/version-dependent values of OK_FINDBUGS_WARNINGS and OK_RELEASEAUDIT_WARNINGS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2156">MAPREDUCE-2156</a>.
+ Major improvement reported by Patrick Kling and fixed by Patrick Kling (contrib/raid)<br>
+ <b>Raid-aware FSCK</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2155">MAPREDUCE-2155</a>.
+ Major improvement reported by Patrick Kling and fixed by Patrick Kling (contrib/raid)<br>
+ <b>RaidNode should optionally dispatch map reduce jobs to fix corrupt blocks (instead of fixing locally)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2153">MAPREDUCE-2153</a>.
+ Major improvement reported by Ravi Gummadi and fixed by Rajesh Balamohan (tools/rumen)<br>
+ <b>Bring in more job configuration properties in to the trace file</b><br>
+ <blockquote>Adds job configuration parameters to the job trace. The configuration parameters are stored under the 'jobProperties' field as key-value pairs.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2137">MAPREDUCE-2137</a>.
+ Major bug reported by Ravi Gummadi and fixed by Ravi Gummadi (contrib/gridmix)<br>
+ <b>Mapping between Gridmix jobs and the corresponding original MR jobs is needed</b><br>
+ <blockquote>New configuration properties gridmix.job.original-job-id and gridmix.job.original-job-name in the configuration of simulated job are exposed/documented to gridmix user for mapping between original cluster's jobs and simulated jobs.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2127">MAPREDUCE-2127</a>.
+ Major bug reported by Giridharan Kesavan and fixed by Bruno Mahé (build , pipes)<br>
+ <b>mapreduce trunk builds are failing on hudson</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2107">MAPREDUCE-2107</a>.
+ Major improvement reported by Ranjit Mathew and fixed by Amar Kamat (contrib/gridmix)<br>
+ <b>Emulate Memory Usage of Tasks in GridMix3</b><br>
+ <blockquote>Adds total heap usage emulation to Gridmix. Also, Gridmix can configure the simulated task's JVM heap options with max heap options obtained from the original task (via Rumen). Use 'gridmix.task.jvm-options.enable' to disable the task max heap options configuration. </blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2106">MAPREDUCE-2106</a>.
+ Major improvement reported by Ranjit Mathew and fixed by Amar Kamat (contrib/gridmix)<br>
+ <b>Emulate CPU Usage of Tasks in GridMix3</b><br>
+ <blockquote>Adds cumulative cpu usage emulation to Gridmix</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2105">MAPREDUCE-2105</a>.
+ Major improvement reported by Ranjit Mathew and fixed by Amar Kamat (contrib/gridmix)<br>
+ <b>Simulate Load Incrementally and Adaptively in GridMix3</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2104">MAPREDUCE-2104</a>.
+ Major bug reported by Ranjit Mathew and fixed by Amar Kamat (tools/rumen)<br>
+ <b>Rumen TraceBuilder Does Not Emit CPU/Memory Usage Details in Traces</b><br>
+ <blockquote>Adds cpu, physical memory, virtual memory and heap usages to TraceBuilder's output.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2081">MAPREDUCE-2081</a>.
+ Major test reported by Vinay Kumar Thota and fixed by Vinay Kumar Thota (contrib/gridmix)<br>
+ <b>[GridMix3] Implement functionality for get the list of job traces which has different intervals.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2074">MAPREDUCE-2074</a>.
+ Minor bug reported by Koji Noguchi and fixed by Priyo Mustafi (distributed-cache)<br>
+ <b>Task should fail when symlink creation fail</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2053">MAPREDUCE-2053</a>.
+ Major task reported by Vinay Kumar Thota and fixed by Vinay Kumar Thota (contrib/gridmix)<br>
+ <b>[Herriot] Test Gridmix file pool for different input file sizes based on pool minimum size.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2037">MAPREDUCE-2037</a>.
+ Major new feature reported by Dick King and fixed by Dick King <br>
+ <b>Capturing interim progress times, CPU usage, and memory usage, when tasks reach certain progress thresholds</b><br>
+ <blockquote>Capture intermediate task resource consumption information:
+* Time taken so far
+* CPU load [either at the time the data are taken, or exponentially smoothed]
+* Memory load [also either at the time the data are taken, or exponentially smoothed]
+
+This would be taken at intervals that depend on the task progress plateaus. For example, reducers have three progress ranges - [0-1/3], (1/3-2/3], and (2/3-3/3] - where fundamentally different activities happen. Mappers have different boundaries that are not symmetrically placed [0-9/10], (9/10-1]. Data capture boundaries should coincide with activity boundaries. For the state information capture [CPU and memory] we should average over the covered interval.
+</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2033">MAPREDUCE-2033</a>.
+ Major task reported by Vinay Kumar Thota and fixed by Vinay Kumar Thota (contrib/gridmix)<br>
+ <b>[Herriot] Gridmix generate data tests with various submission policies and different user resolvers.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-2026">MAPREDUCE-2026</a>.
+ Major improvement reported by Scott Chen and fixed by Joydeep Sen Sarma <br>
+ <b>JobTracker.getJobCounters() should not hold JobTracker lock while calling JobInProgress.getCounters()</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1996">MAPREDUCE-1996</a>.
+ Trivial bug reported by Glynn Durham and fixed by Harsh J (documentation)<br>
+ <b>API: Reducer.reduce() method detail misstatement</b><br>
+ <blockquote>Fix a misleading documentation note about the usage of Reporter objects in Reducers.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1978">MAPREDUCE-1978</a>.
+ Major improvement reported by Amar Kamat and fixed by Ravi Gummadi (tools/rumen)<br>
+ <b>[Rumen] TraceBuilder should provide recursive input folder scanning</b><br>
+ <blockquote>Adds -recursive option to TraceBuilder for scanning the input directories recursively.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1938">MAPREDUCE-1938</a>.
+ Blocker new feature reported by Devaraj Das and fixed by Krishna Ramachandran (job submission , task , tasktracker)<br>
+ <b>Ability for having user's classes take precedence over the system classes for tasks' classpath</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1927">MAPREDUCE-1927</a>.
+ Minor test reported by Greg Roelofs and fixed by Greg Roelofs (test)<br>
+ <b>unit test for HADOOP-6835 (concatenated gzip support)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1906">MAPREDUCE-1906</a>.
+ Major improvement reported by Scott Carey and fixed by Todd Lipcon (jobtracker , performance , tasktracker)<br>
+ <b>Lower default minimum heartbeat interval for tasktracker > Jobtracker</b><br>
+ <blockquote>The default minimum heartbeat interval has been dropped from 3 seconds to 300ms to increase scheduling throughput on small clusters. Users may tune mapreduce.jobtracker.heartbeats.in.second to adjust this value.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1831">MAPREDUCE-1831</a>.
+ Major improvement reported by Scott Chen and fixed by Scott Chen (contrib/raid)<br>
+ <b>BlockPlacement policy for RAID</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1811">MAPREDUCE-1811</a>.
+ Minor bug reported by Amareshwari Sriramadasu and fixed by Harsh J (client)<br>
+ <b>Job.monitorAndPrintJob() should print status of the job at completion</b><br>
+ <blockquote>Print the resultant status of a Job on completion instead of simply saying 'Complete'.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1788">MAPREDUCE-1788</a>.
+ Major bug reported by Arun C Murthy and fixed by Arun C Murthy (client)<br>
+ <b>o.a.h.mapreduce.Job shouldn't make a copy of the JobConf</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1783">MAPREDUCE-1783</a>.
+ Major improvement reported by Ramkumar Vadali and fixed by Ramkumar Vadali (contrib/fair-share)<br>
+ <b>Task Initialization should be delayed till when a job can be run</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1752">MAPREDUCE-1752</a>.
+ Major improvement reported by Dmytro Molkov and fixed by Dmytro Molkov (harchive)<br>
+ <b>Implement getFileBlockLocations in HarFilesystem</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1738">MAPREDUCE-1738</a>.
+ Major improvement reported by Luke Lu and fixed by Luke Lu <br>
+ <b>MapReduce portion of HADOOP-6728 (ovehaul metrics framework)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1706">MAPREDUCE-1706</a>.
+ Major improvement reported by Rodrigo Schmidt and fixed by Scott Chen (contrib/raid)<br>
+ <b>Log RAID recoveries on HDFS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1702">MAPREDUCE-1702</a>.
+ Minor improvement reported by Jaideep and fixed by (contrib/gridmix)<br>
+ <b>CPU/Memory emulation for GridMix3</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1624">MAPREDUCE-1624</a>.
+ Major improvement reported by Devaraj Das and fixed by Devaraj Das (documentation)<br>
+ <b>Document the job credentials and associated details to do with delegation tokens (on the client side)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1461">MAPREDUCE-1461</a>.
+ Major improvement reported by Rajesh Balamohan and fixed by Rajesh Balamohan (tools/rumen)<br>
+ <b>Feature to instruct rumen-folder utility to skip jobs worth of specific duration</b><br>
+ <blockquote>Added a ''-starts-after' option to Rumen's Folder utility. The time duration specified after the '-starts-after' option is an offset with respect to the submit time of the first job in the input trace. Jobs in the input trace having a submit time (relative to the first job's submit time) lesser than the specified offset will be ignored.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1334">MAPREDUCE-1334</a>.
+ Major bug reported by Karthik K and fixed by Karthik K (contrib/index)<br>
+ <b>contrib/index - test - TestIndexUpdater fails due to an additional presence of file _SUCCESS in hdfs </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1242">MAPREDUCE-1242</a>.
+ Trivial bug reported by Amogh Vasekar and fixed by Harsh J <br>
+ <b>Chain APIs error misleading</b><br>
+ <blockquote>Fix a misleading exception message in case the Chained Mappers have mismatch in input/output Key/Value pairs between them.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1207">MAPREDUCE-1207</a>.
+ Blocker improvement reported by Arun C Murthy and fixed by Arun C Murthy (client , mrv2)<br>
+ <b>Allow admins to set java options for map/reduce tasks</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-1159">MAPREDUCE-1159</a>.
+ Trivial improvement reported by Zheng Shao and fixed by Harsh J <br>
+ <b>Limit Job name on jobtracker.jsp to be 80 char long</b><br>
+ <blockquote>Job names on jobtracker.jsp should be 80 characters long at most.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-993">MAPREDUCE-993</a>.
+ Minor bug reported by Iyappan Srinivasan and fixed by Harsh J (jobtracker)<br>
+ <b>bin/hadoop job -events <jobid> <from-event-#> <#-of-events> help message is confusing</b><br>
+ <blockquote>Added a helpful description message to the `mapred job -events` command.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-901">MAPREDUCE-901</a>.
+ Major improvement reported by Owen O'Malley and fixed by Luke Lu (task)<br>
+ <b>Move Framework Counters into a TaskMetric structure</b><br>
+ <blockquote>Efficient implementation of MapReduce framework counters.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-587">MAPREDUCE-587</a>.
+ Minor bug reported by Steve Loughran and fixed by Amar Kamat (contrib/streaming)<br>
+ <b>Stream test TestStreamingExitStatus fails with Out of Memory</b><br>
+ <blockquote>Fixed the streaming test TestStreamingExitStatus's failure due to an OutOfMemory error by reducing the testcase's io.sort.mb.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-517">MAPREDUCE-517</a>.
+ Critical bug reported by Arun C Murthy and fixed by Arun C Murthy <br>
+ <b>The capacity-scheduler should assign multiple tasks per heartbeat</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-461">MAPREDUCE-461</a>.
+ Minor new feature reported by Fredrik Hedberg and fixed by Fredrik Hedberg <br>
+ <b>Enable ServicePlugins for the JobTracker</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/MAPREDUCE-279">MAPREDUCE-279</a>.
+ Major improvement reported by Arun C Murthy and fixed by (mrv2)<br>
+ <b>Map-Reduce 2.0</b><br>
+ <blockquote>MapReduce has undergone a complete re-haul in hadoop-0.23 and we now have, what we call, MapReduce 2.0 (MRv2).
+
+The fundamental idea of MRv2 is to split up the two major functionalities of the JobTracker, resource management and job scheduling/monitoring, into separate daemons. The idea is to have a global ResourceManager (RM) and per-application ApplicationMaster (AM). An application is either a single job in the classical sense of Map-Reduce jobs or a DAG of jobs. The ResourceManager and per-node slave, the NodeManager (NM), form the data-computation framework. The ResourceManager is the ultimate authority that arbitrates resources among all the applications in the system. The per-application ApplicationMaster is, in effect, a framework specific library and is tasked with negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the tasks.
+
+The ResourceManager has two main components:
+* Scheduler (S)
+* ApplicationsManager (ASM)
+
+The Scheduler is responsible for allocating resources to the various running applications subject to familiar constraints of capacities, queues etc. The Scheduler is pure scheduler in the sense that it performs no monitoring or tracking of status for the application. Also, it offers no guarantees on restarting failed tasks either due to application failure or hardware failures. The Scheduler performs its scheduling function based the resource requirements of the applications; it does so based on the abstract notion of a Resource Container which incorporates elements such as memory, cpu, disk, network etc.
+
+The Scheduler has a pluggable policy plug-in, which is responsible for partitioning the cluster resources among the various queues, applications etc. The current Map-Reduce schedulers such as the CapacityScheduler and the FairScheduler would be some examples of the plug-in.
+
+The CapacityScheduler supports hierarchical queues to allow for more predictable sharing of cluster resources.
+The ApplicationsManager is responsible for accepting job-submissions, negotiating the first container for executing the application specific ApplicationMaster and provides the service for restarting the ApplicationMaster container on failure.
+
+The NodeManager is the per-machine framework agent who is responsible for launching the applications' containers, monitoring their resource usage (cpu, memory, disk, network) and reporting the same to the Scheduler.
+
+The per-application ApplicationMaster has the responsibility of negotiating appropriate resource containers from the Scheduler, tracking their status and monitoring for progress.
+</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2540">HDFS-2540</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE <br>
+ <b>Change WebHdfsFileSystem to two-step create/append</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2539">HDFS-2539</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>Support doAs and GETHOMEDIRECTORY in webhdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2528">HDFS-2528</a>.
+ Major sub-task reported by Arpit Gupta and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>webhdfs rest call to a secure dn fails when a token is sent</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2527">HDFS-2527</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>Remove the use of Range header from webhdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2522">HDFS-2522</a>.
+ Minor test reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>Disable TestDfsOverAvroRpc in 0.23</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2521">HDFS-2521</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node , hdfs client)<br>
+ <b>Remove custom checksum headers from data transfer protocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2512">HDFS-2512</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node , hdfs client)<br>
+ <b>Add textual error message to data transfer protocol responses</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2501">HDFS-2501</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>add version prefix and root methods to webhdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2500">HDFS-2500</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>Avoid file system operations in BPOfferService thread while processing deletes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2494">HDFS-2494</a>.
+ Major sub-task reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (webhdfs)<br>
+ <b>[webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2493">HDFS-2493</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Remove reference to FSNamesystem in blockmanagement classes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2485">HDFS-2485</a>.
+ Trivial improvement reported by Steve Loughran and fixed by Steve Loughran (data-node)<br>
+ <b>Improve code layout and constants in UnderReplicatedBlocks</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2471">HDFS-2471</a>.
+ Major new feature reported by Suresh Srinivas and fixed by Suresh Srinivas (documentation)<br>
+ <b>Add Federation feature, configuration and tools documentation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2467">HDFS-2467</a>.
+ Major bug reported by Owen O'Malley and fixed by Owen O'Malley <br>
+ <b>HftpFileSystem uses incorrect compare for finding delegation tokens</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2465">HDFS-2465</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node , performance)<br>
+ <b>Add HDFS support for fadvise readahead and drop-behind</b><br>
+ <blockquote>HDFS now has the ability to use posix_fadvise and sync_data_range syscalls to manage the OS buffer cache. This support is currently considered experimental, and may be enabled by configuring the following keys:
+dfs.datanode.drop.cache.behind.writes - set to true to drop data out of the buffer cache after writing
+dfs.datanode.drop.cache.behind.reads - set to true to drop data out of the buffer cache when performing sequential reads
+dfs.datanode.sync.behind.writes - set to true to trigger dirty page writeback immediately after writing data
+dfs.datanode.readahead.bytes - set to a non-zero value to trigger readahead for sequential reads</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2453">HDFS-2453</a>.
+ Major sub-task reported by Arpit Gupta and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>tail using a webhdfs uri throws an error</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2452">HDFS-2452</a>.
+ Major bug reported by Konstantin Shvachko and fixed by Uma Maheswara Rao G (data-node)<br>
+ <b>OutOfMemoryError in DataXceiverServer takes down the DataNode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2445">HDFS-2445</a>.
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles (test)<br>
+ <b>Incorrect exit code for hadoop-hdfs-test tests when exception thrown</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2441">HDFS-2441</a>.
+ Major sub-task reported by Arpit Gupta and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>webhdfs returns two content-type headers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2439">HDFS-2439</a>.
+ Major sub-task reported by Arpit Gupta and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>webhdfs open an invalid path leads to a 500 which states a npe, we should return a 404 with appropriate error message</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2436">HDFS-2436</a>.
+ Major bug reported by Arpit Gupta and fixed by Uma Maheswara Rao G <br>
+ <b>FSNamesystem.setTimes(..) expects the path is a file.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2432">HDFS-2432</a>.
+ Major sub-task reported by Arpit Gupta and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>webhdfs setreplication api should return a 403 when called on a directory</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2428">HDFS-2428</a>.
+ Major sub-task reported by Arpit Gupta and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>webhdfs api parameter validation should be better</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2427">HDFS-2427</a>.
+ Major sub-task reported by Arpit Gupta and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>webhdfs mkdirs api call creates path with 777 permission, we should default it to 755</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2422">HDFS-2422</a>.
+ Major bug reported by Jeff Bean and fixed by Aaron T. Myers (name-node)<br>
+ <b>The NN should tolerate the same number of low-resource volumes as failed volumes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2416">HDFS-2416</a>.
+ Major sub-task reported by Arpit Gupta and fixed by Jitendra Nath Pandey (webhdfs)<br>
+ <b>distcp with a webhdfs uri on a secure cluster fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2414">HDFS-2414</a>.
+ Critical bug reported by Robert Joseph Evans and fixed by Todd Lipcon (name-node , test)<br>
+ <b>TestDFSRollback fails intermittently</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2412">HDFS-2412</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Add backwards-compatibility layer for FSConstants</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2411">HDFS-2411</a>.
+ Major bug reported by Arpit Gupta and fixed by Jitendra Nath Pandey (webhdfs)<br>
+ <b>with webhdfs enabled in secure mode the auth to local mappings are not being respected.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2409">HDFS-2409</a>.
+ Major bug reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>_HOST in dfs.web.authentication.kerberos.principal.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2404">HDFS-2404</a>.
+ Major sub-task reported by Arpit Gupta and fixed by Suresh Srinivas (webhdfs)<br>
+ <b>webhdfs liststatus json response is not correct</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2403">HDFS-2403</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE <br>
+ <b>The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2401">HDFS-2401</a>.
+ Major improvement reported by Jonathan Eagles and fixed by Jonathan Eagles (build)<br>
+ <b>Running a set of methods in a Single Test Class</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2395">HDFS-2395</a>.
+ Critical sub-task reported by Arpit Gupta and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>webhdfs api's should return a root element in the json response</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2385">HDFS-2385</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>Support delegation token renewal in webhdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2371">HDFS-2371</a>.
+ Major improvement reported by Suresh Srinivas and fixed by Suresh Srinivas (data-node)<br>
+ <b>Refactor BlockSender.java for better readability</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2368">HDFS-2368</a>.
+ Major bug reported by Arpit Gupta and fixed by Tsz Wo (Nicholas), SZE <br>
+ <b>defaults created for web keytab and principal, these properties should not have defaults</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2366">HDFS-2366</a>.
+ Major sub-task reported by Arpit Gupta and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>webhdfs throws a npe when ugi is null from getDelegationToken</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2363">HDFS-2363</a>.
+ Minor sub-task reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (name-node)<br>
+ <b>Move datanodes size printing to BlockManager from FSNameSystem's metasave API</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2361">HDFS-2361</a>.
+ Critical bug reported by Rajit Saha and fixed by Jitendra Nath Pandey (name-node)<br>
+ <b>hftp is broken</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2356">HDFS-2356</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>webhdfs: support case insensitive query parameter names</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2355">HDFS-2355</a>.
+ Major improvement reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>Federation: enable using the same configuration file across all the nodes in the cluster.</b><br>
+ <blockquote>This change allows when running multiple namenodes on different hosts, sharing the same configuration file across all the nodes in the cluster (Datanodes, NamNode, BackupNode, SecondaryNameNode), without the need to define dfs.federation.nameservice.id parameter.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2348">HDFS-2348</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>Support getContentSummary and getFileChecksum in webhdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2347">HDFS-2347</a>.
+ Trivial bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (name-node)<br>
+ <b>checkpointTxnCount's comment still saying about editlog size</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2346">HDFS-2346</a>.
+ Blocker bug reported by Uma Maheswara Rao G and fixed by Laxman (test)<br>
+ <b>TestHost2NodesMap & TestReplicasMap will fail depending upon execution order of test methods</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2344">HDFS-2344</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (test)<br>
+ <b>Fix the TestOfflineEditsViewer test failure in 0.23 branch</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2340">HDFS-2340</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>Support getFileBlockLocations and getDelegationToken in webhdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2338">HDFS-2338</a>.
+ Major sub-task reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey (webhdfs)<br>
+ <b>Configuration option to enable/disable webhdfs.</b><br>
+ <blockquote>Added a conf property dfs.webhdfs.enabled for enabling/disabling webhdfs.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2333">HDFS-2333</a>.
+ Major bug reported by Ivan Kelly and fixed by Tsz Wo (Nicholas), SZE <br>
+ <b>HDFS-2284 introduced 2 findbugs warnings on trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2332">HDFS-2332</a>.
+ Major test reported by Todd Lipcon and fixed by Todd Lipcon (test)<br>
+ <b>Add test for HADOOP-7629: using an immutable FsPermission as an IPC parameter</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2331">HDFS-2331</a>.
+ Major bug reported by Abhijit Suresh Shingate and fixed by Abhijit Suresh Shingate (hdfs client)<br>
+ <b>Hdfs compilation fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2323">HDFS-2323</a>.
+ Major bug reported by Tom White and fixed by Tom White <br>
+ <b>start-dfs.sh script fails for tarball install</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2322">HDFS-2322</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>the build fails in Windows because commons-daemon TAR cannot be fetched</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2318">HDFS-2318</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (webhdfs)<br>
+ <b>Provide authentication to webhdfs using SPNEGO</b><br>
+ <blockquote>Added two new conf properties dfs.web.authentication.kerberos.principal and dfs.web.authentication.kerberos.keytab for the SPNEGO servlet filter.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2317">HDFS-2317</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE <br>
+ <b>Read access to HDFS using HTTP REST</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2314">HDFS-2314</a>.
+ Major bug reported by Vinod Kumar Vavilapalli and fixed by Todd Lipcon (test)<br>
+ <b>MRV1 test compilation broken after HDFS-2197</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2294">HDFS-2294</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Download of commons-daemon TAR should not be under target</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2290">HDFS-2290</a>.
+ Major bug reported by Konstantin Shvachko and fixed by Benoy Antony (name-node)<br>
+ <b>Block with corrupt replica is not getting replicated</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2289">HDFS-2289</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Alejandro Abdelnur <br>
+ <b>jsvc isn't part of the artifact</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2286">HDFS-2286</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>DataXceiverServer logs AsynchronousCloseException at shutdown</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2284">HDFS-2284</a>.
+ Major sub-task reported by Sanjay Radia and fixed by Tsz Wo (Nicholas), SZE <br>
+ <b>Write Http access to HDFS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2273">HDFS-2273</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Refactor BlockManager.recentInvalidateSets to a new class</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2267">HDFS-2267</a>.
+ Trivial bug reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>DataXceiver thread name incorrect while waiting on op during keepalive</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2266">HDFS-2266</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Add a Namesystem interface to avoid directly referring to FSNamesystem</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2265">HDFS-2265</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Remove unnecessary BlockTokenSecretManager fields/methods from BlockManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2260">HDFS-2260</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (hdfs client)<br>
+ <b>Refactor BlockReader into an interface and implementation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2258">HDFS-2258</a>.
+ Major bug reported by Konstantin Shvachko and fixed by Konstantin Shvachko (name-node , test)<br>
+ <b>TestLeaseRecovery2 fails as lease hard limit is not reset to default</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2245">HDFS-2245</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>BlockManager.chooseTarget(..) throws NPE</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2241">HDFS-2241</a>.
+ Major improvement reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>Remove implementing FSConstants interface just to access the constants defined in the interface</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2240">HDFS-2240</a>.
+ Critical bug reported by Todd Lipcon and fixed by Tsz Wo (Nicholas), SZE (hdfs client)<br>
+ <b>Possible deadlock between LeaseRenewer and its factory</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2239">HDFS-2239</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Reduce access levels of the fields and methods in FSNamesystem</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2238">HDFS-2238</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Uma Maheswara Rao G (name-node)<br>
+ <b>NamenodeFsck.toString() uses StringBuilder with + operator </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2237">HDFS-2237</a>.
+ Minor sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Change UnderReplicatedBlocks from public to package private</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2235">HDFS-2235</a>.
+ Major bug reported by Eli Collins and fixed by Eli Collins (name-node)<br>
+ <b>Encode servlet paths</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2233">HDFS-2233</a>.
+ Major test reported by Eli Collins and fixed by Eli Collins (name-node)<br>
+ <b>Add WebUI tests with URI reserved chars in the path and filename</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2232">HDFS-2232</a>.
+ Blocker bug reported by Konstantin Shvachko and fixed by Plamen Jeliazkov (test)<br>
+ <b>TestHDFSCLI fails on 0.22 branch</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2230">HDFS-2230</a>.
+ Major improvement reported by Giridharan Kesavan and fixed by Giridharan Kesavan (build)<br>
+ <b>hdfs it not resolving the latest common test jars published post common mavenization</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2229">HDFS-2229</a>.
+ Blocker bug reported by Vinod Kumar Vavilapalli and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Deadlock in NameNode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2228">HDFS-2228</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Move block and datanode code from FSNamesystem to BlockManager and DatanodeManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2227">HDFS-2227</a>.
+ Major improvement reported by Ivan Kelly and fixed by Ivan Kelly <br>
+ <b>HDFS-2018 Part 2 : getRemoteEditLogManifest should pull it's information from FileJournalManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2226">HDFS-2226</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Clean up counting of operations in FSEditLogLoader</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2225">HDFS-2225</a>.
+ Major improvement reported by Ivan Kelly and fixed by Ivan Kelly <br>
+ <b>HDFS-2018 Part 1 : Refactor file management so its not in classes which should be generic</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2212">HDFS-2212</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Refactor double-buffering code out of EditLogOutputStreams</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2210">HDFS-2210</a>.
+ Major task reported by Eli Collins and fixed by Eli Collins (contrib/hdfsproxy)<br>
+ <b>Remove hdfsproxy</b><br>
+ <blockquote>The hdfsproxy contrib component is no longer supported.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2209">HDFS-2209</a>.
+ Minor improvement reported by Steve Loughran and fixed by Steve Loughran (test)<br>
+ <b>Make MiniDFS easier to embed in other apps</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2205">HDFS-2205</a>.
+ Major improvement reported by Ravi Prakash and fixed by Ravi Prakash (hdfs client)<br>
+ <b>Log message for failed connection to datanode is not followed by a success message.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2202">HDFS-2202</a>.
+ Major new feature reported by Eric Payne and fixed by Eric Payne (balancer , data-node)<br>
+ <b>Changes to balancer bandwidth should not require datanode restart.</b><br>
+ <blockquote>New dfsadmin command added: [-setBalancerBandwidth <bandwidth>] where bandwidth is max network bandwidth in bytes per second that the balancer is allowed to use on each datanode during balacing.
+
+This is an incompatible change in 0.23. The versions of ClientProtocol and DatanodeProtocol are changed.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2200">HDFS-2200</a>.
+ Minor sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Set FSNamesystem.LOG to package private</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2199">HDFS-2199</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Uma Maheswara Rao G (name-node)<br>
+ <b>Move blockTokenSecretManager from FSNamesystem to BlockManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2198">HDFS-2198</a>.
+ Minor improvement reported by Suresh Srinivas and fixed by Suresh Srinivas (data-node , hdfs client , name-node)<br>
+ <b>Remove hardcoded configuration keys</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2197">HDFS-2197</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Refactor RPC call implementations out of NameNode class</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2196">HDFS-2196</a>.
+ Major task reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Make ant build system work with hadoop-common JAR generated by Maven</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2191">HDFS-2191</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Move datanodeMap from FSNamesystem to DatanodeManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2187">HDFS-2187</a>.
+ Major improvement reported by Ivan Kelly and fixed by Ivan Kelly <br>
+ <b>HDFS-1580: Make EditLogInputStream act like an iterator over FSEditLogOps</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2186">HDFS-2186</a>.
+ Major bug reported by Eli Collins and fixed by Eli Collins (data-node)<br>
+ <b>DN volume failures on startup are not counted</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2180">HDFS-2180</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Refactor NameNode HTTP server into new class</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2167">HDFS-2167</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Move dnsToSwitchMapping and hostsReader from FSNamesystem to DatanodeManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2161">HDFS-2161</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (balancer , data-node , hdfs client , name-node , security)<br>
+ <b>Move utilities to DFSUtil</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2159">HDFS-2159</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (hdfs client)<br>
+ <b>Deprecate DistributedFileSystem.getClient()</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2157">HDFS-2157</a>.
+ Major improvement reported by Aaron T. Myers and fixed by Aaron T. Myers (documentation , name-node)<br>
+ <b>Improve header comment in o.a.h.hdfs.server.namenode.NameNode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2156">HDFS-2156</a>.
+ Major bug reported by Owen O'Malley and fixed by Eric Yang <br>
+ <b>rpm should only require the same major version as common</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2154">HDFS-2154</a>.
+ Minor test reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (test)<br>
+ <b>TestDFSShell should use test dir</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2153">HDFS-2153</a>.
+ Minor bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (test)<br>
+ <b>DFSClientAdapter should be put under test</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2149">HDFS-2149</a>.
+ Major sub-task reported by Ivan Kelly and fixed by Ivan Kelly (name-node)<br>
+ <b>Move EditLogOp serialization formats into FsEditLogOp implementations</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2147">HDFS-2147</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Move cluster network topology to block management</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2144">HDFS-2144</a>.
+ Major improvement reported by Ravi Prakash and fixed by Ravi Prakash (name-node)<br>
+ <b>If SNN shuts down during initialization it does not log the cause</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2143">HDFS-2143</a>.
+ Major improvement reported by Ravi Prakash and fixed by Ravi Prakash <br>
+ <b>Federation: we should link to the live nodes and dead nodes to cluster web console</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2141">HDFS-2141</a>.
+ Major sub-task reported by Suresh Srinivas and fixed by Suresh Srinivas (ha , name-node)<br>
+ <b>Remove NameNode roles Active and Standby (they become states)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2140">HDFS-2140</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Move Host2NodesMap to block management</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2134">HDFS-2134</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Move DecommissionManager to block management</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2132">HDFS-2132</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers <br>
+ <b>Potential resource leak in EditLogFileOutputStream.close</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2131">HDFS-2131</a>.
+ Major test reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (test)<br>
+ <b>Tests for HADOOP-7361</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2118">HDFS-2118</a>.
+ Minor improvement reported by Eli Collins and fixed by Eli Collins (data-node)<br>
+ <b>Couple dfs data dir improvements</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2116">HDFS-2116</a>.
+ Minor improvement reported by Eli Collins and fixed by Plamen Jeliazkov (test)<br>
+ <b>Cleanup TestStreamFile and TestByteRangeInputStream </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2114">HDFS-2114</a>.
+ Major bug reported by John George and fixed by John George <br>
+ <b>re-commission of a decommissioned node does not delete excess replica</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2112">HDFS-2112</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Uma Maheswara Rao G (name-node)<br>
+ <b>Move ReplicationMonitor to block management</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2111">HDFS-2111</a>.
+ Major test reported by Harsh J and fixed by Harsh J (data-node , test)<br>
+ <b>Add tests for ensuring that the DN will start with a few bad data directories (Part 1 of testing DiskChecker)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2110">HDFS-2110</a>.
+ Minor improvement reported by Eli Collins and fixed by Eli Collins (name-node)<br>
+ <b>Some StreamFile and ByteRangeInputStream cleanup</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2109">HDFS-2109</a>.
+ Major bug reported by Bharath Mundlapudi and fixed by Bharath Mundlapudi (hdfs client)<br>
+ <b>Store uMask as member variable to DFSClient.Conf</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2108">HDFS-2108</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Move datanode heartbeat handling to BlockManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2107">HDFS-2107</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Move block management code to a package</b><br>
+ <blockquote>Moved block management codes to a new package org.apache.hadoop.hdfs.server.blockmanagement.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2100">HDFS-2100</a>.
+ Minor test reported by Aaron T. Myers and fixed by Aaron T. Myers (test)<br>
+ <b>Improve TestStorageRestore</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2096">HDFS-2096</a>.
+ Major task reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Mavenization of hadoop-hdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2092">HDFS-2092</a>.
+ Major bug reported by Bharath Mundlapudi and fixed by Bharath Mundlapudi (hdfs client)<br>
+ <b>Create a light inner conf class in DFSClient</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2086">HDFS-2086</a>.
+ Major bug reported by Tanping Wang and fixed by Tanping Wang (name-node)<br>
+ <b>If the include hosts list contains host name, after restarting namenode, datanodes registrant is denied </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2083">HDFS-2083</a>.
+ Major new feature reported by Tanping Wang and fixed by Tanping Wang <br>
+ <b>Adopt JMXJsonServlet into HDFS in order to query statistics</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2082">HDFS-2082</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers <br>
+ <b>SecondaryNameNode web interface doesn't show the right info</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2073">HDFS-2073</a>.
+ Minor improvement reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>Namenode is missing @Override annotations</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2069">HDFS-2069</a>.
+ Trivial sub-task reported by Ravi Phulari and fixed by Harsh J (documentation)<br>
+ <b>Incorrect default trash interval value in the docs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2067">HDFS-2067</a>.
+ Major bug reported by Todd Lipcon and fixed by Tsz Wo (Nicholas), SZE (data-node , hdfs client)<br>
+ <b>Bump DATA_TRANSFER_VERSION in trunk for protobufs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2066">HDFS-2066</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node , hdfs client , name-node)<br>
+ <b>Create a package and individual class files for DataTransferProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2065">HDFS-2065</a>.
+ Major bug reported by Bharath Mundlapudi and fixed by Uma Maheswara Rao G <br>
+ <b>Fix NPE in DFSClient.getFileChecksum</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2061">HDFS-2061</a>.
+ Minor bug reported by Matt Foley and fixed by Matt Foley (name-node)<br>
+ <b>two minor bugs in BlockManager block report processing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2058">HDFS-2058</a>.
+ Major new feature reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>DataTransfer Protocol using protobufs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2056">HDFS-2056</a>.
+ Minor improvement reported by Tanping Wang and fixed by Tanping Wang (documentation , tools)<br>
+ <b>Update fetchdt usage</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2055">HDFS-2055</a>.
+ Major new feature reported by Travis Crawford and fixed by Travis Crawford (libhdfs)<br>
+ <b>Add hflush support to libhdfs</b><br>
+ <blockquote>Add hdfsHFlush to libhdfs.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2054">HDFS-2054</a>.
+ Minor improvement reported by Kihwal Lee and fixed by Kihwal Lee (data-node)<br>
+ <b>BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully()</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2053">HDFS-2053</a>.
+ Minor bug reported by Michael Noll and fixed by Michael Noll (name-node)<br>
+ <b>Bug in INodeDirectory#computeContentSummary warning</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2046">HDFS-2046</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (build , test)<br>
+ <b>Force entropy to come from non-true random for tests</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2041">HDFS-2041</a>.
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Some mtimes and atimes are lost when edit logs are replayed</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2040">HDFS-2040</a>.
+ Minor improvement reported by Eli Collins and fixed by Eli Collins <br>
+ <b>Only build libhdfs if a flag is passed</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2034">HDFS-2034</a>.
+ Minor bug reported by John George and fixed by John George (hdfs client)<br>
+ <b>length in getBlockRange becomes -ve when reading only from currently being written blk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2030">HDFS-2030</a>.
+ Minor bug reported by Bharath Mundlapudi and fixed by Bharath Mundlapudi <br>
+ <b>Fix the usability of namenode upgrade command</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2029">HDFS-2029</a>.
+ Trivial improvement reported by Tsz Wo (Nicholas), SZE and fixed by John George (test)<br>
+ <b>Improve TestWriteRead</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2024">HDFS-2024</a>.
+ Trivial improvement reported by CW Chung and fixed by CW Chung (test)<br>
+ <b>Eclipse format HDFS Junit test hdfs/TestWriteRead.java </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2022">HDFS-2022</a>.
+ Major bug reported by Eli Collins and fixed by Eric Yang (build)<br>
+ <b>ant binary should build libhdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2021">HDFS-2021</a>.
+ Major bug reported by CW Chung and fixed by John George (data-node)<br>
+ <b>TestWriteRead failed with inconsistent visible length of a file </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2020">HDFS-2020</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas (data-node , test)<br>
+ <b>TestDFSUpgradeFromImage fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2019">HDFS-2019</a>.
+ Minor bug reported by Bharath Mundlapudi and fixed by Bharath Mundlapudi (data-node)<br>
+ <b>Fix all the places where Java method File.list is used with FileUtil.list API</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2014">HDFS-2014</a>.
+ Critical bug reported by Todd Lipcon and fixed by Eric Yang (scripts)<br>
+ <b>bin/hdfs no longer works from a source checkout</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2011">HDFS-2011</a>.
+ Major bug reported by Ravi Prakash and fixed by Ravi Prakash (name-node)<br>
+ <b>Removal and restoration of storage directories on checkpointing failure doesn't work properly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2003">HDFS-2003</a>.
+ Major improvement reported by Ivan Kelly and fixed by Ivan Kelly <br>
+ <b>Separate FSEditLog reading logic from editLog memory state building logic</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-2002">HDFS-2002</a>.
+ Major bug reported by Konstantin Shvachko and fixed by Plamen Jeliazkov (name-node)<br>
+ <b>Incorrect computation of needed blocks in getTurnOffTip()</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1999">HDFS-1999</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (test)<br>
+ <b>Tests use deprecated configs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1998">HDFS-1998</a>.
+ Minor bug reported by Tanping Wang and fixed by Tanping Wang (scripts)<br>
+ <b>make refresh-namodenodes.sh refreshing all namenodes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1996">HDFS-1996</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Eric Yang (build)<br>
+ <b>ivy: hdfs test jar should be independent to common test jar</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1995">HDFS-1995</a>.
+ Minor improvement reported by Tanping Wang and fixed by Tanping Wang <br>
+ <b>Minor modification to both dfsclusterhealth and dfshealth pages for Web UI</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1990">HDFS-1990</a>.
+ Minor bug reported by ramkrishna.s.vasudevan and fixed by Uma Maheswara Rao G (data-node)<br>
+ <b>Resource leaks in HDFS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1986">HDFS-1986</a>.
+ Minor bug reported by Tanping Wang and fixed by Tanping Wang (tools)<br>
+ <b>Add an option for user to return http or https ports regardless of security is on/off in DFSUtil.getInfoServer()</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1983">HDFS-1983</a>.
+ Major test reported by Daryn Sharp and fixed by Daryn Sharp (test)<br>
+ <b>Fix path display for copy & rm</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1968">HDFS-1968</a>.
+ Minor test reported by CW Chung and fixed by CW Chung (test)<br>
+ <b>Enhance TestWriteRead to support File Append and Position Read </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1966">HDFS-1966</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node , hdfs client)<br>
+ <b>Encapsulate individual DataTransferProtocol op header</b><br>
+ <blockquote>Added header classes for individual DataTransferProtocol op headers.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1964">HDFS-1964</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers <br>
+ <b>Incorrect HTML unescaping in DatanodeJspHelper.java</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1963">HDFS-1963</a>.
+ Major new feature reported by Eric Yang and fixed by Eric Yang (build)<br>
+ <b>HDFS rpm integration project</b><br>
+ <blockquote>Create HDFS RPM package</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1959">HDFS-1959</a>.
+ Minor improvement reported by Eli Collins and fixed by Eli Collins <br>
+ <b>Better error message for missing namenode directory</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1958">HDFS-1958</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Format confirmation prompt should be more lenient of its input</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1955">HDFS-1955</a>.
+ Major bug reported by Matt Foley and fixed by Matt Foley (name-node)<br>
+ <b>FSImage.doUpgrade() was made too fault-tolerant by HDFS-1826</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1953">HDFS-1953</a>.
+ Minor bug reported by Tanping Wang and fixed by Tanping Wang <br>
+ <b>Change name node mxbean name in cluster web console</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1952">HDFS-1952</a>.
+ Major bug reported by Matt Foley and fixed by Andrew Wang <br>
+ <b>FSEditLog.open() appears to succeed even if all EDITS directories fail</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1945">HDFS-1945</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node , hdfs client)<br>
+ <b>Removed deprecated fields in DataTransferProtocol</b><br>
+ <blockquote>Removed the deprecated fields in DataTransferProtocol.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1943">HDFS-1943</a>.
+ Blocker bug reported by Wei Yongjun and fixed by Matt Foley (scripts)<br>
+ <b>fail to start datanode while start-dfs.sh is executed by root user</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1939">HDFS-1939</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Eric Yang (build)<br>
+ <b>ivy: test conf should not extend common conf</b><br>
+ <blockquote>* Removed duplicated jars in test class path.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1938">HDFS-1938</a>.
+ Minor bug reported by Tsz Wo (Nicholas), SZE and fixed by Eric Yang (build)<br>
+ <b> Reference ivy-hdfs.classpath not found.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1937">HDFS-1937</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node , hdfs client)<br>
+ <b>Umbrella JIRA for improving DataTransferProtocol</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1936">HDFS-1936</a>.
+ Blocker bug reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>Updating the layout version from HDFS-1822 causes upgrade problems.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1934">HDFS-1934</a>.
+ Major bug reported by Bharath Mundlapudi and fixed by Bharath Mundlapudi <br>
+ <b>Fix NullPointerException when File.listFiles() API returns null</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1933">HDFS-1933</a>.
+ Major test reported by Daryn Sharp and fixed by Daryn Sharp (test)<br>
+ <b>Update tests for FsShell's "test"</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1931">HDFS-1931</a>.
+ Major test reported by Daryn Sharp and fixed by Daryn Sharp <br>
+ <b>Update tests for du/dus/df</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1928">HDFS-1928</a>.
+ Major test reported by Daryn Sharp and fixed by Daryn Sharp (test)<br>
+ <b>Fix path display for touchz</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1927">HDFS-1927</a>.
+ Major bug reported by John George and fixed by John George (name-node)<br>
+ <b>audit logs could ignore certain xsactions and also could contain "ip=null"</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1923">HDFS-1923</a>.
+ Major sub-task reported by Matt Foley and fixed by Tsz Wo (Nicholas), SZE (test)<br>
+ <b>Intermittent recurring failure in TestFiDataTransferProtocol2.pipeline_Fi_29</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1922">HDFS-1922</a>.
+ Major sub-task reported by Matt Foley and fixed by Luke Lu (test)<br>
+ <b>Recurring failure in TestJMXGet.testNameNode since build 477 on May 11</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1921">HDFS-1921</a>.
+ Blocker bug reported by Aaron T. Myers and fixed by Matt Foley <br>
+ <b>Save namespace can cause NN to be unable to come up on restart</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1920">HDFS-1920</a>.
+ Major bug reported by Trevor Robinson and fixed by Trevor Robinson (libhdfs)<br>
+ <b>libhdfs does not build for ARM processors</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1917">HDFS-1917</a>.
+ Major bug reported by Eric Yang and fixed by Eric Yang (build)<br>
+ <b>Clean up duplication of dependent jar files</b><br>
+ <blockquote>Remove packaging of duplicated third party jar files</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1914">HDFS-1914</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>Federation: namenode storage directory must be configurable specific to a namenode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1912">HDFS-1912</a>.
+ Major test reported by Daryn Sharp and fixed by Daryn Sharp (test)<br>
+ <b>Update tests for FsShell standardized error messages</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1911">HDFS-1911</a>.
+ Major test reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>HDFS tests for viewfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1908">HDFS-1908</a>.
+ Minor bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (test)<br>
+ <b>DataTransferTestUtil$CountdownDoosAction.run(..) throws NullPointerException</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1907">HDFS-1907</a>.
+ Major bug reported by CW Chung and fixed by John George (hdfs client)<br>
+ <b>BlockMissingException upon concurrent read and write: reader was doing file position read while writer is doing write without hflush</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1906">HDFS-1906</a>.
+ Minor improvement reported by Suresh Srinivas and fixed by Suresh Srinivas (hdfs client)<br>
+ <b>Remove logging exception stack trace when one of the datanode targets to read from is not reachable</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1905">HDFS-1905</a>.
+ Minor bug reported by Bharath Mundlapudi and fixed by Bharath Mundlapudi (name-node)<br>
+ <b>Improve the usability of namenode -format </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1903">HDFS-1903</a>.
+ Major test reported by Daryn Sharp and fixed by Daryn Sharp (test)<br>
+ <b>Fix path display for rm/rmr</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1902">HDFS-1902</a>.
+ Major test reported by Daryn Sharp and fixed by Daryn Sharp (test)<br>
+ <b>Fix path display for setrep</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1899">HDFS-1899</a>.
+ Major improvement reported by Todd Lipcon and fixed by Ted Yu <br>
+ <b>GenericTestUtils.formatNamenode is misplaced</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1898">HDFS-1898</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Tests failing on trunk due to use of NameNode.format</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1890">HDFS-1890</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (hdfs client)<br>
+ <b>A few improvements on the LeaseRenewer.pendingCreates map</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1889">HDFS-1889</a>.
+ Major bug reported by John George and fixed by John George <br>
+ <b>incorrect path in start/stop dfs script</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1888">HDFS-1888</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas <br>
+ <b>MiniDFSCluster#corruptBlockOnDatanodes() access must be public for MapReduce contrib raid</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1884">HDFS-1884</a>.
+ Major sub-task reported by Matt Foley and fixed by Aaron T. Myers (test)<br>
+ <b>Improve TestDFSStorageStateRecovery</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1883">HDFS-1883</a>.
+ Major sub-task reported by Matt Foley and fixed by (test)<br>
+ <b>Recurring failures in TestBackupNode since HDFS-1052</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1881">HDFS-1881</a>.
+ Major bug reported by Tanping Wang and fixed by Tanping Wang (data-node)<br>
+ <b>Federation: after taking snapshot the current directory of datanode is empty</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1877">HDFS-1877</a>.
+ Minor test reported by CW Chung and fixed by CW Chung (test)<br>
+ <b>Create a functional test for file read/write</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1876">HDFS-1876</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>One MiniDFSCluster ignores numDataNodes parameter</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1875">HDFS-1875</a>.
+ Major bug reported by Eric Payne and fixed by Eric Payne (test)<br>
+ <b>MiniDFSCluster hard-codes dfs.datanode.address to localhost</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1873">HDFS-1873</a>.
+ Major new feature reported by Tanping Wang and fixed by Tanping Wang <br>
+ <b>Federation Cluster Management Web Console</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1871">HDFS-1871</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas (test)<br>
+ <b>Tests using MiniDFSCluster fail to compile due to HDFS-1052 changes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1870">HDFS-1870</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (hdfs client)<br>
+ <b>Refactor DFSClient.LeaseChecker</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1869">HDFS-1869</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (name-node)<br>
+ <b>mkdirs should use the supplied permission for all of the created directories</b><br>
+ <blockquote>A multi-level mkdir is now POSIX compliant. Instead of creating intermediate directories with the permissions of the parent directory, intermediate directories are created with permission bits of rwxrwxrwx (0777) as modified by the current umask, plus write and search permission for the owner.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1865">HDFS-1865</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (hdfs client)<br>
+ <b>Share LeaseChecker thread among DFSClients</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1862">HDFS-1862</a>.
+ Major test reported by Aaron T. Myers and fixed by Aaron T. Myers (test)<br>
+ <b>Improve test reliability of HDFS-1594</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1861">HDFS-1861</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins (data-node)<br>
+ <b>Rename dfs.datanode.max.xcievers and bump its default value</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1856">HDFS-1856</a>.
+ Major sub-task reported by Matt Foley and fixed by Matt Foley (test)<br>
+ <b>TestDatanodeBlockScanner waits forever, errs without giving information</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1855">HDFS-1855</a>.
+ Major test reported by Matt Foley and fixed by Matt Foley (test)<br>
+ <b>TestDatanodeBlockScanner.testBlockCorruptionRecoveryPolicy() part 2 fails in two different ways</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1854">HDFS-1854</a>.
+ Major sub-task reported by Matt Foley and fixed by Matt Foley (test)<br>
+ <b>make failure message more useful in DFSTestUtil.waitReplication()</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1846">HDFS-1846</a>.
+ Major improvement reported by Aaron T. Myers and fixed by Aaron T. Myers (name-node)<br>
+ <b>Don't fill preallocated portion of edits log with 0x00</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1845">HDFS-1845</a>.
+ Major bug reported by John George and fixed by John George <br>
+ <b>symlink comes up as directory after namenode restart</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1844">HDFS-1844</a>.
+ Major test reported by Daryn Sharp and fixed by Daryn Sharp (test)<br>
+ <b>Move -fs usage tests from hdfs into common</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1843">HDFS-1843</a>.
+ Minor improvement reported by Bharath Mundlapudi and fixed by Bharath Mundlapudi <br>
+ <b>Discover file not found early for file append </b><br>
+ <blockquote>I have committed this. Thanks to Bharath!</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1840">HDFS-1840</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (hdfs client)<br>
+ <b>Terminate LeaseChecker when all writing files are closed.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1835">HDFS-1835</a>.
+ Major bug reported by John Carrino and fixed by John Carrino (data-node)<br>
+ <b>DataNode.setNewStorageID pulls entropy from /dev/random</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1833">HDFS-1833</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Refactor BlockReceiver</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1831">HDFS-1831</a>.
+ Major improvement reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>HDFS equivalent of HADOOP-7223 changes to handle FileContext createFlag combinations</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1829">HDFS-1829</a>.
+ Major bug reported by Matt Foley and fixed by Matt Foley (name-node)<br>
+ <b>TestNodeCount waits forever, errs without giving information</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1827">HDFS-1827</a>.
+ Major bug reported by Matt Foley and fixed by Matt Foley (name-node)<br>
+ <b>TestBlockReplacement waits forever, errs without giving information</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1826">HDFS-1826</a>.
+ Major sub-task reported by Hairong Kuang and fixed by Matt Foley (name-node)<br>
+ <b>NameNode should save image to name directories in parallel during upgrade</b><br>
+ <blockquote>I've committed this. Thanks, Matt!</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1823">HDFS-1823</a>.
+ Blocker bug reported by Tom White and fixed by Tom White (scripts)<br>
+ <b>start-dfs.sh script fails if HADOOP_HOME is not set</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1822">HDFS-1822</a>.
+ Blocker bug reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>Editlog opcodes overlap between 20 security and later releases</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1821">HDFS-1821</a>.
+ Major bug reported by John George and fixed by John George <br>
+ <b>FileContext.createSymlink with kerberos enabled sets wrong owner</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1818">HDFS-1818</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (test)<br>
+ <b>TestHDFSCLI is failing on trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1817">HDFS-1817</a>.
+ Trivial improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (test)<br>
+ <b>Split TestFiDataTransferProtocol.java into two files</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1814">HDFS-1814</a>.
+ Major new feature reported by Aaron T. Myers and fixed by Aaron T. Myers (hdfs client , name-node)<br>
+ <b>HDFS portion of HADOOP-7214 - Hadoop /usr/bin/groups equivalent</b><br>
+ <blockquote>Introduces a new command, "hdfs groups", which displays what groups are associated with a user as seen by the NameNode.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1812">HDFS-1812</a>.
+ Minor bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (test)<br>
+ <b>Address the cleanup issues in TestHDFSCLI.java</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1808">HDFS-1808</a>.
+ Major bug reported by Matt Foley and fixed by Matt Foley (data-node , name-node)<br>
+ <b>TestBalancer waits forever, errs without giving information</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1806">HDFS-1806</a>.
+ Major bug reported by Matt Foley and fixed by Matt Foley (data-node , name-node)<br>
+ <b>TestBlockReport.blockReport_08() and _09() are timing-dependent and likely to fail on fast servers</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1797">HDFS-1797</a>.
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>New findbugs warning introduced by HDFS-1120</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1789">HDFS-1789</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node , hdfs client)<br>
+ <b>Refactor frequently used codes from DFSOutputStream, BlockReceiver and DataXceiver</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1786">HDFS-1786</a>.
+ Minor bug reported by Tsz Wo (Nicholas), SZE and fixed by Uma Maheswara Rao G (test)<br>
+ <b>Some cli test cases expect a "null" message</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1785">HDFS-1785</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Cleanup BlockReceiver and DataXceiver</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1782">HDFS-1782</a>.
+ Major bug reported by John George and fixed by John George (name-node)<br>
+ <b>FSNamesystem.startFileInternal(..) throws NullPointerException</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1781">HDFS-1781</a>.
+ Major bug reported by John George and fixed by John George (scripts)<br>
+ <b>jsvc executable delivered into wrong package...</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1776">HDFS-1776</a>.
+ Major bug reported by Dmytro Molkov and fixed by Bharath Mundlapudi <br>
+ <b>Bug in Concat code</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1774">HDFS-1774</a>.
+ Minor improvement reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (data-node)<br>
+ <b>Small optimization to FSDataset</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1773">HDFS-1773</a>.
+ Minor improvement reported by Tanping Wang and fixed by Tanping Wang (name-node)<br>
+ <b>Remove a datanode from cluster if include list is not empty and this datanode is removed from both include and exclude lists</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1770">HDFS-1770</a>.
+ Minor test reported by Eli Collins and fixed by Eli Collins <br>
+ <b>TestFiRename fails due to invalid block size</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1767">HDFS-1767</a>.
+ Major sub-task reported by Matt Foley and fixed by Matt Foley (data-node)<br>
+ <b>Namenode should ignore non-initial block reports from datanodes when in safemode during startup</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1763">HDFS-1763</a>.
+ Minor improvement reported by Eli Collins and fixed by Eli Collins <br>
+ <b>Replace hard-coded option strings with variables from DFSConfigKeys</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1761">HDFS-1761</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Add a new DataTransferProtocol operation, Op.TRANSFER_BLOCK, instead of using RPC</b><br>
+ <blockquote>Add a new DataTransferProtocol operation, Op.TRANSFER_BLOCK, for transferring RBW/Finalized with acknowledgement and without using RPC.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1760">HDFS-1760</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (name-node)<br>
+ <b>problems with getFullPathName</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1757">HDFS-1757</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins (fuse-dfs)<br>
+ <b>Don't compile fuse-dfs by default</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1751">HDFS-1751</a>.
+ Major new feature reported by Daryn Sharp and fixed by Daryn Sharp (data-node)<br>
+ <b>Intrinsic limits for HDFS files, directories</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1750">HDFS-1750</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE <br>
+ <b>fs -ls hftp://file not working</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1748">HDFS-1748</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (balancer)<br>
+ <b>Balancer utilization classification is incomplete</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1741">HDFS-1741</a>.
+ Major improvement reported by Konstantin Boudnik and fixed by Konstantin Boudnik (build)<br>
+ <b>Provide a minimal pom file to allow integration of HDFS into Sonar analysis</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1739">HDFS-1739</a>.
+ Minor improvement reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (data-node)<br>
+ <b>When DataNode throws DiskOutOfSpaceException, it will be helpfull to the user if we log the available volume size and configured block size.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1734">HDFS-1734</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (name-node)<br>
+ <b>'Chunk size to view' option is not working in Name Node UI.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1731">HDFS-1731</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Allow using a file to exclude certain tests from build</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1728">HDFS-1728</a>.
+ Minor bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>SecondaryNameNode.checkpointSize is in byte but not MB.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1727">HDFS-1727</a>.
+ Minor bug reported by Uma Maheswara Rao G and fixed by sravankorumilli <br>
+ <b>fsck command can display command usage if user passes any illegal argument</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1723">HDFS-1723</a>.
+ Minor improvement reported by Allen Wittenauer and fixed by Jim Plush <br>
+ <b>quota errors messages should use the same scale</b><br>
+ <blockquote>Updated the Quota exceptions to now use human readable output.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1703">HDFS-1703</a>.
+ Minor sub-task reported by Tanping Wang and fixed by Tanping Wang (scripts)<br>
+ <b>HDFS federation: Improve start/stop scripts and add script to decommission datanodes</b><br>
+ <blockquote>The masters file is no longer used to indicate which hosts to start the 2NN on. The 2NN is now started on hosts when dfs.namenode.secondary.http-address is configured with a non-wildcard IP.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1692">HDFS-1692</a>.
+ Major bug reported by Bharath Mundlapudi and fixed by Bharath Mundlapudi (data-node)<br>
+ <b>In secure mode, Datanode process doesn't exit when disks fail.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1691">HDFS-1691</a>.
+ Minor bug reported by Alexey Diomin and fixed by Alexey Diomin (tools)<br>
+ <b>double static declaration in Configuration.addDefaultResource("hdfs-default.xml");</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1675">HDFS-1675</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node)<br>
+ <b>Transfer RBW between datanodes</b><br>
+ <blockquote>Added a new stage TRANSFER_RBW to DataTransferProtocol</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1665">HDFS-1665</a>.
+ Minor bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (balancer)<br>
+ <b>Balancer sleeps inadequately</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1656">HDFS-1656</a>.
+ Major bug reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>getDelegationToken in HftpFileSystem should renew TGT if needed.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1636">HDFS-1636</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Harsh J (name-node)<br>
+ <b>If dfs.name.dir points to an empty dir, namenode format shouldn't require confirmation</b><br>
+ <blockquote>If dfs.name.dir points to an empty dir, namenode -format no longer requires confirmation.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1630">HDFS-1630</a>.
+ Major improvement reported by Hairong Kuang and fixed by Hairong Kuang (name-node)<br>
+ <b>Checksum fsedits</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1629">HDFS-1629</a>.
+ Major sub-task reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Add a method to BlockPlacementPolicy for not removing the chosen nodes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1628">HDFS-1628</a>.
+ Minor improvement reported by Ramya Sunil and fixed by John George (name-node)<br>
+ <b>AccessControlException should display the full path</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1627">HDFS-1627</a>.
+ Major bug reported by Hairong Kuang and fixed by Hairong Kuang (name-node)<br>
+ <b>Fix NullPointerException in Secondary NameNode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1626">HDFS-1626</a>.
+ Minor improvement reported by Arun C Murthy and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>Make BLOCK_INVALIDATE_LIMIT configurable</b><br>
+ <blockquote>Added a new configuration property dfs.block.invalidate.limit for FSNamesystem.blockInvalidateLimit.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1625">HDFS-1625</a>.
+ Minor bug reported by Todd Lipcon and fixed by Tsz Wo (Nicholas), SZE (test)<br>
+ <b>TestDataNodeMXBean fails if disk space usage changes during test run</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1620">HDFS-1620</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Harsh J <br>
+ <b>Rename HdfsConstants -> HdfsServerConstants, FSConstants -> HdfsConstants</b><br>
+ <blockquote>Rename HdfsConstants interface to HdfsServerConstants, FSConstants interface to HdfsConstants</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1612">HDFS-1612</a>.
+ Minor bug reported by Joe Crobak and fixed by Joe Crobak (documentation)<br>
+ <b>HDFS Design Documentation is outdated</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1611">HDFS-1611</a>.
+ Minor bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (hdfs client , name-node)<br>
+ <b>Some logical issues need to address.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1606">HDFS-1606</a>.
+ Major new feature reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (data-node , hdfs client , name-node)<br>
+ <b>Provide a stronger data guarantee in the write pipeline</b><br>
+ <blockquote>Added two configuration properties, dfs.client.block.write.replace-datanode-on-failure.enable and dfs.client.block.write.replace-datanode-on-failure.policy. Added a new feature to replace datanode on failure in DataTransferProtocol. Added getAdditionalDatanode(..) in ClientProtocol.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1602">HDFS-1602</a>.
+ Major bug reported by Konstantin Boudnik and fixed by Boris Shkolnik (name-node)<br>
+ <b>NameNode storage failed replica restoration is broken</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1601">HDFS-1601</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>Pipeline ACKs are sent as lots of tiny TCP packets</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1600">HDFS-1600</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Todd Lipcon (build , test)<br>
+ <b>editsStored.xml cause release audit warning</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1598">HDFS-1598</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (name-node)<br>
+ <b>ListPathsServlet excludes .*.crc files</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1596">HDFS-1596</a>.
+ Major improvement reported by Patrick Angeles and fixed by Harsh J (documentation , name-node)<br>
+ <b>Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml</b><br>
+ <blockquote>Removed references to the older fs.checkpoint.* properties that resided in core-site.xml</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1594">HDFS-1594</a>.
+ Major bug reported by Devaraj K and fixed by Aaron T. Myers (name-node)<br>
+ <b>When the disk becomes full Namenode is getting shutdown and not able to recover</b><br>
+ <blockquote>Implemented a daemon thread to monitor the disk usage for periodically and if the disk usage reaches the threshold value, put the name node into Safe mode so that no modification to file system will occur. Once the disk usage reaches below the threshold, name node will be put out of the safe mode. Here threshold value and interval to check the disk usage are configurable.
+</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1592">HDFS-1592</a>.
+ Major bug reported by Bharath Mundlapudi and fixed by Bharath Mundlapudi <br>
+ <b>Datanode startup doesn't honor volumes.tolerated </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1588">HDFS-1588</a>.
+ Major improvement reported by Erik Steffl and fixed by Erik Steffl <br>
+ <b>Add dfs.hosts.exclude to DFSConfigKeys and use constant in stead of hardcoded string</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1585">HDFS-1585</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Todd Lipcon (test)<br>
+ <b>HDFS-1547 broke MR build</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1583">HDFS-1583</a>.
+ Major improvement reported by Liyin Liang and fixed by Liyin Liang (name-node)<br>
+ <b>Improve backup-node sync performance by wrapping RPC parameters</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1582">HDFS-1582</a>.
+ Major improvement reported by Roman Shaposhnik and fixed by Roman Shaposhnik (libhdfs)<br>
+ <b>Remove auto-generated native build files</b><br>
+ <blockquote>The native build run when from trunk now requires autotools, libtool and openssl dev libraries.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1573">HDFS-1573</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon (hdfs client)<br>
+ <b>LeaseChecker thread name trace not that useful</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1568">HDFS-1568</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Joey Echeverria (data-node)<br>
+ <b>Improve DataXceiver error logging</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1560">HDFS-1560</a>.
+ Minor bug reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>dfs.data.dir permissions should default to 700</b><br>
+ <blockquote>The permissions on datanode data directories (configured by dfs.datanode.data.dir.perm) now default to 0700. Upon startup, the datanode will automatically change the permissions to match the configured value.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1557">HDFS-1557</a>.
+ Major sub-task reported by Ivan Kelly and fixed by Ivan Kelly (name-node)<br>
+ <b>Separate Storage from FSImage</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1551">HDFS-1551</a>.
+ Major bug reported by Giridharan Kesavan and fixed by Giridharan Kesavan (build)<br>
+ <b>fix the pom template's version</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1547">HDFS-1547</a>.
+ Major improvement reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>Improve decommission mechanism</b><br>
+ <blockquote>Summary of changes to the decommissioning process:
+# After nodes are decommissioned, they are not shutdown. The decommissioned nodes are not used for writes. For reads, the decommissioned nodes are given as the last location to read from.
+# Number of live and dead decommissioned nodes are displayed in the namenode webUI.
+# Decommissioned nodes free capacity is not count towards the the cluster free capacity.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1541">HDFS-1541</a>.
+ Major sub-task reported by Hairong Kuang and fixed by Hairong Kuang (name-node)<br>
+ <b>Not marking datanodes dead When namenode in safemode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1540">HDFS-1540</a>.
+ Major bug reported by dhruba borthakur and fixed by dhruba borthakur (data-node)<br>
+ <b>Make Datanode handle errors to namenode.register call more elegantly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1539">HDFS-1539</a>.
+ Major improvement reported by dhruba borthakur and fixed by dhruba borthakur (data-node , hdfs client , name-node)<br>
+ <b>prevent data loss when a cluster suffers a power loss</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1536">HDFS-1536</a>.
+ Major improvement reported by Hairong Kuang and fixed by Hairong Kuang <br>
+ <b>Improve HDFS WebUI</b><br>
+ <blockquote>On web UI, missing block number now becomes accurate and under-replicated blocks do not include missing blocks.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1534">HDFS-1534</a>.
+ Minor improvement reported by Eli Collins and fixed by Eli Collins (name-node)<br>
+ <b>Fix some incorrect logs in FSDirectory</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1533">HDFS-1533</a>.
+ Major bug reported by Patrick Kling and fixed by Patrick Kling (hdfs client)<br>
+ <b>A more elegant FileSystem#listCorruptFileBlocks API (HDFS portion)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1526">HDFS-1526</a>.
+ Major bug reported by Hairong Kuang and fixed by Hairong Kuang (hdfs client)<br>
+ <b>Dfs client name for a map/reduce task should have some randomness</b><br>
+ <blockquote>Make a client name has this format: DFSClient_applicationid_randomint_threadid, where applicationid = mapred.task.id or else = "NONMAPREDUCE".</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1524">HDFS-1524</a>.
+ Blocker bug reported by Hairong Kuang and fixed by Hairong Kuang (name-node)<br>
+ <b>Image loader should make sure to read every byte in image file</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1523">HDFS-1523</a>.
+ Major bug reported by Konstantin Boudnik and fixed by Konstantin Boudnik (test)<br>
+ <b>TestLargeBlock is failing on trunk</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1518">HDFS-1518</a>.
+ Minor improvement reported by Jingguo Yao and fixed by Jingguo Yao (name-node)<br>
+ <b>Wrong description in FSNamesystem's javadoc</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1516">HDFS-1516</a>.
+ Major bug reported by Konstantin Boudnik and fixed by Konstantin Boudnik (build)<br>
+ <b>mvn-install is broken after 0.22 branch creation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1513">HDFS-1513</a>.
+ Minor improvement reported by Eli Collins and fixed by Eli Collins <br>
+ <b>Fix a number of warnings</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1511">HDFS-1511</a>.
+ Blocker bug reported by Nigel Daley and fixed by Jakob Homan <br>
+ <b>98 Release Audit warnings on trunk and branch-0.22</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1510">HDFS-1510</a>.
+ Minor improvement reported by Nigel Daley and fixed by Nigel Daley <br>
+ <b>Add test-patch.properties required by test-patch.sh</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1509">HDFS-1509</a>.
+ Major improvement reported by dhruba borthakur and fixed by dhruba borthakur (name-node)<br>
+ <b>Resync discarded directories in fs.name.dir during saveNamespace command</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1506">HDFS-1506</a>.
+ Major improvement reported by Hairong Kuang and fixed by Hairong Kuang (name-node)<br>
+ <b>Refactor fsimage loading code</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1505">HDFS-1505</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Aaron T. Myers <br>
+ <b>saveNamespace appears to succeed even if all directories fail to save</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1503">HDFS-1503</a>.
+ Minor bug reported by Eli Collins and fixed by Todd Lipcon (test)<br>
+ <b>TestSaveNamespace fails</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1502">HDFS-1502</a>.
+ Minor bug reported by Eli Collins and fixed by Hairong Kuang <br>
+ <b>TestBlockRecovery triggers NPE in assert</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1486">HDFS-1486</a>.
+ Major improvement reported by Konstantin Boudnik and fixed by Konstantin Boudnik (test)<br>
+ <b>Generalize CLITest structure and interfaces to facilitate upstream adoption (e.g. for web testing)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1481">HDFS-1481</a>.
+ Major improvement reported by Hairong Kuang and fixed by Hairong Kuang (name-node)<br>
+ <b>NameNode should validate fsimage before rolling</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1480">HDFS-1480</a>.
+ Major bug reported by T Meyarivan and fixed by Todd Lipcon (name-node)<br>
+ <b>All replicas of a block can end up on the same rack when some datanodes are decommissioning.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1476">HDFS-1476</a>.
+ Major improvement reported by Patrick Kling and fixed by Patrick Kling (name-node)<br>
+ <b>listCorruptFileBlocks should be functional while the name node is still in safe mode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1473">HDFS-1473</a>.
+ Major sub-task reported by Todd Lipcon and fixed by Todd Lipcon (name-node)<br>
+ <b>Refactor storage management into separate classes than fsimage file reading/writing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1467">HDFS-1467</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Todd Lipcon (data-node)<br>
+ <b>Append pipeline never succeeds with more than one replica</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1463">HDFS-1463</a>.
+ Major bug reported by dhruba borthakur and fixed by dhruba borthakur (name-node)<br>
+ <b>accessTime updates should not occur in safeMode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1458">HDFS-1458</a>.
+ Major improvement reported by Hairong Kuang and fixed by Hairong Kuang (name-node)<br>
+ <b>Improve checkpoint performance by avoiding unnecessary image downloads</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1448">HDFS-1448</a>.
+ Major new feature reported by Erik Steffl and fixed by Erik Steffl (tools)<br>
+ <b>Create multi-format parser for edits logs file, support binary and XML formats initially</b><br>
+ <blockquote>Offline edits viewer feature adds oev tool to hdfs script. Oev makes it possible to convert edits logs to/from native binary and XML formats. It uses the same framework as Offline image viewer.
+
+Example usage:
+
+$HADOOP_HOME/bin/hdfs oev -i edits -o output.xml</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1445">HDFS-1445</a>.
+ Major sub-task reported by Matt Foley and fixed by Matt Foley (data-node)<br>
+ <b>Batch the calls in DataStorage to FileUtil.createHardLink(), so we call it once per directory instead of once per file</b><br>
+ <blockquote>Batch hardlinking during "upgrade" snapshots, cutting time from aprx 8 minutes per volume to aprx 8 seconds. Validated in both Linux and Windows. Depends on prior integration with patch for HADOOP-7133.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1442">HDFS-1442</a>.
+ Major improvement reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Api to get delegation token in Hdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1381">HDFS-1381</a>.
+ Major bug reported by Jakob Homan and fixed by Jim Plush (test)<br>
+ <b>HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters</b><br>
+ <blockquote>Updated the JavaDocs to appropriately represent the new Configuration Keys that are used in the code. The docs did not match the code.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1378">HDFS-1378</a>.
+ Major improvement reported by Todd Lipcon and fixed by Colin Patrick McCabe (name-node)<br>
+ <b>Edit log replay should track and report file offsets in case of errors</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1377">HDFS-1377</a>.
+ Blocker bug reported by Eli Collins and fixed by Eli Collins (name-node)<br>
+ <b>Quota bug for partial blocks allows quotas to be violated </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1371">HDFS-1371</a>.
+ Major bug reported by Koji Noguchi and fixed by Tanping Wang (hdfs client , name-node)<br>
+ <b>One bad node can incorrectly flag many files as corrupt</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1360">HDFS-1360</a>.
+ Minor bug reported by Todd Lipcon and fixed by Todd Lipcon (test)<br>
+ <b>TestBlockRecovery should bind ephemeral ports</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1335">HDFS-1335</a>.
+ Major improvement reported by Hairong Kuang and fixed by Hairong Kuang (hdfs client , name-node)<br>
+ <b>HDFS side of HADOOP-6904: first step towards inter-version communications between dfs client and NameNode</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1332">HDFS-1332</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Ted Yu (name-node)<br>
+ <b>When unable to place replicas, BlockPlacementPolicy should log reasons nodes were excluded</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1330">HDFS-1330</a>.
+ Major new feature reported by Hairong Kuang and fixed by John George (data-node)<br>
+ <b>Make RPCs to DataNodes timeout</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1321">HDFS-1321</a>.
+ Minor bug reported by gary murry and fixed by Jim Plush (name-node)<br>
+ <b>If service port and main port are the same, there is no clear log message explaining the issue.</b><br>
+ <blockquote>Added a check to match the sure RPC and HTTP Port's on the NameNode were not set to the same value, otherwise an IOException is throw with the appropriate message.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1295">HDFS-1295</a>.
+ Major sub-task reported by dhruba borthakur and fixed by Matt Foley (name-node)<br>
+ <b>Improve namenode restart times by short-circuiting the first block reports from datanodes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1257">HDFS-1257</a>.
+ Major bug reported by Ramkumar Vadali and fixed by Eric Payne (name-node)<br>
+ <b>Race condition on FSNamesystem#recentInvalidateSets introduced by HADOOP-5124</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1206">HDFS-1206</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Konstantin Boudnik (test)<br>
+ <b>TestFiHFlush fails intermittently</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1189">HDFS-1189</a>.
+ Major bug reported by Kang Xiao and fixed by John George (name-node)<br>
+ <b>Quota counts missed between clear quota and set quota</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1149">HDFS-1149</a>.
+ Major bug reported by Todd Lipcon and fixed by Aaron T. Myers (name-node)<br>
+ <b>Lease reassignment is not persisted to edit log</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1120">HDFS-1120</a>.
+ Major improvement reported by Jeff Hammerbacher and fixed by Harsh J (data-node)<br>
+ <b>Make DataNode's block-to-device placement policy pluggable</b><br>
+ <blockquote>Make the DataNode's block-volume choosing policy pluggable.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1117">HDFS-1117</a>.
+ Major improvement reported by Luke Lu and fixed by Luke Lu <br>
+ <b>HDFS portion of HADOOP-6728 (ovehaul metrics framework)</b><br>
+ <blockquote>Metrics names are standardized to use CapitalizedCamelCase. Some examples:
+# Metrics names using "_" is changed to new naming scheme. Eg: bytes_written changes to BytesWritten.
+# All metrics names start with capitals. Example: threadsBlocked changes to ThreadsBlocked.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1073">HDFS-1073</a>.
+ Major improvement reported by Sanjay Radia and fixed by Todd Lipcon <br>
+ <b>Simpler model for Namenode's fs Image and edit Logs </b><br>
+ <blockquote>The NameNode's storage layout for its name directories has been reorganized to be more robust. Each edit now has a unique transaction ID, and each file is associated with a transaction ID (for checkpoints) or a range of transaction IDs (for edit logs).</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1070">HDFS-1070</a>.
+ Major sub-task reported by Hairong Kuang and fixed by Hairong Kuang (name-node)<br>
+ <b>Speedup NameNode image loading and saving by storing local file names</b><br>
+ <blockquote>This changes the fsimage format to be
+root directory-1 directory-2 ... directoy-n.
+Each directory stores all its children in the following format:
+Directory_full_path_name num_of_children child-1 ... child-n.
+Each inode stores only the last component of its path name into fsimage.
+This change requires an upgrade at deployment.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1052">HDFS-1052</a>.
+ Major new feature reported by Suresh Srinivas and fixed by Suresh Srinivas (name-node)<br>
+ <b>HDFS scalability with multiple namenodes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-1001">HDFS-1001</a>.
+ Minor bug reported by bc Wong and fixed by bc Wong (data-node)<br>
+ <b>DataXceiver and BlockReader disagree on when to send/recv CHECKSUM_OK</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-863">HDFS-863</a>.
+ Major bug reported by Todd Lipcon and fixed by Ken Goodhope (test)<br>
+ <b>Potential deadlock in TestOverReplicatedBlocks</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-780">HDFS-780</a>.
+ Major test reported by Eli Collins and fixed by Eli Collins (fuse-dfs)<br>
+ <b>Revive TestFuseDFS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-560">HDFS-560</a>.
+ Minor improvement reported by Steve Loughran and fixed by Steve Loughran (build)<br>
+ <b>Proposed enhancements/tuning to hadoop-hdfs/build.xml</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-420">HDFS-420</a>.
+ Major improvement reported by Dima Brodsky and fixed by Brian Bockelman (fuse-dfs)<br>
+ <b>Fuse-dfs should cache fs handles</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-73">HDFS-73</a>.
+ Blocker bug reported by Raghu Angadi and fixed by Uma Maheswara Rao G (hdfs client)<br>
+ <b>DFSOutputStream does not close all the sockets</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-8619">HADOOP-8619</a>.
+ Major improvement reported by Radim Kolar and fixed by Chris Douglas (io)<br>
+ <b>WritableComparator must implement no-arg constructor</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7798">HADOOP-7798</a>.
+ Blocker bug reported by Arun C Murthy and fixed by Doug Cutting (build)<br>
+ <b>Release artifacts need to be signed for Nexus</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7797">HADOOP-7797</a>.
+ Major bug reported by Owen O'Malley and fixed by Owen O'Malley (build)<br>
+ <b>Fix the repository name to support pushing to the staging area of Nexus</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7792">HADOOP-7792</a>.
+ Major improvement reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Common component for HDFS-2416: Add verifyToken method to AbstractDelegationTokenSecretManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7789">HADOOP-7789</a>.
+ Major improvement reported by Arun C Murthy and fixed by Arun C Murthy <br>
+ <b>Minor edits to top-level site</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7785">HADOOP-7785</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (io , util)<br>
+ <b>Add equals, hashcode, toString to DataChecksum</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7782">HADOOP-7782</a>.
+ Critical bug reported by Arun C Murthy and fixed by Tom White (build)<br>
+ <b>Aggregate project javadocs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7778">HADOOP-7778</a>.
+ Major bug reported by Tom White and fixed by Tom White <br>
+ <b>FindBugs warning in Token.getKind()</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7772">HADOOP-7772</a>.
+ Trivial improvement reported by Steve Loughran and fixed by Steve Loughran <br>
+ <b>javadoc the topology classes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7771">HADOOP-7771</a>.
+ Blocker bug reported by John George and fixed by John George <br>
+ <b>NPE when running hdfs dfs -copyToLocal, -get etc</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7770">HADOOP-7770</a>.
+ Blocker bug reported by Ravi Prakash and fixed by Ravi Prakash (viewfs)<br>
+ <b>ViewFS getFileChecksum throws FileNotFoundException for files in /tmp and /user</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7768">HADOOP-7768</a>.
+ Blocker bug reported by Jonathan Eagles and fixed by Tom White (build)<br>
+ <b>PreCommit-HADOOP-Build is failing on hadoop-auth-examples</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7766">HADOOP-7766</a>.
+ Major bug reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>The auth to local mappings are not being respected, with webhdfs and security enabled.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7764">HADOOP-7764</a>.
+ Blocker bug reported by Jonathan Eagles and fixed by Jonathan Eagles <br>
+ <b>Allow both ACL list and global path spec filters to HttpServer</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7763">HADOOP-7763</a>.
+ Major improvement reported by Tom White and fixed by Tom White (documentation)<br>
+ <b>Add top-level navigation to APT docs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7762">HADOOP-7762</a>.
+ Major task reported by Eli Collins and fixed by Eli Collins (scripts)<br>
+ <b>Common side of MR-2736 (MR1 removal)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7755">HADOOP-7755</a>.
+ Blocker bug reported by Jonathan Eagles and fixed by Jonathan Eagles (build)<br>
+ <b>Detect MapReduce PreCommit Trunk builds silently failing when running test-patch.sh</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7753">HADOOP-7753</a>.
+ Major sub-task reported by Todd Lipcon and fixed by Todd Lipcon (io , native , performance)<br>
+ <b>Support fadvise and sync_data_range in NativeIO, add ReadaheadPool class</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7749">HADOOP-7749</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (util)<br>
+ <b>Add NetUtils call which provides more help in exception messages</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7745">HADOOP-7745</a>.
+ Major bug reported by Ravi Prakash and fixed by Ravi Prakash <br>
+ <b>I switched variable names in HADOOP-7509</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7744">HADOOP-7744</a>.
+ Major bug reported by Jonathan Eagles and fixed by Jonathan Eagles (test)<br>
+ <b>Incorrect exit code for hadoop-core-test tests when exception thrown</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7743">HADOOP-7743</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Add Maven profile to create a full source tarball</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7740">HADOOP-7740</a>.
+ Minor bug reported by Arpit Gupta and fixed by Arpit Gupta (conf)<br>
+ <b>security audit logger is not on by default, fix the log4j properties to enable the logger</b><br>
+ <blockquote>Fixed security audit logger configuration. (Arpit Gupta via Eric Yang)</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7737">HADOOP-7737</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>normalize hadoop-mapreduce & hadoop-dist dist/tar build with common/hdfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7728">HADOOP-7728</a>.
+ Major bug reported by Ramya Sunil and fixed by Ramya Sunil (conf)<br>
+ <b>hadoop-setup-conf.sh should be modified to enable task memory manager</b><br>
+ <blockquote>Enable task memory management to be configurable via hadoop config setup script.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7724">HADOOP-7724</a>.
+ Major bug reported by Giridharan Kesavan and fixed by Arpit Gupta <br>
+ <b>hadoop-setup-conf.sh should put proxy user info into the core-site.xml </b><br>
+ <blockquote>Fixed hadoop-setup-conf.sh to put proxy user in core-site.xml. (Arpit Gupta via Eric Yang)</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7721">HADOOP-7721</a>.
+ Major bug reported by Arpit Gupta and fixed by Jitendra Nath Pandey <br>
+ <b>dfs.web.authentication.kerberos.principal expects the full hostname and does not replace _HOST with the hostname</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7720">HADOOP-7720</a>.
+ Major improvement reported by Arpit Gupta and fixed by Arpit Gupta (conf)<br>
+ <b>improve the hadoop-setup-conf.sh to read in the hbase user and setup the configs</b><br>
+ <blockquote>Added parameter for HBase user to setup config script. (Arpit Gupta via Eric Yang)</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7715">HADOOP-7715</a>.
+ Major bug reported by Arpit Gupta and fixed by Eric Yang (conf)<br>
+ <b>see log4j Error when running mr jobs and certain dfs calls</b><br>
+ <blockquote>Removed unnecessary security logger configuration. (Eric Yang)</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7711">HADOOP-7711</a>.
+ Major bug reported by Arpit Gupta and fixed by Arpit Gupta (conf)<br>
+ <b>hadoop-env.sh generated from templates has duplicate info</b><br>
+ <blockquote>Fixed recursive sourcing of HADOOP_OPTS environment variables (Arpit Gupta via Eric Yang)</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7710">HADOOP-7710</a>.
+ Major improvement reported by Arpit Gupta and fixed by Arpit Gupta <br>
+ <b>create a script to setup application in order to create root directories for application such hbase, hcat, hive etc</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7709">HADOOP-7709</a>.
+ Major improvement reported by Jonathan Eagles and fixed by Jonathan Eagles <br>
+ <b>Running a set of methods in a Single Test Class</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7708">HADOOP-7708</a>.
+ Critical bug reported by Arpit Gupta and fixed by Eric Yang (conf)<br>
+ <b>config generator does not update the properties file if on exists already</b><br>
+ <blockquote>Fixed hadoop-setup-conf.sh to handle config file consistently. (Eric Yang)</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7707">HADOOP-7707</a>.
+ Major improvement reported by Arpit Gupta and fixed by Arpit Gupta (conf)<br>
+ <b>improve config generator to allow users to specify proxy user, turn append on or off, turn webhdfs on or off</b><br>
+ <blockquote>Added toggle for dfs.support.append, webhdfs and hadoop proxy user to setup config script. (Arpit Gupta via Eric Yang)</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7705">HADOOP-7705</a>.
+ Minor new feature reported by Steve Loughran and fixed by Steve Loughran (util)<br>
+ <b>Add a log4j back end that can push out JSON data, one per line</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7691">HADOOP-7691</a>.
+ Major bug reported by Giridharan Kesavan and fixed by Eric Yang <br>
+ <b>hadoop deb pkg should take a diff group id</b><br>
+ <blockquote>Fixed conflict uid for install packages. (Eric Yang)</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7684">HADOOP-7684</a>.
+ Major bug reported by Eric Yang and fixed by Eric Yang (scripts)<br>
+ <b>jobhistory server and secondarynamenode should have init.d script</b><br>
+ <blockquote>Added init.d script for jobhistory server and secondary namenode. (Eric Yang)</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7681">HADOOP-7681</a>.
+ Minor bug reported by Arpit Gupta and fixed by Arpit Gupta (conf)<br>
+ <b>log4j.properties is missing properties for security audit and hdfs audit should be changed to info</b><br>
+ <blockquote>HADOOP-7681. Fixed security and hdfs audit log4j properties
+(Arpit Gupta via Eric Yang)</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7671">HADOOP-7671</a>.
+ Major bug reported by Ravi Prakash and fixed by Ravi Prakash <br>
+ <b>Add license headers to hadoop-common/src/main/packages/templates/conf/</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7668">HADOOP-7668</a>.
+ Minor improvement reported by Suresh Srinivas and fixed by Steve Loughran (util)<br>
+ <b>Add a NetUtils method that can tell if an InetAddress belongs to local host</b><br>
+ <blockquote>closing again</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7664">HADOOP-7664</a>.
+ Minor improvement reported by Ravi Prakash and fixed by Ravi Prakash (conf)<br>
+ <b>o.a.h.conf.Configuration complains of overriding final parameter even if the value with which its attempting to override is the same. </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7663">HADOOP-7663</a>.
+ Major bug reported by Mayank Bansal and fixed by Mayank Bansal (test)<br>
+ <b>TestHDFSTrash failing on 22</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7662">HADOOP-7662</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves <br>
+ <b>logs servlet should use pathspec of /*</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7658">HADOOP-7658</a>.
+ Major bug reported by Giridharan Kesavan and fixed by Eric Yang <br>
+ <b>to fix hadoop config template</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7655">HADOOP-7655</a>.
+ Major improvement reported by Arpit Gupta and fixed by Arpit Gupta <br>
+ <b>provide a small validation script that smoke tests the installed cluster</b><br>
+ <blockquote>Committed to trunk and v23, since code reviewed by Eric.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7642">HADOOP-7642</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Tom White (build)<br>
+ <b>create hadoop-dist module where TAR stitching would happen</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7639">HADOOP-7639</a>.
+ Major bug reported by Thomas Graves and fixed by Thomas Graves <br>
+ <b>yarn ui not properly filtered in HttpServer</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7637">HADOOP-7637</a>.
+ Major bug reported by Eric Yang and fixed by Eric Yang (build)<br>
+ <b>Fair scheduler configuration file is not bundled in RPM</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7633">HADOOP-7633</a>.
+ Major bug reported by Arpit Gupta and fixed by Eric Yang (conf)<br>
+ <b>log4j.properties should be added to the hadoop conf on deploy</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7631">HADOOP-7631</a>.
+ Major bug reported by Ramya Sunil and fixed by Eric Yang (conf)<br>
+ <b>In mapred-site.xml, stream.tmpdir is mapped to ${mapred.temp.dir} which is undeclared.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7630">HADOOP-7630</a>.
+ Major bug reported by Arpit Gupta and fixed by Eric Yang (conf)<br>
+ <b>hadoop-metrics2.properties should have a property *.period set to a default value foe metrics</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7629">HADOOP-7629</a>.
+ Major bug reported by Patrick Hunt and fixed by Todd Lipcon <br>
+ <b>regression with MAPREDUCE-2289 - setPermission passed immutable FsPermission (rpc failure)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7627">HADOOP-7627</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (metrics , test)<br>
+ <b>Improve MetricsAsserts to give more understandable output on failure</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7626">HADOOP-7626</a>.
+ Major bug reported by Eric Yang and fixed by Eric Yang (scripts)<br>
+ <b>Allow overwrite of HADOOP_CLASSPATH and HADOOP_OPTS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7624">HADOOP-7624</a>.
+ Major improvement reported by Vinod Kumar Vavilapalli and fixed by Alejandro Abdelnur (build)<br>
+ <b>Set things up for a top level hadoop-tools module</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7612">HADOOP-7612</a>.
+ Major improvement reported by Tom White and fixed by Tom White (build)<br>
+ <b>Change test-patch to run tests for all nested modules</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7610">HADOOP-7610</a>.
+ Major bug reported by Eric Yang and fixed by Eric Yang (scripts)<br>
+ <b>/etc/profile.d does not exist on Debian</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7608">HADOOP-7608</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (io)<br>
+ <b>SnappyCodec check for Hadoop native lib is wrong</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7606">HADOOP-7606</a>.
+ Major bug reported by Aaron T. Myers and fixed by Alejandro Abdelnur (test)<br>
+ <b>Upgrade Jackson to version 1.7.1 to match the version required by Jersey</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7604">HADOOP-7604</a>.
+ Critical bug reported by Mahadev konar and fixed by Mahadev konar <br>
+ <b>Hadoop Auth examples pom in 0.23 point to 0.24 versions.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7603">HADOOP-7603</a>.
+ Major bug reported by Eric Yang and fixed by Eric Yang <br>
+ <b>Set default hdfs, mapred uid, and hadoop group gid for RPM packages</b><br>
+ <blockquote>Set hdfs uid, mapred uid, and hadoop gid to fixed numbers (201, 202, and 123, respectively).</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7599">HADOOP-7599</a>.
+ Major bug reported by Eric Yang and fixed by Eric Yang (scripts)<br>
+ <b>Improve hadoop setup conf script to setup secure Hadoop cluster</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7598">HADOOP-7598</a>.
+ Major bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (build)<br>
+ <b>smart-apply-patch.sh does not handle patching from a sub directory correctly.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7595">HADOOP-7595</a>.
+ Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Upgrade dependency to Avro 1.5.3</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7594">HADOOP-7594</a>.
+ Major new feature reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE <br>
+ <b>Support HTTP REST in HttpServer</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7593">HADOOP-7593</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Uma Maheswara Rao G (test)<br>
+ <b>AssertionError in TestHttpServer.testMaxThreads()</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7589">HADOOP-7589</a>.
+ Major bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (build)<br>
+ <b>Prefer mvn test -DskipTests over mvn compile in test-patch.sh</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7580">HADOOP-7580</a>.
+ Major bug reported by Siddharth Seth and fixed by Siddharth Seth <br>
+ <b>Add a version of getLocalPathForWrite to LocalDirAllocator which doesn't create dirs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7579">HADOOP-7579</a>.
+ Major task reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)<br>
+ <b>Rename package names from alfredo to auth</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7578">HADOOP-7578</a>.
+ Major bug reported by Mahadev konar and fixed by Mahadev konar <br>
+ <b>Fix test-patch to be able to run on MR patches.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7576">HADOOP-7576</a>.
+ Major bug reported by Tom White and fixed by Tsz Wo (Nicholas), SZE (security)<br>
+ <b>Fix findbugs warnings in Hadoop Auth (Alfredo)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7575">HADOOP-7575</a>.
+ Minor bug reported by Jonathan Eagles and fixed by Jonathan Eagles (fs)<br>
+ <b>Support fully qualified paths as part of LocalDirAllocator</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7568">HADOOP-7568</a>.
+ Major bug reported by Konstantin Shvachko and fixed by Plamen Jeliazkov (io)<br>
+ <b>SequenceFile should not print into stdout</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7566">HADOOP-7566</a>.
+ Major bug reported by Mahadev konar and fixed by Alejandro Abdelnur <br>
+ <b>MR tests are failing webapps/hdfs not found in CLASSPATH</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7564">HADOOP-7564</a>.
+ Major sub-task reported by Tom White and fixed by Tom White <br>
+ <b>Remove test-patch SVN externals</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7561">HADOOP-7561</a>.
+ Major sub-task reported by Tom White and fixed by Tom White <br>
+ <b>Make test-patch only run tests for changed modules</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7560">HADOOP-7560</a>.
+ Major sub-task reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur <br>
+ <b>Make hadoop-common a POM module with sub-modules (common & alfredo)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7555">HADOOP-7555</a>.
+ Trivial improvement reported by Aaron T. Myers and fixed by Aaron T. Myers (build)<br>
+ <b>Add a eclipse-generated files to .gitignore</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7552">HADOOP-7552</a>.
+ Minor improvement reported by Eli Collins and fixed by Eli Collins (fs)<br>
+ <b>FileUtil#fullyDelete doesn't throw IOE but lists it in the throws clause</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7547">HADOOP-7547</a>.
+ Minor bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (io)<br>
+ <b>Fix the warning in writable classes.[ WritableComparable is a raw type. References to generic type WritableComparable<T> should be parameterized ]</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7545">HADOOP-7545</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (build , test)<br>
+ <b>common -tests jar should not include properties and configs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7536">HADOOP-7536</a>.
+ Major bug reported by Kihwal Lee and fixed by Alejandro Abdelnur (build)<br>
+ <b>Correct the dependency version regressions introduced in HADOOP-6671</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7533">HADOOP-7533</a>.
+ Major sub-task reported by Tom White and fixed by Tom White <br>
+ <b>Allow test-patch to be run from any subproject directory </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7531">HADOOP-7531</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins (util)<br>
+ <b>Add servlet util methods for handling paths in requests </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7529">HADOOP-7529</a>.
+ Critical bug reported by Todd Lipcon and fixed by Luke Lu (metrics)<br>
+ <b>Possible deadlock in metrics2</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7528">HADOOP-7528</a>.
+ Major sub-task reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Maven build fails in Windows</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7526">HADOOP-7526</a>.
+ Minor test reported by Eli Collins and fixed by Eli Collins (fs)<br>
+ <b>Add TestPath tests for URI conversion and reserved characters </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7525">HADOOP-7525</a>.
+ Major sub-task reported by Tom White and fixed by Tom White (scripts)<br>
+ <b>Make arguments to test-patch optional</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7523">HADOOP-7523</a>.
+ Blocker bug reported by John Lee and fixed by John Lee (test)<br>
+ <b>Test org.apache.hadoop.fs.TestFilterFileSystem fails due to java.lang.NoSuchMethodException</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7520">HADOOP-7520</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>hadoop-main fails to deploy</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7515">HADOOP-7515</a>.
+ Major sub-task reported by Tom White and fixed by Tom White (build)<br>
+ <b>test-patch reports the wrong number of javadoc warnings</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7512">HADOOP-7512</a>.
+ Trivial task reported by Harsh J and fixed by Harsh J (documentation)<br>
+ <b>Fix example mistake in WritableComparable javadocs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7509">HADOOP-7509</a>.
+ Trivial improvement reported by Ravi Prakash and fixed by Ravi Prakash <br>
+ <b>Improve message when Authentication is required</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7508">HADOOP-7508</a>.
+ Major sub-task reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>compiled nativelib is in wrong directory and it is not picked up by surefire setup</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7507">HADOOP-7507</a>.
+ Major bug reported by Jeff Bean and fixed by Alejandro Abdelnur (metrics)<br>
+ <b>jvm metrics all use the same namespace</b><br>
+ <blockquote>JVM metrics published to Ganglia now include the process name as part of the gmetric name.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7502">HADOOP-7502</a>.
+ Major sub-task reported by Luke Lu and fixed by Luke Lu <br>
+ <b>Use canonical (IDE friendly) generated-sources directory for generated sources</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7501">HADOOP-7501</a>.
+ Major sub-task reported by Alejandro Abdelnur and fixed by Tom White (build)<br>
+ <b>publish Hadoop Common artifacts (post HADOOP-6671) to Apache SNAPSHOTs repo</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7499">HADOOP-7499</a>.
+ Major bug reported by Jeffrey Naisbitt and fixed by Jeffrey Naisbitt (util)<br>
+ <b>Add method for doing a sanity check on hostnames in NetUtils</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7498">HADOOP-7498</a>.
+ Major sub-task reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>Remove legacy TAR layout creation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7496">HADOOP-7496</a>.
+ Major sub-task reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>break Maven TAR & bintar profiles into just LAYOUT & TAR proper</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7493">HADOOP-7493</a>.
+ Major new feature reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (io)<br>
+ <b>[HDFS-362] Provide ShortWritable class in hadoop.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7491">HADOOP-7491</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins (scripts)<br>
+ <b>hadoop command should respect HADOOP_OPTS when given a class name </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7474">HADOOP-7474</a>.
+ Major improvement reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Refactor ClientCache out of WritableRpcEngine.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7472">HADOOP-7472</a>.
+ Minor improvement reported by Kihwal Lee and fixed by Kihwal Lee (ipc)<br>
+ <b>RPC client should deal with the IP address changes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7471">HADOOP-7471</a>.
+ Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)<br>
+ <b>the saveVersion.sh script sometimes fails to extract SVN URL</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7469">HADOOP-7469</a>.
+ Minor sub-task reported by Steve Loughran and fixed by Steve Loughran (util)<br>
+ <b>add a standard handler for socket connection problems which improves diagnostics</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7465">HADOOP-7465</a>.
+ Trivial sub-task reported by XieXianshan and fixed by XieXianshan (fs , ipc)<br>
+ <b>A several tiny improvements for the LOG format</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7463">HADOOP-7463</a>.
+ Minor improvement reported by Mahadev konar and fixed by Mahadev konar <br>
+ <b>Adding a configuration parameter to SecurityInfo interface.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7460">HADOOP-7460</a>.
+ Major improvement reported by dhruba borthakur and fixed by Usman Masood (fs)<br>
+ <b>Support for pluggable Trash policies</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7457">HADOOP-7457</a>.
+ Blocker improvement reported by Jakob Homan and fixed by Jakob Homan (documentation)<br>
+ <b>Remove out-of-date Chinese language documentation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7451">HADOOP-7451</a>.
+ Major improvement reported by Matt Foley and fixed by Matt Foley <br>
+ <b>merge for MR-279: Generalize StringUtils#join</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7449">HADOOP-7449</a>.
+ Major improvement reported by Matt Foley and fixed by Matt Foley <br>
+ <b>merge for MR-279: add Data(In,Out)putByteBuffer to work with ByteBuffer similar to Data(In,Out)putBuffer for byte[]</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7448">HADOOP-7448</a>.
+ Major improvement reported by Matt Foley and fixed by Matt Foley <br>
+ <b>merge for MR-279: HttpServer /stacks servlet should use plain text content type</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7446">HADOOP-7446</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (native , performance)<br>
+ <b>Implement CRC32C native code using SSE4.2 instructions</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7445">HADOOP-7445</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (native , util)<br>
+ <b>Implement bulk checksum verification using efficient native code</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7444">HADOOP-7444</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Add Checksum API to verify and calculate checksums "in bulk"</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7443">HADOOP-7443</a>.
+ Major new feature reported by Todd Lipcon and fixed by Todd Lipcon (io , util)<br>
+ <b>Add CRC32C as another DataChecksum implementation</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7442">HADOOP-7442</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (conf , documentation)<br>
+ <b>Docs in core-default.xml still reference deprecated config "topology.script.file.name"</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7440">HADOOP-7440</a>.
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>HttpServer.getParameterValues throws NPE for missing parameters</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7438">HADOOP-7438</a>.
+ Major improvement reported by Ravi Prakash and fixed by Ravi Prakash <br>
+ <b>Using the hadoop-deamon.sh script to start nodes leads to a depricated warning </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7437">HADOOP-7437</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (io)<br>
+ <b>IOUtils.copybytes will suppress the stream closure exceptions. </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7434">HADOOP-7434</a>.
+ Minor improvement reported by 严金双 and fixed by 严金双 <br>
+ <b>Display error when using "daemonlog -setlevel" with illegal level</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7430">HADOOP-7430</a>.
+ Minor improvement reported by Ravi Prakash and fixed by Ravi Prakash (fs)<br>
+ <b>Improve error message when moving to trash fails due to quota issue</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7428">HADOOP-7428</a>.
+ Major bug reported by Todd Lipcon and fixed by Todd Lipcon (ipc)<br>
+ <b>IPC connection is orphaned with null 'out' member</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7419">HADOOP-7419</a>.
+ Major bug reported by Todd Lipcon and fixed by Bing Zheng <br>
+ <b>new hadoop-config.sh doesn't manage classpath for HADOOP_CONF_DIR correctly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7402">HADOOP-7402</a>.
+ Trivial bug reported by Aaron T. Myers and fixed by Aaron T. Myers (test)<br>
+ <b>TestConfiguration doesn't clean up after itself</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7392">HADOOP-7392</a>.
+ Major improvement reported by Tanping Wang and fixed by Tanping Wang <br>
+ <b>Implement capability of querying individual property of a mbean using JMXProxyServlet </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7389">HADOOP-7389</a>.
+ Major bug reported by Aaron T. Myers and fixed by Aaron T. Myers (test)<br>
+ <b>Use of TestingGroups by tests causes subsequent tests to fail</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7385">HADOOP-7385</a>.
+ Minor bug reported by Bharath Mundlapudi and fixed by Bharath Mundlapudi <br>
+ <b>Remove StringUtils.stringifyException(ie) in logger functions</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7384">HADOOP-7384</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Allow test-patch to be more flexible about patch format</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7383">HADOOP-7383</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Todd Lipcon (build)<br>
+ <b>HDFS needs to export protobuf library dependency in pom</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7380">HADOOP-7380</a>.
+ Major sub-task reported by Aaron T. Myers and fixed by Aaron T. Myers (ha , ipc)<br>
+ <b>Add client failover functionality to o.a.h.io.(ipc|retry)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7379">HADOOP-7379</a>.
+ Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (io , ipc)<br>
+ <b>Add ability to include Protobufs in ObjectWritable</b><br>
+ <blockquote>Protocol buffer-generated types may now be used as arguments or return values for Hadoop RPC.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7377">HADOOP-7377</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Fix command name handling affecting DFSAdmin</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7375">HADOOP-7375</a>.
+ Major improvement reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>Add resolvePath method to FileContext</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7374">HADOOP-7374</a>.
+ Major improvement reported by Eli Collins and fixed by Eli Collins (scripts)<br>
+ <b>Don't add tools.jar to the classpath when running Hadoop</b><br>
+ <blockquote>The scripts that run Hadoop no longer automatically add tools.jar from the JDK to the classpath (if it is present). If your job depends on tools.jar in the JDK you will need to add this dependency in your job.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7361">HADOOP-7361</a>.
+ Minor improvement reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (fs)<br>
+ <b>Provide overwrite option (-overwrite/-f) in put and copyFromLocal command line options</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7360">HADOOP-7360</a>.
+ Major improvement reported by Daryn Sharp and fixed by Kihwal Lee (fs)<br>
+ <b>FsShell does not preserve relative paths with globs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7357">HADOOP-7357</a>.
+ Trivial bug reported by Philip Zeyliger and fixed by Philip Zeyliger (test)<br>
+ <b>hadoop.io.compress.TestCodec#main() should exit with non-zero exit code if test failed</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7356">HADOOP-7356</a>.
+ Blocker bug reported by Eric Yang and fixed by Eric Yang <br>
+ <b>RPM packages broke bin/hadoop script for hadoop 0.20.205</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7353">HADOOP-7353</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Cleanup FsShell and prevent masking of RTE stacktraces</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7342">HADOOP-7342</a>.
+ Minor bug reported by Bharath Mundlapudi and fixed by Bharath Mundlapudi <br>
+ <b>Add an utility API in FileUtil for JDK File.list</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7341">HADOOP-7341</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Fix option parsing in CommandFormat</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7337">HADOOP-7337</a>.
+ Minor improvement reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (util)<br>
+ <b>Annotate PureJavaCrc32 as a public API</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7336">HADOOP-7336</a>.
+ Minor bug reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>TestFileContextResolveAfs will fail with default test.build.data property.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7333">HADOOP-7333</a>.
+ Minor improvement reported by Eric Caspole and fixed by Eric Caspole (performance , util)<br>
+ <b>Performance improvement in PureJavaCrc32</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7331">HADOOP-7331</a>.
+ Trivial improvement reported by Tanping Wang and fixed by Tanping Wang (scripts)<br>
+ <b>Make hadoop-daemon.sh to return 1 if daemon processes did not get started</b><br>
+ <blockquote>hadoop-daemon.sh now returns a non-zero exit code if it detects that the daemon was not still running after 3 seconds.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7329">HADOOP-7329</a>.
+ Minor improvement reported by XieXianshan and fixed by XieXianshan (fs)<br>
+ <b>incomplete help message is displayed for df -h option</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7328">HADOOP-7328</a>.
+ Major improvement reported by Harsh J and fixed by Harsh J (io)<br>
+ <b>When a serializer class is missing, return null, not throw an NPE.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7327">HADOOP-7327</a>.
+ Minor bug reported by Matt Foley and fixed by Matt Foley (fs)<br>
+ <b>FileSystem.listStatus() throws NullPointerException instead of IOException upon access permission failure</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7324">HADOOP-7324</a>.
+ Blocker bug reported by Luke Lu and fixed by Priyo Mustafi (metrics)<br>
+ <b>Ganglia plugins for metrics v2</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7322">HADOOP-7322</a>.
+ Minor bug reported by Bharath Mundlapudi and fixed by Bharath Mundlapudi <br>
+ <b>Adding a util method in FileUtil for JDK File.listFiles</b><br>
+ <blockquote>Use of this new utility method avoids null result from File.listFiles(), and consequent NPEs.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7320">HADOOP-7320</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp <br>
+ <b>Refactor FsShell's copy & move commands</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7316">HADOOP-7316</a>.
+ Major improvement reported by Jonathan Hsieh and fixed by Eli Collins (documentation)<br>
+ <b>Add public javadocs to FSDataInputStream and FSDataOutputStream</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7314">HADOOP-7314</a>.
+ Major improvement reported by Jeffrey Naisbitt and fixed by Jeffrey Naisbitt <br>
+ <b>Add support for throwing UnknownHostException when a host doesn't resolve</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7306">HADOOP-7306</a>.
+ Major improvement reported by Luke Lu and fixed by Luke Lu (metrics)<br>
+ <b>Start metrics system even if config files are missing</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7305">HADOOP-7305</a>.
+ Minor improvement reported by Niels Basjes and fixed by Niels Basjes (build)<br>
+ <b>Eclipse project files are incomplete</b><br>
+ <blockquote>Added missing library during creation of the eclipse project files.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7301">HADOOP-7301</a>.
+ Major improvement reported by Jonathan Hsieh and fixed by Jonathan Hsieh <br>
+ <b>FSDataInputStream should expose a getWrappedStream method</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7298">HADOOP-7298</a>.
+ Major test reported by Todd Lipcon and fixed by Todd Lipcon (test)<br>
+ <b>Add test utility for writing multi-threaded tests</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7292">HADOOP-7292</a>.
+ Minor bug reported by Luke Lu and fixed by Luke Lu (metrics)<br>
+ <b>Metrics 2 TestSinkQueue is racy</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7289">HADOOP-7289</a>.
+ Major improvement reported by Tsz Wo (Nicholas), SZE and fixed by Eric Yang (build)<br>
+ <b>ivy: test conf should not extend common conf</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7287">HADOOP-7287</a>.
+ Blocker bug reported by Todd Lipcon and fixed by Aaron T. Myers (conf)<br>
+ <b>Configuration deprecation mechanism doesn't work properly for GenericOptionsParser/Tools</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7286">HADOOP-7286</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Refactor FsShell's du/dus/df</b><br>
+ <blockquote>The "Found X items" header on the output of the "du" command has been removed to more closely match unix. The displayed paths now correspond to the command line arguments instead of always being a fully qualified URI. For example, the output will have relative paths if the command line arguments are relative paths.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7285">HADOOP-7285</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Refactor FsShell's test</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7284">HADOOP-7284</a>.
+ Major bug reported by Sanjay Radia and fixed by Sanjay Radia (viewfs)<br>
+ <b>Trash and shell's rm does not work for viewfs</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7282">HADOOP-7282</a>.
+ Major bug reported by John George and fixed by John George (ipc)<br>
+ <b>getRemoteIp could return null in cases where the call is ongoing but the ip went away.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7276">HADOOP-7276</a>.
+ Major bug reported by Trevor Robinson and fixed by Trevor Robinson (native)<br>
+ <b>Hadoop native builds fail on ARM due to -m32</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7275">HADOOP-7275</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Refactor FsShell's stat</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7271">HADOOP-7271</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Standardize error messages</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7268">HADOOP-7268</a>.
+ Major bug reported by Devaraj Das and fixed by Jitendra Nath Pandey (fs , security)<br>
+ <b>FileContext.getLocalFSFileContext() behavior needs to be fixed w.r.t tokens</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7267">HADOOP-7267</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Refactor FsShell's rm/rmr/expunge</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7265">HADOOP-7265</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Keep track of relative paths</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7264">HADOOP-7264</a>.
+ Major improvement reported by Luke Lu and fixed by Luke Lu (io)<br>
+ <b>Bump avro version to at least 1.4.1</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7261">HADOOP-7261</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas (test)<br>
+ <b>Disable IPV6 for junit tests</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7259">HADOOP-7259</a>.
+ Major bug reported by Owen O'Malley and fixed by Owen O'Malley (build)<br>
+ <b>contrib modules should include build.properties from parent.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7257">HADOOP-7257</a>.
+ Major new feature reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>A client side mount table to give per-application/per-job file system view</b><br>
+ <blockquote>viewfs - client-side mount table.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7251">HADOOP-7251</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Refactor FsShell's getmerge</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7250">HADOOP-7250</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Refactor FsShell's setrep</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7249">HADOOP-7249</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Refactor FsShell's chmod/chown/chgrp</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7241">HADOOP-7241</a>.
+ Minor improvement reported by Wei Yongjun and fixed by Wei Yongjun (fs , test)<br>
+ <b>fix typo of command 'hadoop fs -help tail'</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7238">HADOOP-7238</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Refactor FsShell's cat & text</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7237">HADOOP-7237</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Refactor FsShell's touchz</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7236">HADOOP-7236</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Refactor FsShell's mkdir</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7235">HADOOP-7235</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp <br>
+ <b>Refactor FsShell's tail</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7233">HADOOP-7233</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Refactor FsShell's ls</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7231">HADOOP-7231</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (util)<br>
+ <b>Fix synopsis for -count</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7230">HADOOP-7230</a>.
+ Major test reported by Daryn Sharp and fixed by Daryn Sharp (test)<br>
+ <b>Move -fs usage tests from hdfs into common</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7227">HADOOP-7227</a>.
+ Major improvement reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey (ipc)<br>
+ <b>Remove protocol version check at proxy creation in Hadoop RPC.</b><br>
+ <blockquote>1. Protocol version check is removed from proxy creation, instead version check is performed at server in every rpc call.
+2. This change is backward incompatible because format of the rpc messages is changed to include client version, client method hash and rpc version.
+3. rpc version is introduced which should change when the format of rpc messages is changed.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7223">HADOOP-7223</a>.
+ Major bug reported by Suresh Srinivas and fixed by Suresh Srinivas (fs)<br>
+ <b>FileContext createFlag combinations during create are not clearly defined</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7216">HADOOP-7216</a>.
+ Major bug reported by Aaron T. Myers and fixed by Daryn Sharp (test)<br>
+ <b>HADOOP-7202 broke TestDFSShell in HDFS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7215">HADOOP-7215</a>.
+ Blocker bug reported by Suresh Srinivas and fixed by Suresh Srinivas (security)<br>
+ <b>RPC clients must connect over a network interface corresponding to the host name in the client's kerberos principal key</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7214">HADOOP-7214</a>.
+ Major new feature reported by Aaron T. Myers and fixed by Aaron T. Myers <br>
+ <b>Hadoop /usr/bin/groups equivalent</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7210">HADOOP-7210</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (fs)<br>
+ <b>Chown command is not working from FSShell.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7209">HADOOP-7209</a>.
+ Major improvement reported by Olga Natkovich and fixed by Daryn Sharp <br>
+ <b>Extensions to FsShell</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7208">HADOOP-7208</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G <br>
+ <b>equals() and hashCode() implementation need to change in StandardSocketFactory</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7206">HADOOP-7206</a>.
+ Major new feature reported by Eli Collins and fixed by Alejandro Abdelnur <br>
+ <b>Integrate Snappy compression</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7205">HADOOP-7205</a>.
+ Trivial improvement reported by Daryn Sharp and fixed by Daryn Sharp <br>
+ <b>automatically determine JAVA_HOME on OS X</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7202">HADOOP-7202</a>.
+ Major improvement reported by Daryn Sharp and fixed by Daryn Sharp <br>
+ <b>Improve Command base class</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7194">HADOOP-7194</a>.
+ Major bug reported by Devaraj K and fixed by Devaraj K (io)<br>
+ <b>Potential Resource leak in IOUtils.java</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7193">HADOOP-7193</a>.
+ Minor improvement reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (fs)<br>
+ <b>Help message is wrong for touchz command.</b><br>
+ <blockquote>Updated the help for the touchz command.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7187">HADOOP-7187</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (metrics)<br>
+ <b>Socket Leak in org.apache.hadoop.metrics.ganglia.GangliaContext</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7180">HADOOP-7180</a>.
+ Minor improvement reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Improve CommandFormat</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7178">HADOOP-7178</a>.
+ Major bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (fs)<br>
+ <b>FileSystem should have an option to control the .crc file creations at Local.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7177">HADOOP-7177</a>.
+ Trivial improvement reported by Allen Wittenauer and fixed by Allen Wittenauer (native)<br>
+ <b>CodecPool should report which compressor it is using</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7176">HADOOP-7176</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp <br>
+ <b>Redesign FsShell</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7175">HADOOP-7175</a>.
+ Major bug reported by Daryn Sharp and fixed by Daryn Sharp (fs)<br>
+ <b>Add isEnabled() to Trash</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7174">HADOOP-7174</a>.
+ Minor bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (fs)<br>
+ <b>null is displayed in the console,if the src path is invalid while doing copyToLocal operation from commandLine</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7172">HADOOP-7172</a>.
+ Critical bug reported by Todd Lipcon and fixed by Todd Lipcon (io , security)<br>
+ <b>SecureIO should not check owner on non-secure clusters that have no native support</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7171">HADOOP-7171</a>.
+ Major bug reported by Owen O'Malley and fixed by Jitendra Nath Pandey (security)<br>
+ <b>Support UGI in FileContext API</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7167">HADOOP-7167</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Allow using a file to exclude certain tests from build</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7162">HADOOP-7162</a>.
+ Minor bug reported by Alexey Diomin and fixed by Alexey Diomin (fs)<br>
+ <b>FsShell: call srcFs.listStatus(src) twice</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7159">HADOOP-7159</a>.
+ Trivial improvement reported by Scott Chen and fixed by Scott Chen (ipc)<br>
+ <b>RPC server should log the client hostname when read exception happened</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7153">HADOOP-7153</a>.
+ Minor improvement reported by Nicholas Telford and fixed by Nicholas Telford (io)<br>
+ <b>MapWritable violates contract of Map interface for equals() and hashCode()</b><br>
+ <blockquote>MapWritable now implements equals() and hashCode() based on the map contents rather than object identity in order to correctly implement the Map interface.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7151">HADOOP-7151</a>.
+ Minor bug reported by Dmitriy V. Ryaboy and fixed by Dmitriy V. Ryaboy <br>
+ <b>Document need for stable hashCode() in WritableComparable</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7144">HADOOP-7144</a>.
+ Major new feature reported by Luke Lu and fixed by Robert Joseph Evans <br>
+ <b>Expose JMX with something like JMXProxyServlet </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7136">HADOOP-7136</a>.
+ Major task reported by Nigel Daley and fixed by Nigel Daley <br>
+ <b>Remove failmon contrib</b><br>
+ <blockquote>Failmon removed from contrib codebase.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7133">HADOOP-7133</a>.
+ Major improvement reported by Matt Foley and fixed by Matt Foley (util)<br>
+ <b>CLONE to COMMON - HDFS-1445 Batch the calls in DataStorage to FileUtil.createHardLink(), so we call it once per directory instead of once per file</b><br>
+ <blockquote>This is the COMMON portion of a fix requiring coordinated change of COMMON and HDFS. Please see HDFS-1445 for HDFS portion and release note.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7131">HADOOP-7131</a>.
+ Minor improvement reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (io)<br>
+ <b>set() and toString Methods of the org.apache.hadoop.io.Text class does not include the root exception, in the wrapping RuntimeException.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7120">HADOOP-7120</a>.
+ Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Tsz Wo (Nicholas), SZE (test)<br>
+ <b>200 new Findbugs warnings</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7119">HADOOP-7119</a>.
+ Major new feature reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)<br>
+ <b>add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles</b><br>
+ <blockquote>Adding support for Kerberos HTTP SPNEGO authentication to the Hadoop web-consoles</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7117">HADOOP-7117</a>.
+ Major improvement reported by Patrick Angeles and fixed by Harsh J (conf)<br>
+ <b>Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml</b><br>
+ <blockquote>Removed references to the older fs.checkpoint.* properties that resided in core-site.xml</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7114">HADOOP-7114</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (fs)<br>
+ <b>FsShell should dump all exceptions at DEBUG level</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7112">HADOOP-7112</a>.
+ Major improvement reported by Tom White and fixed by Tom White (conf , filecache)<br>
+ <b>Issue a warning when GenericOptionsParser libjars are not on local filesystem</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7111">HADOOP-7111</a>.
+ Critical bug reported by Todd Lipcon and fixed by Aaron T. Myers (io)<br>
+ <b>Several TFile tests failing when native libraries are present</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7098">HADOOP-7098</a>.
+ Major bug reported by Bernd Fondermann and fixed by Bernd Fondermann (conf)<br>
+ <b>tasktracker property not set in conf/hadoop-env.sh</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7096">HADOOP-7096</a>.
+ Major improvement reported by Ahmed Radwan and fixed by Ahmed Radwan <br>
+ <b>Allow setting of end-of-record delimiter for TextInputFormat</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7090">HADOOP-7090</a>.
+ Major bug reported by Gokul and fixed by Uma Maheswara Rao G (fs/s3 , io)<br>
+ <b>Possible resource leaks in hadoop core code</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7089">HADOOP-7089</a>.
+ Minor bug reported by Eli Collins and fixed by Eli Collins (scripts)<br>
+ <b>Fix link resolution logic in hadoop-config.sh</b><br>
+ <blockquote>Updates hadoop-config.sh to always resolve symlinks when determining HADOOP_HOME. Bash built-ins or POSIX:2001 compliant cmds are now required.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7078">HADOOP-7078</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Harsh J <br>
+ <b>Add better javadocs for RawComparator interface</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7071">HADOOP-7071</a>.
+ Minor bug reported by Nigel Daley and fixed by Nigel Daley (build)<br>
+ <b>test-patch.sh has bad ps arg</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7061">HADOOP-7061</a>.
+ Minor improvement reported by Jingguo Yao and fixed by Jingguo Yao (io)<br>
+ <b>unprecise javadoc for CompressionCodec</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7060">HADOOP-7060</a>.
+ Major improvement reported by Hairong Kuang and fixed by Patrick Kling (fs)<br>
+ <b>A more elegant FileSystem#listCorruptFileBlocks API</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7059">HADOOP-7059</a>.
+ Major improvement reported by Noah Watkins and fixed by Noah Watkins (native)<br>
+ <b>Remove "unused" warning in native code</b><br>
+ <blockquote>Adds __attribute__ ((unused))</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7058">HADOOP-7058</a>.
+ Trivial improvement reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Expose number of bytes in FSOutputSummer buffer to implementatins</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7057">HADOOP-7057</a>.
+ Minor bug reported by Konstantin Boudnik and fixed by Konstantin Boudnik (util)<br>
+ <b>IOUtils.readFully and IOUtils.skipFully have typo in exception creation's message</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7055">HADOOP-7055</a>.
+ Major bug reported by Jingguo Yao and fixed by Jingguo Yao (metrics)<br>
+ <b>Update of commons logging libraries causes EventCounter to count logging events incorrectly</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7053">HADOOP-7053</a>.
+ Minor bug reported by Jingguo Yao and fixed by Jingguo Yao (conf)<br>
+ <b>wrong FSNamesystem Audit logging setting in conf/log4j.properties</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7052">HADOOP-7052</a>.
+ Major bug reported by Jingguo Yao and fixed by Jingguo Yao (conf)<br>
+ <b>misspelling of threshold in conf/log4j.properties</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7049">HADOOP-7049</a>.
+ Trivial improvement reported by Patrick Kling and fixed by Patrick Kling (conf)<br>
+ <b>TestReconfiguration should be junit v4</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7048">HADOOP-7048</a>.
+ Minor improvement reported by Jingguo Yao and fixed by Jingguo Yao (io)<br>
+ <b>Wrong description of Block-Compressed SequenceFile Format in SequenceFile's javadoc</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7046">HADOOP-7046</a>.
+ Blocker bug reported by Nigel Daley and fixed by Po Cheung (security)<br>
+ <b>1 Findbugs warning on trunk and branch-0.22</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7045">HADOOP-7045</a>.
+ Minor bug reported by Eli Collins and fixed by Eli Collins (fs)<br>
+ <b>TestDU fails on systems with local file systems with extended attributes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7042">HADOOP-7042</a>.
+ Minor improvement reported by Nigel Daley and fixed by Nigel Daley (test)<br>
+ <b>Update test-patch.sh to include failed test names and move test-patch.properties</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7023">HADOOP-7023</a>.
+ Major improvement reported by Patrick Kling and fixed by Patrick Kling <br>
+ <b>Add listCorruptFileBlocks to FileSystem</b><br>
+ <blockquote>Add a new API listCorruptFileBlocks to FIleContext that returns a list of files that have corrupt blocks. </blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7015">HADOOP-7015</a>.
+ Minor bug reported by Sanjay Radia and fixed by Sanjay Radia <br>
+ <b>RawLocalFileSystem#listStatus does not deal with a directory whose entries are changing ( e.g. in a multi-thread or multi-process environment)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7014">HADOOP-7014</a>.
+ Major improvement reported by Konstantin Boudnik and fixed by Konstantin Boudnik (test)<br>
+ <b>Generalize CLITest structure and interfaces to facilitate upstream adoption (e.g. for web testing)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-7001">HADOOP-7001</a>.
+ Major task reported by Patrick Kling and fixed by Patrick Kling (conf)<br>
+ <b>Allow configuration changes without restarting configured nodes</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6994">HADOOP-6994</a>.
+ Major improvement reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Api to get delegation token in AbstractFileSystem</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6949">HADOOP-6949</a>.
+ Major improvement reported by Navis and fixed by Matt Foley (io)<br>
+ <b>Reduces RPC packet size for primitive arrays, especially long[], which is used at block reporting</b><br>
+ <blockquote>Increments the RPC protocol version in org.apache.hadoop.ipc.Server from 4 to 5.
+Introduces ArrayPrimitiveWritable for a much more efficient wire format to transmit arrays of primitives over RPC. ObjectWritable uses the new writable for array of primitives for RPC and continues to use existing format for on-disk data.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6939">HADOOP-6939</a>.
+ Minor bug reported by Todd Lipcon and fixed by Todd Lipcon <br>
+ <b>Inconsistent lock ordering in AbstractDelegationTokenSecretManager</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6929">HADOOP-6929</a>.
+ Major improvement reported by Sharad Agarwal and fixed by Sharad Agarwal (ipc , security)<br>
+ <b>RPC should have a way to pass Security information other than protocol annotations</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6921">HADOOP-6921</a>.
+ Major sub-task reported by Luke Lu and fixed by Luke Lu <br>
+ <b>metrics2: metrics plugins</b><br>
+ <blockquote>Metrics names are standardized to CapitalizedCamelCase. See release note of HADOOP-6918 and HADOOP-6920.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6920">HADOOP-6920</a>.
+ Major sub-task reported by Luke Lu and fixed by Luke Lu <br>
+ <b>Metrics2: metrics instrumentation</b><br>
+ <blockquote>Metrics names are standardized to use CapitalizedCamelCase. Some examples of this is:
+# Metrics names using "_" is changed to new naming scheme. Eg: bytes_written changes to BytesWritten.
+# All metrics names start with capitals. Example: threadsBlocked changes to ThreadsBlocked.
+</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6919">HADOOP-6919</a>.
+ Major sub-task reported by Luke Lu and fixed by Luke Lu (metrics)<br>
+ <b>Metrics2: metrics framework</b><br>
+ <blockquote>New metrics2 framework for Hadoop.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6912">HADOOP-6912</a>.
+ Major bug reported by Kan Zhang and fixed by Kan Zhang (security)<br>
+ <b>Guard against NPE when calling UGI.isLoginKeytabBased()</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6904">HADOOP-6904</a>.
+ Major new feature reported by Hairong Kuang and fixed by Hairong Kuang (ipc)<br>
+ <b>A baby step towards inter-version RPC communications</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6889">HADOOP-6889</a>.
+ Major new feature reported by Hairong Kuang and fixed by John George (ipc)<br>
+ <b>Make RPC to have an option to timeout</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6887">HADOOP-6887</a>.
+ Major improvement reported by Bharath Mundlapudi and fixed by Luke Lu (metrics)<br>
+ <b>Need a separate metrics per garbage collector</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6864">HADOOP-6864</a>.
+ Major improvement reported by Erik Steffl and fixed by Boris Shkolnik (security)<br>
+ <b>Provide a JNI-based implementation of ShellBasedUnixGroupsNetgroupMapping (implementation of GroupMappingServiceProvider)</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6764">HADOOP-6764</a>.
+ Major improvement reported by Dmytro Molkov and fixed by Dmytro Molkov (ipc)<br>
+ <b>Add number of reader threads and queue length as configuration parameters in RPC.getServer</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6754">HADOOP-6754</a>.
+ Major bug reported by Aaron Kimball and fixed by Aaron Kimball (io)<br>
+ <b>DefaultCodec.createOutputStream() leaks memory</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6683">HADOOP-6683</a>.
+ Minor sub-task reported by Kang Xiao and fixed by Kang Xiao (io)<br>
+ <b>the first optimization: ZlibCompressor does not fully utilize the buffer</b><br>
+ <blockquote>Improve the buffer utilization of ZlibCompressor to avoid invoking a JNI per write request.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6671">HADOOP-6671</a>.
+ Major sub-task reported by Giridharan Kesavan and fixed by Alejandro Abdelnur (build)<br>
+ <b>To use maven for hadoop common builds</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6622">HADOOP-6622</a>.
+ Major bug reported by Jitendra Nath Pandey and fixed by Eli Collins (security)<br>
+ <b>Token should not print the password in toString.</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6578">HADOOP-6578</a>.
+ Minor improvement reported by Todd Lipcon and fixed by Michele Catasta (conf)<br>
+ <b>Configuration should trim whitespace around a lot of value types</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6508">HADOOP-6508</a>.
+ Major bug reported by Amareshwari Sriramadasu and fixed by Luke Lu (metrics)<br>
+ <b>Incorrect values for metrics with CompositeContext</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6436">HADOOP-6436</a>.
+ Major improvement reported by Eli Collins and fixed by Roman Shaposhnik <br>
+ <b>Remove auto-generated native build files </b><br>
+ <blockquote>The native build run when from trunk now requires autotools, libtool and openssl dev libraries.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6432">HADOOP-6432</a>.
+ Major new feature reported by Jitendra Nath Pandey and fixed by Jitendra Nath Pandey <br>
+ <b>Statistics support in FileContext</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6385">HADOOP-6385</a>.
+ Minor new feature reported by Scott Phillips and fixed by Daryn Sharp (fs)<br>
+ <b>dfs does not support -rmdir (was HDFS-639)</b><br>
+ <blockquote>The "rm" family of FsShell commands now supports -rmdir and -f options.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6376">HADOOP-6376</a>.
+ Minor improvement reported by Karthik K and fixed by Karthik K (conf)<br>
+ <b>slaves file to have a header specifying the format of conf/slaves file </b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6255">HADOOP-6255</a>.
+ Major new feature reported by Owen O'Malley and fixed by Eric Yang <br>
+ <b>Create an rpm integration project</b><br>
+ <blockquote>Added RPM/DEB packages to build system.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-6158">HADOOP-6158</a>.
+ Minor task reported by Owen O'Malley and fixed by Eli Collins (util)<br>
+ <b>Move CyclicIteration to HDFS</b><br>
+ <blockquote></blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-5647">HADOOP-5647</a>.
+ Major bug reported by Ravi Gummadi and fixed by Ravi Gummadi (test)<br>
+ <b>TestJobHistory fails if /tmp/_logs is not writable to. Testcase should not depend on /tmp</b><br>
+ <blockquote>Removed dependency of testcase on /tmp and made it to use test.build.data directory instead.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-2081">HADOOP-2081</a>.
+ Major bug reported by Owen O'Malley and fixed by Harsh J (conf)<br>
+ <b>Configuration getInt, getLong, and getFloat replace invalid numbers with the default value</b><br>
+ <blockquote>Invalid configuration values now result in a number format exception rather than the default value being used.</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HADOOP-1886">HADOOP-1886</a>.
+ Trivial improvement reported by Konstantin Shvachko and fixed by Frank Conrad (fs)<br>
+ <b>Undocumented parameters in FilesSystem</b><br>
+ <blockquote></blockquote></li>
+</ul>
+</body></html>
diff --git a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java
index 05e6297..7cbd1fb 100644
--- a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java
+++ b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java
@@ -69,8 +69,9 @@
public TrashPolicyDefault() { }
- private TrashPolicyDefault(Path home, Configuration conf) throws IOException {
- initialize(conf, home.getFileSystem(conf), home);
+ private TrashPolicyDefault(FileSystem fs, Path home, Configuration conf)
+ throws IOException {
+ initialize(conf, fs, home);
}
@Override
@@ -278,7 +279,8 @@
if (!home.isDirectory())
continue;
try {
- TrashPolicyDefault trash = new TrashPolicyDefault(home.getPath(), conf);
+ TrashPolicyDefault trash = new TrashPolicyDefault(
+ fs, home.getPath(), conf);
trash.deleteCheckpoint();
trash.createCheckpoint();
} catch (IOException e) {
diff --git a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java
index 25bae83..4c09746 100644
--- a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java
+++ b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java
@@ -25,6 +25,7 @@
import java.net.UnknownHostException;
import java.security.AccessController;
import java.security.PrivilegedAction;
+import java.security.PrivilegedExceptionAction;
import java.util.Arrays;
import java.util.List;
import java.util.ServiceLoader;
@@ -451,6 +452,41 @@
return action.run();
}
}
+
+ /**
+ * Perform the given action as the daemon's login user. If an
+ * InterruptedException is thrown, it is converted to an IOException.
+ *
+ * @param action the action to perform
+ * @return the result of the action
+ * @throws IOException in the event of error
+ */
+ public static <T> T doAsLoginUser(PrivilegedExceptionAction<T> action)
+ throws IOException {
+ return doAsUser(UserGroupInformation.getLoginUser(), action);
+ }
+
+ /**
+ * Perform the given action as the daemon's current user. If an
+ * InterruptedException is thrown, it is converted to an IOException.
+ *
+ * @param action the action to perform
+ * @return the result of the action
+ * @throws IOException in the event of error
+ */
+ public static <T> T doAsCurrentUser(PrivilegedExceptionAction<T> action)
+ throws IOException {
+ return doAsUser(UserGroupInformation.getCurrentUser(), action);
+ }
+
+ private static <T> T doAsUser(UserGroupInformation ugi,
+ PrivilegedExceptionAction<T> action) throws IOException {
+ try {
+ return ugi.doAs(action);
+ } catch (InterruptedException ie) {
+ throw new IOException(ie);
+ }
+ }
/**
* Open a (if need be) secure connection to a URL in a secure environment
diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
index 487da55..a0aa844 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
+++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
@@ -252,8 +252,6 @@
HDFS-3907. Allow multiple users for local block readers. (eli)
- HDFS-3910. DFSTestUtil#waitReplication should timeout. (eli)
-
OPTIMIZATIONS
HDFS-2982. Startup performance suffers when there are many edit log
@@ -578,6 +576,8 @@
HDFS-3928. MiniDFSCluster should reset the first ExitException on shutdown. (eli)
+ HDFS-3972. Trash emptier fails in secure HA cluster. (todd via eli)
+
BREAKDOWN OF HDFS-3042 SUBTASKS
HDFS-2185. HDFS portion of ZK-based FailoverController (todd)
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
index d682e4c..32b219f 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
@@ -22,6 +22,7 @@
import java.io.PrintStream;
import java.net.InetSocketAddress;
import java.net.URI;
+import java.security.PrivilegedExceptionAction;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
@@ -513,7 +514,7 @@
stopHttpServer();
}
- private void startTrashEmptier(Configuration conf) throws IOException {
+ private void startTrashEmptier(final Configuration conf) throws IOException {
long trashInterval =
conf.getLong(FS_TRASH_INTERVAL_KEY, FS_TRASH_INTERVAL_DEFAULT);
if (trashInterval == 0) {
@@ -522,7 +523,18 @@
throw new IOException("Cannot start tresh emptier with negative interval."
+ " Set " + FS_TRASH_INTERVAL_KEY + " to a positive value.");
}
- this.emptier = new Thread(new Trash(conf).getEmptier(), "Trash Emptier");
+
+ // This may be called from the transitionToActive code path, in which
+ // case the current user is the administrator, not the NN. The trash
+ // emptier needs to run as the NN. See HDFS-3972.
+ FileSystem fs = SecurityUtil.doAsLoginUser(
+ new PrivilegedExceptionAction<FileSystem>() {
+ @Override
+ public FileSystem run() throws IOException {
+ return FileSystem.get(conf);
+ }
+ });
+ this.emptier = new Thread(new Trash(fs, conf).getEmptier(), "Trash Emptier");
this.emptier.setDaemon(true);
this.emptier.start();
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
index c2d5520..b20baa9 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
@@ -273,7 +273,7 @@
* specified target.
*/
public void waitReplication(FileSystem fs, String topdir, short value)
- throws IOException, InterruptedException, TimeoutException {
+ throws IOException {
Path root = new Path(topdir);
/** wait for the replication factor to settle down */
@@ -498,44 +498,36 @@
return fileNames;
}
}
-
- /**
- * Wait for the given file to reach the given replication factor.
- * @throws TimeoutException if we fail to sufficiently replicate the file
- */
- public static void waitReplication(FileSystem fs, Path fileName, short replFactor)
- throws IOException, InterruptedException, TimeoutException {
- boolean correctReplFactor;
- final int ATTEMPTS = 20;
- int count = 0;
-
+
+ /** wait for the file's replication to be done */
+ public static void waitReplication(FileSystem fs, Path fileName,
+ short replFactor) throws IOException {
+ boolean good;
do {
- correctReplFactor = true;
+ good = true;
BlockLocation locs[] = fs.getFileBlockLocations(
fs.getFileStatus(fileName), 0, Long.MAX_VALUE);
- count++;
for (int j = 0; j < locs.length; j++) {
String[] hostnames = locs[j].getNames();
if (hostnames.length != replFactor) {
- correctReplFactor = false;
- System.out.println("Block " + j + " of file " + fileName
- + " has replication factor " + hostnames.length
- + " (desired " + replFactor + "); locations "
- + Joiner.on(' ').join(hostnames));
- Thread.sleep(1000);
+ String hostNameList = "";
+ for (String h : hostnames) hostNameList += h + " ";
+ System.out.println("Block " + j + " of file " + fileName
+ + " has replication factor " + hostnames.length + "; locations "
+ + hostNameList);
+ good = false;
+ try {
+ System.out.println("Waiting for replication factor to drain");
+ Thread.sleep(100);
+ } catch (InterruptedException e) {}
break;
}
}
- if (correctReplFactor) {
+ if (good) {
System.out.println("All blocks of file " + fileName
+ " verified to have replication factor " + replFactor);
}
- } while (!correctReplFactor && count < ATTEMPTS);
-
- if (count == ATTEMPTS) {
- throw new TimeoutException("Timed out waiting for " + fileName +
- " to reach " + replFactor + " replicas");
- }
+ } while(!good);
}
/** delete directory and everything underneath it.*/
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocal.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocal.java
index 7ccd5b6..ea1b58c7 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocal.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocal.java
@@ -61,7 +61,7 @@
* of this class might immediately issue a retry on failure, so it's polite.
*/
@Test
- public void testStablePositionAfterCorruptRead() throws Exception {
+ public void testStablePositionAfterCorruptRead() throws IOException {
final short REPL_FACTOR = 1;
final long FILE_LENGTH = 512L;
cluster.waitActive();
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
index 51fab66..327dd7c 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
@@ -24,7 +24,6 @@
import java.io.PrintStream;
import java.io.RandomAccessFile;
import java.util.Random;
-import java.util.concurrent.TimeoutException;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
@@ -200,11 +199,11 @@
}
/**
- * Create a file with one block and corrupt some/all of the block replicas.
+ * create a file with one block and corrupt some/all of the block replicas.
*/
private void createAFileWithCorruptedBlockReplicas(Path filePath, short repl,
int corruptBlockCount) throws IOException, AccessControlException,
- FileNotFoundException, UnresolvedLinkException, InterruptedException, TimeoutException {
+ FileNotFoundException, UnresolvedLinkException {
DFSTestUtil.createFile(dfs, filePath, BLOCK_SIZE, repl, 0);
DFSTestUtil.waitReplication(dfs, filePath, repl);
// Locate the file blocks by asking name node
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
index c89d9f6..c16beed 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
@@ -1037,7 +1037,8 @@
* way. See HDFS-3067.
*/
@Test
- public void testRetryOnChecksumFailure() throws Exception {
+ public void testRetryOnChecksumFailure()
+ throws UnresolvedLinkException, IOException {
HdfsConfiguration conf = new HdfsConfiguration();
MiniDFSCluster cluster =
new MiniDFSCluster.Builder(conf).numDataNodes(1).build();
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java
index a3cfb25..1e8191f 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDatanodeBlockScanner.java
@@ -182,7 +182,7 @@
}
@Test
- public void testBlockCorruptionPolicy() throws Exception {
+ public void testBlockCorruptionPolicy() throws IOException {
Configuration conf = new HdfsConfiguration();
conf.setLong(DFSConfigKeys.DFS_BLOCKREPORT_INTERVAL_MSEC_KEY, 1000L);
Random random = new Random();
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileStatus.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileStatus.java
index 28c3e9c..56cb450 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileStatus.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileStatus.java
@@ -25,7 +25,6 @@
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.Random;
-import java.util.concurrent.TimeoutException;
import org.apache.commons.logging.LogFactory;
import org.apache.commons.logging.impl.Log4JLogger;
@@ -101,7 +100,7 @@
}
private void checkFile(FileSystem fileSys, Path name, int repl)
- throws IOException, InterruptedException, TimeoutException {
+ throws IOException {
DFSTestUtil.waitReplication(fileSys, name, (short) repl);
}
@@ -130,7 +129,7 @@
/** Test the FileStatus obtained calling getFileStatus on a file */
@Test
- public void testGetFileStatusOnFile() throws Exception {
+ public void testGetFileStatusOnFile() throws IOException {
checkFile(fs, file1, 1);
// test getFileStatus on a file
FileStatus status = fs.getFileStatus(file1);
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
index 7ea963d..e819e02 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReplication.java
@@ -27,7 +27,6 @@
import java.net.InetSocketAddress;
import java.util.Iterator;
import java.util.Random;
-import java.util.concurrent.TimeoutException;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
@@ -421,8 +420,8 @@
}
}
- private void changeBlockLen(MiniDFSCluster cluster, int lenDelta)
- throws IOException, InterruptedException, TimeoutException {
+ private void changeBlockLen(MiniDFSCluster cluster,
+ int lenDelta) throws IOException, InterruptedException {
final Path fileName = new Path("/file1");
final short REPLICATION_FACTOR = (short)1;
final FileSystem fs = cluster.getFileSystem();
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
index 49fbd10..4929408 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
@@ -88,7 +88,7 @@
/* create a file with a length of <code>fileLen</code> */
static void createFile(MiniDFSCluster cluster, Path filePath, long fileLen,
short replicationFactor, int nnIndex)
- throws IOException, InterruptedException, TimeoutException {
+ throws IOException {
FileSystem fs = cluster.getFileSystem(nnIndex);
DFSTestUtil.createFile(fs, filePath, fileLen,
replicationFactor, r.nextLong());
@@ -100,7 +100,7 @@
* whose used space to be <code>size</code>
*/
private ExtendedBlock[] generateBlocks(Configuration conf, long size,
- short numNodes) throws IOException, InterruptedException, TimeoutException {
+ short numNodes) throws IOException {
cluster = new MiniDFSCluster.Builder(conf).numDataNodes(numNodes).build();
try {
cluster.waitActive();
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithMultipleNameNodes.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithMultipleNameNodes.java
index f584804..dfd0b94 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithMultipleNameNodes.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithMultipleNameNodes.java
@@ -23,7 +23,6 @@
import java.util.Collection;
import java.util.List;
import java.util.Random;
-import java.util.concurrent.TimeoutException;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
@@ -97,7 +96,7 @@
/* create a file with a length of <code>fileLen</code> */
private static void createFile(Suite s, int index, long len
- ) throws IOException, InterruptedException, TimeoutException {
+ ) throws IOException {
final FileSystem fs = s.cluster.getFileSystem(index);
DFSTestUtil.createFile(fs, FILE_PATH, len, s.replication, RANDOM.nextLong());
DFSTestUtil.waitReplication(fs, FILE_PATH, s.replication);
@@ -107,7 +106,7 @@
* whose used space to be <code>size</code>
*/
private static ExtendedBlock[][] generateBlocks(Suite s, long size
- ) throws IOException, InterruptedException, TimeoutException {
+ ) throws IOException {
final ExtendedBlock[][] blocks = new ExtendedBlock[s.clients.length][];
for(int n = 0; n < s.clients.length; n++) {
final long fileLen = size/s.replication;
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
index 7978596..dc8578e 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
@@ -53,7 +53,7 @@
* corrupt ones.
*/
@Test
- public void testProcesOverReplicateBlock() throws Exception {
+ public void testProcesOverReplicateBlock() throws IOException {
Configuration conf = new HdfsConfiguration();
conf.setLong(DFSConfigKeys.DFS_BLOCKREPORT_INTERVAL_MSEC_KEY, 1000L);
conf.set(
@@ -141,7 +141,7 @@
* send heartbeats.
*/
@Test
- public void testChooseReplicaToDelete() throws Exception {
+ public void testChooseReplicaToDelete() throws IOException {
MiniDFSCluster cluster = null;
FileSystem fs = null;
try {
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReplacement.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReplacement.java
index a138071..b9f58ba 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReplacement.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReplacement.java
@@ -89,7 +89,7 @@
}
@Test
- public void testBlockReplacement() throws Exception {
+ public void testBlockReplacement() throws IOException, TimeoutException {
final Configuration CONF = new HdfsConfiguration();
final String[] INITIAL_RACKS = {"/RACK0", "/RACK1", "/RACK2"};
final String[] NEW_RACKS = {"/RACK2"};
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReport.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReport.java
index c4f4c4e..1661d8c 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReport.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReport.java
@@ -27,9 +27,6 @@
import java.util.List;
import java.util.Random;
import java.util.concurrent.CountDownLatch;
-import java.util.concurrent.TimeoutException;
-
-import junit.framework.Assert;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
@@ -68,7 +65,7 @@
/**
* This test simulates a variety of situations when blocks are being
- * intentionally corrupted, unexpectedly modified, and so on before a block
+ * intentionally orrupted, unexpectedly modified, and so on before a block
* report is happening
*/
public class TestBlockReport {
@@ -319,7 +316,7 @@
* @throws IOException in case of an error
*/
@Test
- public void blockReport_06() throws Exception {
+ public void blockReport_06() throws IOException {
final String METHOD_NAME = GenericTestUtils.getMethodName();
Path filePath = new Path("/" + METHOD_NAME + ".dat");
final int DN_N1 = DN_N0 + 1;
@@ -356,7 +353,7 @@
@Test
// Currently this test is failing as expected 'cause the correct behavior is
// not yet implemented (9/15/09)
- public void blockReport_07() throws Exception {
+ public void blockReport_07() throws IOException {
final String METHOD_NAME = GenericTestUtils.getMethodName();
Path filePath = new Path("/" + METHOD_NAME + ".dat");
final int DN_N1 = DN_N0 + 1;
@@ -673,24 +670,21 @@
}
private void startDNandWait(Path filePath, boolean waitReplicas)
- throws IOException, InterruptedException, TimeoutException {
- if (LOG.isDebugEnabled()) {
+ throws IOException {
+ if(LOG.isDebugEnabled()) {
LOG.debug("Before next DN start: " + cluster.getDataNodes().size());
}
cluster.startDataNodes(conf, 1, true, null, null);
- cluster.waitClusterUp();
ArrayList<DataNode> datanodes = cluster.getDataNodes();
assertEquals(datanodes.size(), 2);
- if (LOG.isDebugEnabled()) {
+ if(LOG.isDebugEnabled()) {
int lastDn = datanodes.size() - 1;
LOG.debug("New datanode "
+ cluster.getDataNodes().get(lastDn).getDisplayName()
+ " has been started");
}
- if (waitReplicas) {
- DFSTestUtil.waitReplication(fs, filePath, REPL_FACTOR);
- }
+ if (waitReplicas) DFSTestUtil.waitReplication(fs, filePath, REPL_FACTOR);
}
private ArrayList<Block> prepareForRide(final Path filePath,
@@ -842,9 +836,8 @@
public void run() {
try {
startDNandWait(filePath, true);
- } catch (Exception e) {
- e.printStackTrace();
- Assert.fail("Failed to start BlockChecker: " + e);
+ } catch (IOException e) {
+ LOG.warn("Shouldn't happen", e);
}
}
}
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
index 5a80098..ac1cf03 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeVolumeFailure.java
@@ -105,7 +105,7 @@
* failure if the configuration parameter allows this.
*/
@Test
- public void testVolumeFailure() throws Exception {
+ public void testVolumeFailure() throws IOException {
FileSystem fs = cluster.getFileSystem();
dataDir = new File(cluster.getDataDirectory());
System.out.println("Data dir: is " + dataDir.getPath());
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestDatanodeRestart.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestDatanodeRestart.java
index a91baec..9b4e0d4 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestDatanodeRestart.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestDatanodeRestart.java
@@ -137,7 +137,7 @@
}
// test recovering unlinked tmp replicas
- @Test public void testRecoverReplicas() throws Exception {
+ @Test public void testRecoverReplicas() throws IOException {
Configuration conf = new HdfsConfiguration();
conf.setLong(DFSConfigKeys.DFS_BLOCK_SIZE_KEY, 1024L);
conf.setInt(DFSConfigKeys.DFS_CLIENT_WRITE_PACKET_SIZE_KEY, 512);
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java
index 7b6f3eb..0fd2e5f 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java
@@ -114,7 +114,7 @@
* automatically bumped up to the new minimum upon restart.
*/
@Test
- public void testReplicationAdjusted() throws Exception {
+ public void testReplicationAdjusted() throws IOException {
// start a cluster
Configuration conf = new HdfsConfiguration();
// Replicate and heartbeat fast to shave a few seconds off test
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
index d6d849d..a843962 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
@@ -53,7 +53,7 @@
* replicas (2) is equal to replication factor (2))
*/
@Test
- public void testWhenDecreasingReplication() throws Exception {
+ public void testWhenDecreasingReplication() throws IOException {
Configuration conf = new HdfsConfiguration();
conf.setLong(DFSConfigKeys.DFS_BLOCKREPORT_INTERVAL_MSEC_KEY, 1000L);
conf.set(DFSConfigKeys.DFS_NAMENODE_REPLICATION_PENDING_TIMEOUT_SEC_KEY, Integer.toString(2));
@@ -108,7 +108,7 @@
*
*/
@Test
- public void testByAddingAnExtraDataNode() throws Exception {
+ public void testByAddingAnExtraDataNode() throws IOException {
Configuration conf = new HdfsConfiguration();
conf.setLong(DFSConfigKeys.DFS_BLOCKREPORT_INTERVAL_MSEC_KEY, 1000L);
conf.set(DFSConfigKeys.DFS_NAMENODE_REPLICATION_PENDING_TIMEOUT_SEC_KEY, Integer.toString(2));
@@ -159,7 +159,7 @@
* replicas (1) is equal to replication factor (1))
*/
@Test
- public void testWithReplicationFactorAsOne() throws Exception {
+ public void testWithReplicationFactorAsOne() throws IOException {
Configuration conf = new HdfsConfiguration();
conf.setLong(DFSConfigKeys.DFS_BLOCKREPORT_INTERVAL_MSEC_KEY, 1000L);
conf.set(DFSConfigKeys.DFS_NAMENODE_REPLICATION_PENDING_TIMEOUT_SEC_KEY, Integer.toString(2));
@@ -208,7 +208,7 @@
* Verify that all replicas are corrupt and 3 replicas are present.
*/
@Test
- public void testWithAllCorruptReplicas() throws Exception {
+ public void testWithAllCorruptReplicas() throws IOException {
Configuration conf = new HdfsConfiguration();
conf.setLong(DFSConfigKeys.DFS_BLOCKREPORT_INTERVAL_MSEC_KEY, 1000L);
conf.set(DFSConfigKeys.DFS_NAMENODE_REPLICATION_PENDING_TIMEOUT_SEC_KEY, Integer.toString(2));
diff --git a/hadoop-mapreduce-project/CHANGES.txt b/hadoop-mapreduce-project/CHANGES.txt
index b759a4c..900fade 100644
--- a/hadoop-mapreduce-project/CHANGES.txt
+++ b/hadoop-mapreduce-project/CHANGES.txt
@@ -43,9 +43,6 @@
MAPREDUCE-3921. MR AM should act on node health status changes.
(Bikas Saha via sseth)
- MAPREDUCE-4253. Tests for mapreduce-client-core are lying under
- mapreduce-client-jobclient (Tsuyoshi Ozawa via harsh)
-
MAPREDUCE-4355. Add RunningJob.getJobStatus() (kkambatl via tucu)
MAPREDUCE-4427. Added an 'unmanaged' mode for AMs so as to ease
@@ -396,6 +393,15 @@
BUG FIXES
+ MAPREDUCE-4647. We should only unjar jobjar if there is a lib directory
+ in it. (Robert Evans via tgraves)
+
+ MAPREDUCE-4691. Historyserver can report "Unknown job" after RM says job
+ has completed (Robert Joseph Evans via jlowe)
+
+ MAPREDUCE-4689. JobClient.getMapTaskReports on failed job results in NPE
+ (jlowe via bobby)
+
Release 0.23.3 - UNRELEASED
INCOMPATIBLE CHANGES
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
index f47e03a..1409a3b 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
@@ -45,6 +45,7 @@
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.DataOutputBuffer;
import org.apache.hadoop.mapred.JobConf;
+import org.apache.hadoop.mapred.JobContext;
import org.apache.hadoop.mapred.MapReduceChildJVM;
import org.apache.hadoop.mapred.ShuffleHandler;
import org.apache.hadoop.mapred.Task;
@@ -605,10 +606,12 @@
if (jobJar != null) {
Path remoteJobJar = (new Path(jobJar)).makeQualified(remoteFS
.getUri(), remoteFS.getWorkingDirectory());
- localResources.put(
- MRJobConfig.JOB_JAR,
- createLocalResource(remoteFS, remoteJobJar,
- LocalResourceType.ARCHIVE, LocalResourceVisibility.APPLICATION));
+ LocalResource rc = createLocalResource(remoteFS, remoteJobJar,
+ LocalResourceType.PATTERN, LocalResourceVisibility.APPLICATION);
+ String pattern = conf.getPattern(JobContext.JAR_UNPACK_PATTERN,
+ JobConf.UNPACK_JAR_PATTERN_DEFAULT).pattern();
+ rc.setPattern(pattern);
+ localResources.put(MRJobConfig.JOB_JAR, rc);
LOG.info("The job-jar file on the remote FS is "
+ remoteJobJar.toUri().toASCIIString());
} else {
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapred/LocalDistributedCacheManager.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapred/LocalDistributedCacheManager.java
index fb26245..3368d51 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapred/LocalDistributedCacheManager.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapred/LocalDistributedCacheManager.java
@@ -152,6 +152,10 @@
localArchives.add(pathString);
} else if (resource.getType() == LocalResourceType.FILE) {
localFiles.add(pathString);
+ } else if (resource.getType() == LocalResourceType.PATTERN) {
+ //PATTERN is not currently used in local mode
+ throw new IllegalArgumentException("Resource type PATTERN is not " +
+ "implemented yet. " + resource.getResource());
}
Path resourcePath;
try {
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
index 4235e72..055b079 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java
@@ -210,7 +210,7 @@
Apps.addToEnvironment(
environment,
Environment.CLASSPATH.name(),
- MRJobConfig.JOB_JAR + Path.SEPARATOR);
+ MRJobConfig.JOB_JAR + Path.SEPARATOR + MRJobConfig.JOB_JAR);
Apps.addToEnvironment(
environment,
Environment.CLASSPATH.name(),
@@ -281,7 +281,7 @@
}
private static String getResourceDescription(LocalResourceType type) {
- if(type == LocalResourceType.ARCHIVE) {
+ if(type == LocalResourceType.ARCHIVE || type == LocalResourceType.PATTERN) {
return "cache archive (" + MRJobConfig.CACHE_ARCHIVES + ") ";
}
return "cache file (" + MRJobConfig.CACHE_FILES + ") ";
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java
index 345539d..dbd3538 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java
@@ -166,7 +166,7 @@
}
String env_str = env.get("CLASSPATH");
assertSame("MAPREDUCE_JOB_USER_CLASSPATH_FIRST set, but not taking effect!",
- env_str.indexOf("$PWD:job.jar/:job.jar/classes/:job.jar/lib/*:$PWD/*"), 0);
+ env_str.indexOf("$PWD:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:$PWD/*"), 0);
}
@Test public void testSetClasspathWithNoUserPrecendence() {
@@ -180,7 +180,7 @@
}
String env_str = env.get("CLASSPATH");
int index =
- env_str.indexOf("job.jar/:job.jar/classes/:job.jar/lib/*:$PWD/*");
+ env_str.indexOf("job.jar/job.jar:job.jar/classes/:job.jar/lib/*:$PWD/*");
assertNotSame("MAPREDUCE_JOB_USER_CLASSPATH_FIRST false, and job.jar is not"
+ " in the classpath!", index, -1);
assertNotSame("MAPREDUCE_JOB_USER_CLASSPATH_FIRST false, but taking effect!",
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTask.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTask.java
index 669eaa4..830b64f 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTask.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedTask.java
@@ -42,6 +42,8 @@
public class CompletedTask implements Task {
+ private static final Counters EMPTY_COUNTERS = new Counters();
+
private final TaskId taskId;
private final TaskInfo taskInfo;
private TaskReport report;
@@ -124,7 +126,11 @@
report.setFinishTime(taskInfo.getFinishTime());
report.setTaskState(getState());
report.setProgress(getProgress());
- report.setCounters(TypeConverter.toYarn(getCounters()));
+ Counters counters = getCounters();
+ if (counters == null) {
+ counters = EMPTY_COUNTERS;
+ }
+ report.setCounters(TypeConverter.toYarn(counters));
if (successfulAttempt != null) {
report.setSuccessfulAttempt(successfulAttempt);
}
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
index 6779588..85fcdc6 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
@@ -23,14 +23,14 @@
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
-import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
-import java.util.Map;
import java.util.Set;
import java.util.SortedMap;
import java.util.TreeMap;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadFactory;
@@ -77,7 +77,7 @@
private static enum HistoryInfoState {
IN_INTERMEDIATE, IN_DONE, DELETED, MOVE_FAILED
};
-
+
private static String DONE_BEFORE_SERIAL_TAIL = JobHistoryUtils
.doneSubdirsBeforeSerialTail();
@@ -199,6 +199,29 @@
}
}
+ /**
+ * This class represents a user dir in the intermediate done directory. This
+ * is mostly for locking purposes.
+ */
+ private class UserLogDir {
+ long modTime = 0;
+
+ public synchronized void scanIfNeeded(FileStatus fs) {
+ long newModTime = fs.getModificationTime();
+ if (modTime != newModTime) {
+ Path p = fs.getPath();
+ try {
+ scanIntermediateDirectory(p);
+ //If scanning fails, we will scan again. We assume the failure is
+ // temporary.
+ modTime = newModTime;
+ } catch (IOException e) {
+ LOG.error("Error while trying to scan the directory " + p, e);
+ }
+ }
+ }
+ }
+
public class HistoryFileInfo {
private Path historyFile;
private Path confFile;
@@ -352,7 +375,8 @@
* Maintains a mapping between intermediate user directories and the last
* known modification time.
*/
- private Map<String, Long> userDirModificationTimeMap = new HashMap<String, Long>();
+ private ConcurrentMap<String, UserLogDir> userDirModificationTimeMap =
+ new ConcurrentHashMap<String, UserLogDir>();
private JobACLsManager aclsMgr;
@@ -584,23 +608,15 @@
for (FileStatus userDir : userDirList) {
String name = userDir.getPath().getName();
- long newModificationTime = userDir.getModificationTime();
- boolean shouldScan = false;
- synchronized (userDirModificationTimeMap) {
- if (!userDirModificationTimeMap.containsKey(name)
- || newModificationTime > userDirModificationTimeMap.get(name)) {
- shouldScan = true;
- userDirModificationTimeMap.put(name, newModificationTime);
+ UserLogDir dir = userDirModificationTimeMap.get(name);
+ if(dir == null) {
+ dir = new UserLogDir();
+ UserLogDir old = userDirModificationTimeMap.putIfAbsent(name, dir);
+ if(old != null) {
+ dir = old;
}
}
- if (shouldScan) {
- try {
- scanIntermediateDirectory(userDir.getPath());
- } catch (IOException e) {
- LOG.error("Error while trying to scan the directory "
- + userDir.getPath(), e);
- }
- }
+ dir.scanIfNeeded(userDir);
}
}
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
index b596a21..f9acb1a 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java
@@ -49,6 +49,7 @@
import org.apache.hadoop.mapreduce.v2.api.records.JobId;
import org.apache.hadoop.mapreduce.v2.api.records.JobState;
import org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId;
+import org.apache.hadoop.mapreduce.v2.api.records.TaskId;
import org.apache.hadoop.mapreduce.v2.api.records.TaskState;
import org.apache.hadoop.mapreduce.v2.app.MRApp;
import org.apache.hadoop.mapreduce.v2.app.job.Job;
@@ -402,6 +403,63 @@
}
}
+ @Test
+ public void testCountersForFailedTask() throws Exception {
+ LOG.info("STARTING testCountersForFailedTask");
+ try {
+ Configuration conf = new Configuration();
+ conf
+ .setClass(
+ CommonConfigurationKeysPublic.NET_TOPOLOGY_NODE_SWITCH_MAPPING_IMPL_KEY,
+ MyResolver.class, DNSToSwitchMapping.class);
+ RackResolver.init(conf);
+ MRApp app = new MRAppWithHistoryWithFailedTask(2, 1, true,
+ this.getClass().getName(), true);
+ app.submit(conf);
+ Job job = app.getContext().getAllJobs().values().iterator().next();
+ JobId jobId = job.getID();
+ app.waitForState(job, JobState.FAILED);
+
+ // make sure all events are flushed
+ app.waitForState(Service.STATE.STOPPED);
+
+ String jobhistoryDir = JobHistoryUtils
+ .getHistoryIntermediateDoneDirForUser(conf);
+ JobHistory jobHistory = new JobHistory();
+ jobHistory.init(conf);
+
+ JobIndexInfo jobIndexInfo = jobHistory.getJobFileInfo(jobId)
+ .getJobIndexInfo();
+ String jobhistoryFileName = FileNameIndexUtils
+ .getDoneFileName(jobIndexInfo);
+
+ Path historyFilePath = new Path(jobhistoryDir, jobhistoryFileName);
+ FSDataInputStream in = null;
+ FileContext fc = null;
+ try {
+ fc = FileContext.getFileContext(conf);
+ in = fc.open(fc.makeQualified(historyFilePath));
+ } catch (IOException ioe) {
+ LOG.info("Can not open history file: " + historyFilePath, ioe);
+ throw (new Exception("Can not open History File"));
+ }
+
+ JobHistoryParser parser = new JobHistoryParser(in);
+ JobInfo jobInfo = parser.parse();
+ Exception parseException = parser.getParseException();
+ Assert.assertNull("Caught an expected exception " + parseException,
+ parseException);
+ for (Map.Entry<TaskID,TaskInfo> entry : jobInfo.getAllTasks().entrySet()) {
+ TaskId yarnTaskID = TypeConverter.toYarn(entry.getKey());
+ CompletedTask ct = new CompletedTask(yarnTaskID, entry.getValue());
+ Assert.assertNotNull("completed task report has null counters",
+ ct.getReport().getCounters());
+ }
+ } finally {
+ LOG.info("FINISHED testCountersForFailedTask");
+ }
+ }
+
static class MRAppWithHistoryWithFailedAttempt extends MRAppWithHistory {
public MRAppWithHistoryWithFailedAttempt(int maps, int reduces, boolean autoComplete,
@@ -422,6 +480,26 @@
}
}
+ static class MRAppWithHistoryWithFailedTask extends MRAppWithHistory {
+
+ public MRAppWithHistoryWithFailedTask(int maps, int reduces, boolean autoComplete,
+ String testName, boolean cleanOnStart) {
+ super(maps, reduces, autoComplete, testName, cleanOnStart);
+ }
+
+ @SuppressWarnings("unchecked")
+ @Override
+ protected void attemptLaunched(TaskAttemptId attemptID) {
+ if (attemptID.getTaskId().getId() == 0) {
+ getContext().getEventHandler().handle(
+ new TaskAttemptEvent(attemptID, TaskAttemptEventType.TA_FAILMSG));
+ } else {
+ getContext().getEventHandler().handle(
+ new TaskAttemptEvent(attemptID, TaskAttemptEventType.TA_DONE));
+ }
+ }
+ }
+
public static void main(String[] args) throws Exception {
TestJobHistoryParsing t = new TestJobHistoryParsing();
t.testHistoryParsing();
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
index f327176..c685072 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java
@@ -346,9 +346,13 @@
jobConfPath, LocalResourceType.FILE));
if (jobConf.get(MRJobConfig.JAR) != null) {
Path jobJarPath = new Path(jobConf.get(MRJobConfig.JAR));
- localResources.put(MRJobConfig.JOB_JAR,
- createApplicationResource(defaultFileContext,
- jobJarPath, LocalResourceType.ARCHIVE));
+ LocalResource rc = createApplicationResource(defaultFileContext,
+ jobJarPath,
+ LocalResourceType.PATTERN);
+ String pattern = conf.getPattern(JobContext.JAR_UNPACK_PATTERN,
+ JobConf.UNPACK_JAR_PATTERN_DEFAULT).pattern();
+ rc.setPattern(pattern);
+ localResources.put(MRJobConfig.JOB_JAR, rc);
} else {
// Job jar may be null. For e.g, for pipes, the job jar is the hadoop
// mapreduce jar itself which is already on the classpath.
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestAuditLogger.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestAuditLogger.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestAuditLogger.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestAuditLogger.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileInputFormat.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileInputFormat.java
index 6b0406f..3c59943 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileInputFormat.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestFileInputFormat.java
@@ -23,7 +23,6 @@
import java.io.DataOutputStream;
import java.io.IOException;
-import java.util.concurrent.TimeoutException;
import junit.framework.TestCase;
@@ -96,7 +95,7 @@
}
private void createInputs(FileSystem fs, Path inDir, String fileName)
- throws IOException, TimeoutException, InterruptedException {
+ throws IOException {
// create a multi-block file on hdfs
Path path = new Path(inDir, fileName);
final short replication = 2;
@@ -158,7 +157,7 @@
}
}
- public void testMultiLevelInput() throws Exception {
+ public void testMultiLevelInput() throws IOException {
JobConf job = new JobConf(conf);
job.setBoolean("dfs.replication.considerLoad", false);
@@ -292,8 +291,7 @@
}
static void writeFile(Configuration conf, Path name,
- short replication, int numBlocks)
- throws IOException, TimeoutException, InterruptedException {
+ short replication, int numBlocks) throws IOException {
FileSystem fileSys = FileSystem.get(conf);
FSDataOutputStream stm = fileSys.create(name, true,
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestIFile.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestIFile.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestIFile.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestIFile.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestIndexCache.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestIndexCache.java
new file mode 100644
index 0000000..b6a2df0
--- /dev/null
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestIndexCache.java
@@ -0,0 +1,324 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.mapred;
+
+import java.io.DataOutputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Random;
+import java.util.zip.CRC32;
+import java.util.zip.CheckedOutputStream;
+
+import org.apache.hadoop.fs.ChecksumException;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.mapreduce.server.tasktracker.TTConfig;
+
+import junit.framework.TestCase;
+
+public class TestIndexCache extends TestCase {
+ private JobConf conf;
+ private FileSystem fs;
+ private Path p;
+
+ @Override
+ public void setUp() throws IOException {
+ conf = new JobConf();
+ fs = FileSystem.getLocal(conf).getRaw();
+ p = new Path(System.getProperty("test.build.data", "/tmp"),
+ "cache").makeQualified(fs.getUri(), fs.getWorkingDirectory());
+ }
+
+ public void testLRCPolicy() throws Exception {
+ Random r = new Random();
+ long seed = r.nextLong();
+ r.setSeed(seed);
+ System.out.println("seed: " + seed);
+ fs.delete(p, true);
+ conf.setInt(TTConfig.TT_INDEX_CACHE, 1);
+ final int partsPerMap = 1000;
+ final int bytesPerFile = partsPerMap * 24;
+ IndexCache cache = new IndexCache(conf);
+
+ // fill cache
+ int totalsize = bytesPerFile;
+ for (; totalsize < 1024 * 1024; totalsize += bytesPerFile) {
+ Path f = new Path(p, Integer.toString(totalsize, 36));
+ writeFile(fs, f, totalsize, partsPerMap);
+ IndexRecord rec = cache.getIndexInformation(
+ Integer.toString(totalsize, 36), r.nextInt(partsPerMap), f,
+ UserGroupInformation.getCurrentUser().getShortUserName());
+ checkRecord(rec, totalsize);
+ }
+
+ // delete files, ensure cache retains all elem
+ for (FileStatus stat : fs.listStatus(p)) {
+ fs.delete(stat.getPath(),true);
+ }
+ for (int i = bytesPerFile; i < 1024 * 1024; i += bytesPerFile) {
+ Path f = new Path(p, Integer.toString(i, 36));
+ IndexRecord rec = cache.getIndexInformation(Integer.toString(i, 36),
+ r.nextInt(partsPerMap), f,
+ UserGroupInformation.getCurrentUser().getShortUserName());
+ checkRecord(rec, i);
+ }
+
+ // push oldest (bytesPerFile) out of cache
+ Path f = new Path(p, Integer.toString(totalsize, 36));
+ writeFile(fs, f, totalsize, partsPerMap);
+ cache.getIndexInformation(Integer.toString(totalsize, 36),
+ r.nextInt(partsPerMap), f,
+ UserGroupInformation.getCurrentUser().getShortUserName());
+ fs.delete(f, false);
+
+ // oldest fails to read, or error
+ boolean fnf = false;
+ try {
+ cache.getIndexInformation(Integer.toString(bytesPerFile, 36),
+ r.nextInt(partsPerMap), new Path(p, Integer.toString(bytesPerFile)),
+ UserGroupInformation.getCurrentUser().getShortUserName());
+ } catch (IOException e) {
+ if (e.getCause() == null ||
+ !(e.getCause() instanceof FileNotFoundException)) {
+ throw e;
+ }
+ else {
+ fnf = true;
+ }
+ }
+ if (!fnf)
+ fail("Failed to push out last entry");
+ // should find all the other entries
+ for (int i = bytesPerFile << 1; i < 1024 * 1024; i += bytesPerFile) {
+ IndexRecord rec = cache.getIndexInformation(Integer.toString(i, 36),
+ r.nextInt(partsPerMap), new Path(p, Integer.toString(i, 36)),
+ UserGroupInformation.getCurrentUser().getShortUserName());
+ checkRecord(rec, i);
+ }
+ IndexRecord rec = cache.getIndexInformation(Integer.toString(totalsize, 36),
+ r.nextInt(partsPerMap), f,
+ UserGroupInformation.getCurrentUser().getShortUserName());
+
+ checkRecord(rec, totalsize);
+ }
+
+ public void testBadIndex() throws Exception {
+ final int parts = 30;
+ fs.delete(p, true);
+ conf.setInt(TTConfig.TT_INDEX_CACHE, 1);
+ IndexCache cache = new IndexCache(conf);
+
+ Path f = new Path(p, "badindex");
+ FSDataOutputStream out = fs.create(f, false);
+ CheckedOutputStream iout = new CheckedOutputStream(out, new CRC32());
+ DataOutputStream dout = new DataOutputStream(iout);
+ for (int i = 0; i < parts; ++i) {
+ for (int j = 0; j < MapTask.MAP_OUTPUT_INDEX_RECORD_LENGTH / 8; ++j) {
+ if (0 == (i % 3)) {
+ dout.writeLong(i);
+ } else {
+ out.writeLong(i);
+ }
+ }
+ }
+ out.writeLong(iout.getChecksum().getValue());
+ dout.close();
+ try {
+ cache.getIndexInformation("badindex", 7, f,
+ UserGroupInformation.getCurrentUser().getShortUserName());
+ fail("Did not detect bad checksum");
+ } catch (IOException e) {
+ if (!(e.getCause() instanceof ChecksumException)) {
+ throw e;
+ }
+ }
+ }
+
+ public void testInvalidReduceNumberOrLength() throws Exception {
+ fs.delete(p, true);
+ conf.setInt(TTConfig.TT_INDEX_CACHE, 1);
+ final int partsPerMap = 1000;
+ final int bytesPerFile = partsPerMap * 24;
+ IndexCache cache = new IndexCache(conf);
+
+ // fill cache
+ Path feq = new Path(p, "invalidReduceOrPartsPerMap");
+ writeFile(fs, feq, bytesPerFile, partsPerMap);
+
+ // Number of reducers should always be less than partsPerMap as reducer
+ // numbers start from 0 and there cannot be more reducer than parts
+
+ try {
+ // Number of reducers equal to partsPerMap
+ cache.getIndexInformation("reduceEqualPartsPerMap",
+ partsPerMap, // reduce number == partsPerMap
+ feq, UserGroupInformation.getCurrentUser().getShortUserName());
+ fail("Number of reducers equal to partsPerMap did not fail");
+ } catch (Exception e) {
+ if (!(e instanceof IOException)) {
+ throw e;
+ }
+ }
+
+ try {
+ // Number of reducers more than partsPerMap
+ cache.getIndexInformation(
+ "reduceMorePartsPerMap",
+ partsPerMap + 1, // reduce number > partsPerMap
+ feq, UserGroupInformation.getCurrentUser().getShortUserName());
+ fail("Number of reducers more than partsPerMap did not fail");
+ } catch (Exception e) {
+ if (!(e instanceof IOException)) {
+ throw e;
+ }
+ }
+ }
+
+ public void testRemoveMap() throws Exception {
+ // This test case use two thread to call getIndexInformation and
+ // removeMap concurrently, in order to construct race condition.
+ // This test case may not repeatable. But on my macbook this test
+ // fails with probability of 100% on code before MAPREDUCE-2541,
+ // so it is repeatable in practice.
+ fs.delete(p, true);
+ conf.setInt(TTConfig.TT_INDEX_CACHE, 10);
+ // Make a big file so removeMapThread almost surely runs faster than
+ // getInfoThread
+ final int partsPerMap = 100000;
+ final int bytesPerFile = partsPerMap * 24;
+ final IndexCache cache = new IndexCache(conf);
+
+ final Path big = new Path(p, "bigIndex");
+ final String user =
+ UserGroupInformation.getCurrentUser().getShortUserName();
+ writeFile(fs, big, bytesPerFile, partsPerMap);
+
+ // run multiple times
+ for (int i = 0; i < 20; ++i) {
+ Thread getInfoThread = new Thread() {
+ @Override
+ public void run() {
+ try {
+ cache.getIndexInformation("bigIndex", partsPerMap, big, user);
+ } catch (Exception e) {
+ // should not be here
+ }
+ }
+ };
+ Thread removeMapThread = new Thread() {
+ @Override
+ public void run() {
+ cache.removeMap("bigIndex");
+ }
+ };
+ if (i%2==0) {
+ getInfoThread.start();
+ removeMapThread.start();
+ } else {
+ removeMapThread.start();
+ getInfoThread.start();
+ }
+ getInfoThread.join();
+ removeMapThread.join();
+ assertEquals(true, cache.checkTotalMemoryUsed());
+ }
+ }
+
+ public void testCreateRace() throws Exception {
+ fs.delete(p, true);
+ conf.setInt(TTConfig.TT_INDEX_CACHE, 1);
+ final int partsPerMap = 1000;
+ final int bytesPerFile = partsPerMap * 24;
+ final IndexCache cache = new IndexCache(conf);
+
+ final Path racy = new Path(p, "racyIndex");
+ final String user =
+ UserGroupInformation.getCurrentUser().getShortUserName();
+ writeFile(fs, racy, bytesPerFile, partsPerMap);
+
+ // run multiple instances
+ Thread[] getInfoThreads = new Thread[50];
+ for (int i = 0; i < 50; i++) {
+ getInfoThreads[i] = new Thread() {
+ @Override
+ public void run() {
+ try {
+ cache.getIndexInformation("racyIndex", partsPerMap, racy, user);
+ cache.removeMap("racyIndex");
+ } catch (Exception e) {
+ // should not be here
+ }
+ }
+ };
+ }
+
+ for (int i = 0; i < 50; i++) {
+ getInfoThreads[i].start();
+ }
+
+ final Thread mainTestThread = Thread.currentThread();
+
+ Thread timeoutThread = new Thread() {
+ @Override
+ public void run() {
+ try {
+ Thread.sleep(15000);
+ mainTestThread.interrupt();
+ } catch (InterruptedException ie) {
+ // we are done;
+ }
+ }
+ };
+
+ for (int i = 0; i < 50; i++) {
+ try {
+ getInfoThreads[i].join();
+ } catch (InterruptedException ie) {
+ // we haven't finished in time. Potential deadlock/race.
+ fail("Unexpectedly long delay during concurrent cache entry creations");
+ }
+ }
+ // stop the timeoutThread. If we get interrupted before stopping, there
+ // must be something wrong, although it wasn't a deadlock. No need to
+ // catch and swallow.
+ timeoutThread.interrupt();
+ }
+
+ private static void checkRecord(IndexRecord rec, long fill) {
+ assertEquals(fill, rec.startOffset);
+ assertEquals(fill, rec.rawLength);
+ assertEquals(fill, rec.partLength);
+ }
+
+ private static void writeFile(FileSystem fs, Path f, long fill, int parts)
+ throws IOException {
+ FSDataOutputStream out = fs.create(f, false);
+ CheckedOutputStream iout = new CheckedOutputStream(out, new CRC32());
+ DataOutputStream dout = new DataOutputStream(iout);
+ for (int i = 0; i < parts; ++i) {
+ for (int j = 0; j < MapTask.MAP_OUTPUT_INDEX_RECORD_LENGTH / 8; ++j) {
+ dout.writeLong(fill);
+ }
+ }
+ out.writeLong(iout.getChecksum().getValue());
+ dout.close();
+ }
+}
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestKeyValueTextInputFormat.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestKeyValueTextInputFormat.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestKeyValueTextInputFormat.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestKeyValueTextInputFormat.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestMultiFileInputFormat.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultiFileInputFormat.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestMultiFileInputFormat.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultiFileInputFormat.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestMultiFileSplit.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultiFileSplit.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestMultiFileSplit.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultiFileSplit.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultipleLevelCaching.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultipleLevelCaching.java
index cc9e88a..6d3fd29 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultipleLevelCaching.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMultipleLevelCaching.java
@@ -71,13 +71,13 @@
return rack.toString();
}
- public void testMultiLevelCaching() throws Exception {
+ public void testMultiLevelCaching() throws IOException {
for (int i = 1 ; i <= MAX_LEVEL; ++i) {
testCachingAtLevel(i);
}
}
- private void testCachingAtLevel(int level) throws Exception {
+ private void testCachingAtLevel(int level) throws IOException {
String namenode = null;
MiniDFSCluster dfs = null;
MiniMRCluster mr = null;
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestReduceTask.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestReduceTask.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestReduceTask.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestReduceTask.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestSequenceFileAsBinaryInputFormat.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestSequenceFileAsBinaryInputFormat.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestSequenceFileAsBinaryInputFormat.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestSequenceFileAsBinaryInputFormat.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestSequenceFileAsBinaryOutputFormat.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestSequenceFileAsBinaryOutputFormat.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestSequenceFileAsBinaryOutputFormat.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestSequenceFileAsBinaryOutputFormat.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestSequenceFileAsTextInputFormat.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestSequenceFileAsTextInputFormat.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestSequenceFileAsTextInputFormat.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestSequenceFileAsTextInputFormat.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestSequenceFileInputFilter.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestSequenceFileInputFilter.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestSequenceFileInputFilter.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestSequenceFileInputFilter.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestSortedRanges.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestSortedRanges.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestSortedRanges.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestSortedRanges.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestStatisticsCollector.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestStatisticsCollector.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestStatisticsCollector.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestStatisticsCollector.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestTaskStatus.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTaskStatus.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestTaskStatus.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestTaskStatus.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestUtils.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestUtils.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestUtils.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestUtils.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/UtilsForTests.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/UtilsForTests.java
index 972391c..63353b6 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/UtilsForTests.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/UtilsForTests.java
@@ -31,7 +31,6 @@
import java.util.Iterator;
import java.util.List;
import java.util.Properties;
-import java.util.concurrent.TimeoutException;
import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.conf.Configuration;
@@ -450,14 +449,11 @@
static void signalTasks(MiniDFSCluster dfs, FileSystem fileSys,
String mapSignalFile,
String reduceSignalFile, int replication)
- throws IOException, TimeoutException {
- try {
- writeFile(dfs.getNameNode(), fileSys.getConf(), new Path(mapSignalFile),
- (short)replication);
- writeFile(dfs.getNameNode(), fileSys.getConf(), new Path(reduceSignalFile), (short)replication);
- } catch (InterruptedException ie) {
- // Ignore
- }
+ throws IOException {
+ writeFile(dfs.getNameNode(), fileSys.getConf(), new Path(mapSignalFile),
+ (short)replication);
+ writeFile(dfs.getNameNode(), fileSys.getConf(), new Path(reduceSignalFile),
+ (short)replication);
}
/**
@@ -466,16 +462,12 @@
static void signalTasks(MiniDFSCluster dfs, FileSystem fileSys,
boolean isMap, String mapSignalFile,
String reduceSignalFile)
- throws IOException, TimeoutException {
- try {
- // signal the maps to complete
- writeFile(dfs.getNameNode(), fileSys.getConf(),
- isMap
- ? new Path(mapSignalFile)
- : new Path(reduceSignalFile), (short)1);
- } catch (InterruptedException ie) {
- // Ignore
- }
+ throws IOException {
+ // signal the maps to complete
+ writeFile(dfs.getNameNode(), fileSys.getConf(),
+ isMap
+ ? new Path(mapSignalFile)
+ : new Path(reduceSignalFile), (short)1);
}
static String getSignalFile(Path dir) {
@@ -491,8 +483,7 @@
}
static void writeFile(NameNode namenode, Configuration conf, Path name,
- short replication)
- throws IOException, TimeoutException, InterruptedException {
+ short replication) throws IOException {
FileSystem fileSys = FileSystem.get(conf);
SequenceFile.Writer writer =
SequenceFile.createWriter(fileSys, conf, name,
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestCounters.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestCounters.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestCounters.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestCounters.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestCombineFileInputFormat.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestCombineFileInputFormat.java
index 2aaa1ef..9621787 100644
--- a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestCombineFileInputFormat.java
+++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/input/TestCombineFileInputFormat.java
@@ -23,7 +23,6 @@
import java.util.List;
import java.util.ArrayList;
import java.util.zip.GZIPOutputStream;
-import java.util.concurrent.TimeoutException;
import junit.framework.TestCase;
@@ -279,7 +278,7 @@
assertFalse(rr.nextKeyValue());
}
- public void testSplitPlacement() throws Exception {
+ public void testSplitPlacement() throws IOException {
MiniDFSCluster dfs = null;
FileSystem fileSys = null;
try {
@@ -679,8 +678,7 @@
}
static void writeFile(Configuration conf, Path name,
- short replication, int numBlocks)
- throws IOException, TimeoutException, InterruptedException {
+ short replication, int numBlocks) throws IOException {
FileSystem fileSys = FileSystem.get(conf);
FSDataOutputStream stm = fileSys.create(name, true,
@@ -691,8 +689,7 @@
// Creates the gzip file and return the FileStatus
static FileStatus writeGzipFile(Configuration conf, Path name,
- short replication, int numBlocks)
- throws IOException, TimeoutException, InterruptedException {
+ short replication, int numBlocks) throws IOException {
FileSystem fileSys = FileSystem.get(conf);
GZIPOutputStream out = new GZIPOutputStream(fileSys.create(name, true, conf
@@ -702,8 +699,7 @@
}
private static void writeDataAndSetReplication(FileSystem fileSys, Path name,
- OutputStream out, short replication, int numBlocks)
- throws IOException, TimeoutException, InterruptedException {
+ OutputStream out, short replication, int numBlocks) throws IOException {
for (int i = 0; i < numBlocks; i++) {
out.write(databuf);
}
@@ -711,7 +707,7 @@
DFSTestUtil.waitReplication(fileSys, name, replication);
}
- public void testSplitPlacementForCompressedFiles() throws Exception {
+ public void testSplitPlacementForCompressedFiles() throws IOException {
MiniDFSCluster dfs = null;
FileSystem fileSys = null;
try {
@@ -1062,7 +1058,7 @@
/**
* Test that CFIF can handle missing blocks.
*/
- public void testMissingBlocks() throws Exception {
+ public void testMissingBlocks() throws IOException {
String namenode = null;
MiniDFSCluster dfs = null;
FileSystem fileSys = null;
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/jobcontrol/TestControlledJob.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/jobcontrol/TestControlledJob.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/jobcontrol/TestControlledJob.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/jobcontrol/TestControlledJob.java
diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/util/TestProcfsBasedProcessTree.java b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/util/TestProcfsBasedProcessTree.java
similarity index 100%
rename from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/util/TestProcfsBasedProcessTree.java
rename to hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/util/TestProcfsBasedProcessTree.java
diff --git a/hadoop-yarn-project/CHANGES.txt b/hadoop-yarn-project/CHANGES.txt
index 8beb1f5..b5cb33f 100644
--- a/hadoop-yarn-project/CHANGES.txt
+++ b/hadoop-yarn-project/CHANGES.txt
@@ -18,6 +18,9 @@
YARN-80. Add support for delaying rack-local containers in
CapacityScheduler. (acmurthy)
+ YARN-137. Change the default YARN scheduler to be the CapacityScheduler.
+ (sseth via acmurthy)
+
OPTIMIZATIONS
BUG FIXES
@@ -47,6 +50,24 @@
YARN-75. Modified ResourceManager's RMContainer to handle a valid RELEASE
event at RUNNING state. (Siddharth Seth via vinodkv)
+ MAPREDUCE-2374. "Text File Busy" errors launching MR tasks. (Andy Isaacson
+ via atm)
+
+ YARN-138. Ensure default values for minimum/maximum container sizes is
+ sane. (harsh & sseth via acmurthy)
+
+Release 0.23.5 - UNRELEASED
+
+ INCOMPATIBLE CHANGES
+
+ NEW FEATURES
+
+ IMPROVEMENTS
+
+ OPTIMIZATIONS
+
+ BUG FIXES
+
Release 0.23.4 - UNRELEASED
INCOMPATIBLE CHANGES
@@ -55,12 +76,21 @@
IMPROVEMENTS
- Change package of YarnClient to org.apache.hadoop. (Bikas Saha via vinodkv)
+ Change package of YarnClient to org.apache.hadoop. (Bikas Saha via vinodkv)
+
+ YARN-108. FSDownload can create cache directories with the wrong
+ permissions (Jason Lowe via bobby)
OPTIMIZATIONS
BUG FIXES
+ YARN-88. DefaultContainerExecutor can fail to set proper permissions.
+ (Jason Lowe via sseth)
+
+ YARN-106. Nodemanager needs to set permissions of local directories (jlowe
+ via bobby)
+
Release 0.23.3 - Unreleased
INCOMPATIBLE CHANGES
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LocalResource.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LocalResource.java
index 10dd23a..1a8790a 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LocalResource.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LocalResource.java
@@ -106,4 +106,20 @@
* localized
*/
public void setVisibility(LocalResourceVisibility visibility);
+
+ /**
+ * Get the <em>pattern</em> that should be used to extract entries from the
+ * archive (only used when type is <code>PATTERN</code>).
+ * @return <em>pattern</em> that should be used to extract entries from the
+ * archive.
+ */
+ public String getPattern();
+
+ /**
+ * Set the <em>pattern</em> that should be used to extract entries from the
+ * archive (only used when type is <code>PATTERN</code>).
+ * @param pattern <em>pattern</em> that should be used to extract entries
+ * from the archive.
+ */
+ public void setPattern(String pattern);
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LocalResourceType.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LocalResourceType.java
index 0cfed1c..2529f12 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LocalResourceType.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/LocalResourceType.java
@@ -55,5 +55,14 @@
/**
* Regular file i.e. uninterpreted bytes.
*/
- FILE
+ FILE,
+
+ /**
+ * A hybrid between archive and file. Only part of the file is unarchived,
+ * and the original file is left in place, but in the same directory as the
+ * unarchived part. The part that is unarchived is determined by pattern
+ * in #{@link LocalResource}. Currently only jars support pattern, all
+ * others will be treated like a #{@link LocalResourceType#ARCHIVE}.
+ */
+ PATTERN
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Priority.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Priority.java
index fea1f48..5c27f38 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Priority.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Priority.java
@@ -64,4 +64,8 @@
return this.getPriority() - other.getPriority();
}
+ @Override
+ public String toString() {
+ return "{Priority: " + getPriority() + "}";
+ }
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/LocalResourcePBImpl.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/LocalResourcePBImpl.java
index dd57f30..60ea0da 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/LocalResourcePBImpl.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/LocalResourcePBImpl.java
@@ -151,6 +151,25 @@
}
builder.setVisibility(convertToProtoFormat(visibility));
}
+
+ @Override
+ public synchronized String getPattern() {
+ LocalResourceProtoOrBuilder p = viaProto ? proto : builder;
+ if (!p.hasPattern()) {
+ return null;
+ }
+ return p.getPattern();
+ }
+
+ @Override
+ public synchronized void setPattern(String pattern) {
+ maybeInitBuilder();
+ if (pattern == null) {
+ builder.clearPattern();
+ return;
+ }
+ builder.setPattern(pattern);
+ }
private LocalResourceTypeProto convertToProtoFormat(LocalResourceType e) {
return ProtoUtils.convertToProtoFormat(e);
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourceRequestPBImpl.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourceRequestPBImpl.java
index f3b8ffa..f3834a4 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourceRequestPBImpl.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourceRequestPBImpl.java
@@ -162,5 +162,10 @@
private ResourceProto convertToProtoFormat(Resource t) {
return ((ResourcePBImpl)t).getProto();
}
-
-}
+
+ @Override
+ public String toString() {
+ return "{Priority: " + getPriority() + ", Capability: " + getCapability()
+ + "}";
+ }
+}
\ No newline at end of file
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
index 8daeddd..55c0d78 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
@@ -135,6 +135,7 @@
enum LocalResourceTypeProto {
ARCHIVE = 1;
FILE = 2;
+ PATTERN = 3;
}
message LocalResourceProto {
@@ -142,7 +143,8 @@
optional int64 size = 2;
optional int64 timestamp = 3;
optional LocalResourceTypeProto type = 4;
- optional LocalResourceVisibilityProto visibility= 5;
+ optional LocalResourceVisibilityProto visibility = 5;
+ optional string pattern = 6;
}
message ApplicationResourceUsageReportProto {
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
index e76e2db..29a053b 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java
@@ -31,6 +31,8 @@
import org.apache.hadoop.util.JarFinder;
import org.apache.hadoop.yarn.conf.YarnConfiguration;
import org.apache.hadoop.yarn.server.MiniYARNCluster;
+import org.apache.hadoop.yarn.server.resourcemanager.scheduler.ResourceScheduler;
+import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
@@ -41,7 +43,7 @@
LogFactory.getLog(TestDistributedShell.class);
protected static MiniYARNCluster yarnCluster = null;
- protected static Configuration conf = new Configuration();
+ protected static Configuration conf = new YarnConfiguration();
protected static String APPMASTER_JAR = JarFinder.getJar(ApplicationMaster.class);
@@ -49,6 +51,8 @@
public static void setup() throws InterruptedException, IOException {
LOG.info("Starting up YARN cluster");
conf.setInt(YarnConfiguration.RM_SCHEDULER_MINIMUM_ALLOCATION_MB, 128);
+ conf.setClass(YarnConfiguration.RM_SCHEDULER,
+ FifoScheduler.class, ResourceScheduler.class);
if (yarnCluster == null) {
yarnCluster = new MiniYARNCluster(TestDistributedShell.class.getName(),
1, 1, 1);
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
index fbe0b92..47cb8e2 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
@@ -114,18 +114,18 @@
/** Miniumum memory request grant-able by the RM scheduler. */
public static final String RM_SCHEDULER_MINIMUM_ALLOCATION_MB =
YARN_PREFIX + "scheduler.minimum-allocation-mb";
- public static final int DEFAULT_RM_SCHEDULER_MINIMUM_ALLOCATION_MB = 128;
+ public static final int DEFAULT_RM_SCHEDULER_MINIMUM_ALLOCATION_MB = 1024;
/** Maximum memory request grant-able by the RM scheduler. */
public static final String RM_SCHEDULER_MAXIMUM_ALLOCATION_MB =
YARN_PREFIX + "scheduler.maximum-allocation-mb";
- public static final int DEFAULT_RM_SCHEDULER_MAXIMUM_ALLOCATION_MB = 10240;
+ public static final int DEFAULT_RM_SCHEDULER_MAXIMUM_ALLOCATION_MB = 8192;
/** Number of threads to handle scheduler interface.*/
public static final String RM_SCHEDULER_CLIENT_THREAD_COUNT =
RM_PREFIX + "scheduler.client.thread-count";
public static final int DEFAULT_RM_SCHEDULER_CLIENT_THREAD_COUNT = 50;
-
+
/** The address of the RM web application.*/
public static final String RM_WEBAPP_ADDRESS =
RM_PREFIX + "webapp.address";
@@ -208,6 +208,8 @@
public static final String RM_SCHEDULER =
RM_PREFIX + "scheduler.class";
+ public static final String DEFAULT_RM_SCHEDULER =
+ "org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler";
//Delegation token related keys
public static final String DELEGATION_KEY_UPDATE_INTERVAL_KEY =
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
index e58f584..68f727d 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/security/ContainerTokenIdentifier.java
@@ -48,14 +48,16 @@
private ContainerId containerId;
private String nmHostAddr;
+ private String appSubmitter;
private Resource resource;
private long expiryTimeStamp;
private int masterKeyId;
public ContainerTokenIdentifier(ContainerId containerID, String hostName,
- Resource r, long expiryTimeStamp, int masterKeyId) {
+ String appSubmitter, Resource r, long expiryTimeStamp, int masterKeyId) {
this.containerId = containerID;
this.nmHostAddr = hostName;
+ this.appSubmitter = appSubmitter;
this.resource = r;
this.expiryTimeStamp = expiryTimeStamp;
this.masterKeyId = masterKeyId;
@@ -71,6 +73,10 @@
return this.containerId;
}
+ public String getApplicationSubmitter() {
+ return this.appSubmitter;
+ }
+
public String getNmHostAddress() {
return this.nmHostAddr;
}
@@ -98,6 +104,7 @@
out.writeInt(applicationAttemptId.getAttemptId());
out.writeInt(this.containerId.getId());
out.writeUTF(this.nmHostAddr);
+ out.writeUTF(this.appSubmitter);
out.writeInt(this.resource.getMemory());
out.writeLong(this.expiryTimeStamp);
out.writeInt(this.masterKeyId);
@@ -112,6 +119,7 @@
this.containerId = BuilderUtils.newContainerId(applicationAttemptId, in
.readInt());
this.nmHostAddr = in.readUTF();
+ this.appSubmitter = in.readUTF();
this.resource = BuilderUtils.newResource(in.readInt());
this.expiryTimeStamp = in.readLong();
this.masterKeyId = in.readInt();
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
index 24a23c8..4d69056 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java
@@ -25,6 +25,7 @@
import java.security.PrivilegedExceptionAction;
import java.util.Random;
import java.util.concurrent.Callable;
+import java.util.regex.Pattern;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
@@ -82,6 +83,13 @@
return resource;
}
+ private void createDir(Path path, FsPermission perm) throws IOException {
+ files.mkdir(path, perm, false);
+ if (!perm.equals(files.getUMask().applyUMask(perm))) {
+ files.setPermission(path, perm);
+ }
+ }
+
private Path copy(Path sCopy, Path dstdir) throws IOException {
FileSystem sourceFs = sCopy.getFileSystem(conf);
Path dCopy = new Path(dstdir, sCopy.getName() + ".tmp");
@@ -96,9 +104,9 @@
return dCopy;
}
- private long unpack(File localrsrc, File dst) throws IOException {
+ private long unpack(File localrsrc, File dst, Pattern pattern) throws IOException {
switch (resource.getType()) {
- case ARCHIVE:
+ case ARCHIVE: {
String lowerDst = dst.getName().toLowerCase();
if (lowerDst.endsWith(".jar")) {
RunJar.unJar(localrsrc, dst);
@@ -115,7 +123,39 @@
+ "] to [" + dst + "]");
}
}
- break;
+ }
+ break;
+ case PATTERN: {
+ String lowerDst = dst.getName().toLowerCase();
+ if (lowerDst.endsWith(".jar")) {
+ RunJar.unJar(localrsrc, dst, pattern);
+ File newDst = new File(dst, dst.getName());
+ if (!dst.exists() && !dst.mkdir()) {
+ throw new IOException("Unable to create directory: [" + dst + "]");
+ }
+ if (!localrsrc.renameTo(newDst)) {
+ throw new IOException("Unable to rename file: [" + localrsrc
+ + "] to [" + newDst + "]");
+ }
+ } else if (lowerDst.endsWith(".zip")) {
+ LOG.warn("Treating [" + localrsrc + "] as an archive even though it " +
+ "was specified as PATTERN");
+ FileUtil.unZip(localrsrc, dst);
+ } else if (lowerDst.endsWith(".tar.gz") ||
+ lowerDst.endsWith(".tgz") ||
+ lowerDst.endsWith(".tar")) {
+ LOG.warn("Treating [" + localrsrc + "] as an archive even though it " +
+ "was specified as PATTERN");
+ FileUtil.unTar(localrsrc, dst);
+ } else {
+ LOG.warn("Cannot unpack " + localrsrc);
+ if (!localrsrc.renameTo(dst)) {
+ throw new IOException("Unable to rename file: [" + localrsrc
+ + "] to [" + dst + "]");
+ }
+ }
+ }
+ break;
case FILE:
default:
if (!localrsrc.renameTo(dst)) {
@@ -144,9 +184,9 @@
} while (files.util().exists(tmp));
destDirPath = tmp;
- files.mkdir(destDirPath, cachePerms, false);
+ createDir(destDirPath, cachePerms);
final Path dst_work = new Path(destDirPath + "_tmp");
- files.mkdir(dst_work, cachePerms, false);
+ createDir(dst_work, cachePerms);
Path dFinal = files.makeQualified(new Path(dst_work, sCopy.getName()));
try {
@@ -156,8 +196,13 @@
public Path run() throws Exception {
return files.makeQualified(copy(sCopy, dst_work));
};
- });
- unpack(new File(dTmp.toUri()), new File(dFinal.toUri()));
+ });
+ Pattern pattern = null;
+ String p = resource.getPattern();
+ if(p != null) {
+ pattern = Pattern.compile(p);
+ }
+ unpack(new File(dTmp.toUri()), new File(dFinal.toUri()), pattern);
changePermissions(dFinal.getFileSystem(conf), dFinal);
files.rename(dst_work, destDirPath, Rename.OVERWRITE);
} catch (Exception e) {
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
index b2ee6119..809a70b 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
@@ -185,6 +185,7 @@
<property>
<description>The class to use as the resource scheduler.</description>
<name>yarn.resourcemanager.scheduler.class</name>
+ <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
</property>
<property>
@@ -192,7 +193,7 @@
in MBs. Memory requests lower than this won't take effect,
and the specified value will get allocated at minimum.</description>
<name>yarn.scheduler.minimum-allocation-mb</name>
- <value>128</value>
+ <value>1024</value>
</property>
<property>
@@ -200,7 +201,7 @@
in MBs. Memory requests higher than this won't take effect,
and will get capped to this value.</description>
<name>yarn.scheduler.maximum-allocation-mb</name>
- <value>10240</value>
+ <value>8192</value>
</property>
<property>
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
index fe1f3ac..25adf31 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestFSDownload.java
@@ -42,6 +42,7 @@
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.CommonConfigurationKeys;
import org.apache.hadoop.fs.FSDataOutputStream;
import org.apache.hadoop.fs.FileContext;
import org.apache.hadoop.fs.FileStatus;
@@ -106,8 +107,9 @@
FileStatus status = files.getFileStatus(p);
ret.setSize(status.getLen());
ret.setTimestamp(status.getModificationTime());
- ret.setType(LocalResourceType.ARCHIVE);
+ ret.setType(LocalResourceType.PATTERN);
ret.setVisibility(vis);
+ ret.setPattern("classes/.*");
return ret;
}
@@ -115,6 +117,7 @@
public void testDownload() throws IOException, URISyntaxException,
InterruptedException {
Configuration conf = new Configuration();
+ conf.set(CommonConfigurationKeys.FS_PERMISSIONS_UMASK_KEY, "077");
FileContext files = FileContext.getLocalFSFileContext(conf);
final Path basedir = files.makeQualified(new Path("target",
TestFSDownload.class.getSimpleName()));
@@ -162,8 +165,14 @@
Path localized = p.getValue().get();
assertEquals(sizes[Integer.valueOf(localized.getName())], p.getKey()
.getSize());
- FileStatus status = files.getFileStatus(localized);
+
+ FileStatus status = files.getFileStatus(localized.getParent());
FsPermission perm = status.getPermission();
+ assertEquals("Cache directory permissions are incorrect",
+ new FsPermission((short)0755), perm);
+
+ status = files.getFileStatus(localized);
+ perm = status.getPermission();
System.out.println("File permission " + perm +
" for rsrc vis " + p.getKey().getVisibility().name());
assert(rsrcVis.containsKey(p.getKey()));
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/BaseContainerTokenSecretManager.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/BaseContainerTokenSecretManager.java
index 16f4b6f..ade32b4 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/BaseContainerTokenSecretManager.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/BaseContainerTokenSecretManager.java
@@ -128,7 +128,8 @@
public byte[] createPassword(ContainerTokenIdentifier identifier) {
if (LOG.isDebugEnabled()) {
LOG.debug("Creating password for " + identifier.getContainerID()
- + " to be run on NM " + identifier.getNmHostAddress());
+ + " for user " + identifier.getUser() + " to be run on NM "
+ + identifier.getNmHostAddress());
}
this.readLock.lock();
try {
@@ -155,7 +156,8 @@
throws org.apache.hadoop.security.token.SecretManager.InvalidToken {
if (LOG.isDebugEnabled()) {
LOG.debug("Retrieving password for " + identifier.getContainerID()
- + " to be run on NM " + identifier.getNmHostAddress());
+ + " for user " + identifier.getUser() + " to be run on NM "
+ + identifier.getNmHostAddress());
}
return createPassword(identifier.getBytes(), masterKey.getSecretKey());
}
@@ -173,11 +175,12 @@
*
* @param containerId
* @param nodeId
+ * @param appSubmitter
* @param capability
* @return the container-token
*/
public ContainerToken createContainerToken(ContainerId containerId,
- NodeId nodeId, Resource capability) {
+ NodeId nodeId, String appSubmitter, Resource capability) {
byte[] password;
ContainerTokenIdentifier tokenIdentifier;
long expiryTimeStamp =
@@ -188,8 +191,8 @@
try {
tokenIdentifier =
new ContainerTokenIdentifier(containerId, nodeId.toString(),
- capability, expiryTimeStamp, this.currentMasterKey.getMasterKey()
- .getKeyId());
+ appSubmitter, capability, expiryTimeStamp, this.currentMasterKey
+ .getMasterKey().getKeyId());
password = this.createPassword(tokenIdentifier);
} finally {
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
index 9a4b8a0..428e24d 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
@@ -109,6 +109,7 @@
String userName, String appId, Path containerWorkDir,
List<String> localDirs, List<String> logDirs) throws IOException {
+ FsPermission dirPerm = new FsPermission(APPDIR_PERM);
ContainerId containerId = container.getContainerID();
// create container dirs on all disks
@@ -123,7 +124,7 @@
Path appCacheDir = new Path(userdir, ContainerLocalizer.APPCACHE);
Path appDir = new Path(appCacheDir, appIdStr);
Path containerDir = new Path(appDir, containerIdStr);
- lfs.mkdir(containerDir, null, false);
+ createDir(containerDir, dirPerm, false);
}
// Create the container log-dirs on all disks
@@ -131,7 +132,7 @@
Path tmpDir = new Path(containerWorkDir,
YarnConfiguration.DEFAULT_CONTAINER_TEMP_DIR);
- lfs.mkdir(tmpDir, null, false);
+ createDir(tmpDir, dirPerm, false);
// copy launch script to work dir
Path launchDst =
@@ -286,20 +287,20 @@
}
/** Permissions for user dir.
- * $loaal.dir/usercache/$user */
- private static final short USER_PERM = (short)0750;
+ * $local.dir/usercache/$user */
+ static final short USER_PERM = (short)0750;
/** Permissions for user appcache dir.
- * $loaal.dir/usercache/$user/appcache */
- private static final short APPCACHE_PERM = (short)0710;
+ * $local.dir/usercache/$user/appcache */
+ static final short APPCACHE_PERM = (short)0710;
/** Permissions for user filecache dir.
- * $loaal.dir/usercache/$user/filecache */
- private static final short FILECACHE_PERM = (short)0710;
+ * $local.dir/usercache/$user/filecache */
+ static final short FILECACHE_PERM = (short)0710;
/** Permissions for user app dir.
- * $loaal.dir/usercache/$user/filecache */
- private static final short APPDIR_PERM = (short)0710;
+ * $local.dir/usercache/$user/appcache/$appId */
+ static final short APPDIR_PERM = (short)0710;
/** Permissions for user log dir.
* $logdir/$user/$appId */
- private static final short LOGDIR_PERM = (short)0710;
+ static final short LOGDIR_PERM = (short)0710;
private Path getFirstApplicationDir(List<String> localDirs, String user,
String appId) {
@@ -324,20 +325,28 @@
ContainerLocalizer.FILECACHE);
}
+ private void createDir(Path dirPath, FsPermission perms,
+ boolean createParent) throws IOException {
+ lfs.mkdir(dirPath, perms, createParent);
+ if (!perms.equals(perms.applyUMask(lfs.getUMask()))) {
+ lfs.setPermission(dirPath, perms);
+ }
+ }
+
/**
* Initialize the local directories for a particular user.
- * <ul>
+ * <ul>.mkdir
* <li>$local.dir/usercache/$user</li>
* </ul>
*/
- private void createUserLocalDirs(List<String> localDirs, String user)
+ void createUserLocalDirs(List<String> localDirs, String user)
throws IOException {
boolean userDirStatus = false;
FsPermission userperms = new FsPermission(USER_PERM);
for (String localDir : localDirs) {
// create $local.dir/usercache/$user and its immediate parent
try {
- lfs.mkdir(getUserCacheDir(new Path(localDir), user), userperms, true);
+ createDir(getUserCacheDir(new Path(localDir), user), userperms, true);
} catch (IOException e) {
LOG.warn("Unable to create the user directory : " + localDir, e);
continue;
@@ -359,7 +368,7 @@
* <li>$local.dir/usercache/$user/filecache</li>
* </ul>
*/
- private void createUserCacheDirs(List<String> localDirs, String user)
+ void createUserCacheDirs(List<String> localDirs, String user)
throws IOException {
LOG.info("Initializing user " + user);
@@ -373,7 +382,7 @@
Path localDirPath = new Path(localDir);
final Path appDir = getAppcacheDir(localDirPath, user);
try {
- lfs.mkdir(appDir, appCachePerms, true);
+ createDir(appDir, appCachePerms, true);
appcacheDirStatus = true;
} catch (IOException e) {
LOG.warn("Unable to create app cache directory : " + appDir, e);
@@ -381,7 +390,7 @@
// create $local.dir/usercache/$user/filecache
final Path distDir = getFileCacheDir(localDirPath, user);
try {
- lfs.mkdir(distDir, fileperms, true);
+ createDir(distDir, fileperms, true);
distributedCacheDirStatus = true;
} catch (IOException e) {
LOG.warn("Unable to create file cache directory : " + distDir, e);
@@ -406,7 +415,7 @@
* </ul>
* @param localDirs
*/
- private void createAppDirs(List<String> localDirs, String user, String appId)
+ void createAppDirs(List<String> localDirs, String user, String appId)
throws IOException {
boolean initAppDirStatus = false;
FsPermission appperms = new FsPermission(APPDIR_PERM);
@@ -414,7 +423,7 @@
Path fullAppDir = getApplicationDir(new Path(localDir), user, appId);
// create $local.dir/usercache/$user/appcache/$appId
try {
- lfs.mkdir(fullAppDir, appperms, true);
+ createDir(fullAppDir, appperms, true);
initAppDirStatus = true;
} catch (IOException e) {
LOG.warn("Unable to create app directory " + fullAppDir.toString(), e);
@@ -430,7 +439,7 @@
/**
* Create application log directories on all disks.
*/
- private void createAppLogDirs(String appId, List<String> logDirs)
+ void createAppLogDirs(String appId, List<String> logDirs)
throws IOException {
boolean appLogDirStatus = false;
@@ -439,7 +448,7 @@
// create $log.dir/$appid
Path appLogDir = new Path(rootLogDir, appId);
try {
- lfs.mkdir(appLogDir, appLogDirPerms, true);
+ createDir(appLogDir, appLogDirPerms, true);
} catch (IOException e) {
LOG.warn("Unable to create the app-log directory : " + appLogDir, e);
continue;
@@ -455,7 +464,7 @@
/**
* Create application log directories on all disks.
*/
- private void createContainerLogDirs(String appId, String containerId,
+ void createContainerLogDirs(String appId, String containerId,
List<String> logDirs) throws IOException {
boolean containerLogDirStatus = false;
@@ -465,7 +474,7 @@
Path appLogDir = new Path(rootLogDir, appId);
Path containerLogDir = new Path(appLogDir, containerId);
try {
- lfs.mkdir(containerLogDir, containerLogDirPerms, true);
+ createDir(containerLogDir, containerLogDirPerms, true);
} catch (IOException e) {
LOG.warn("Unable to create the container-log directory : "
+ appLogDir, e);
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DirectoryCollection.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DirectoryCollection.java
index 9971567..10362d2 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DirectoryCollection.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DirectoryCollection.java
@@ -19,12 +19,17 @@
package org.apache.hadoop.yarn.server.nodemanager;
import java.io.File;
-import java.util.concurrent.CopyOnWriteArrayList;
+import java.io.FileNotFoundException;
+import java.io.IOException;
import java.util.Collections;
import java.util.List;
+import java.util.concurrent.CopyOnWriteArrayList;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
+import org.apache.hadoop.fs.FileContext;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.permission.FsPermission;
import org.apache.hadoop.util.DiskChecker;
import org.apache.hadoop.util.DiskChecker.DiskErrorException;
@@ -66,6 +71,31 @@
}
/**
+ * Create any non-existent directories and parent directories, updating the
+ * list of valid directories if necessary.
+ * @param localFs local file system to use
+ * @param perm absolute permissions to use for any directories created
+ * @return true if there were no errors, false if at least one error occurred
+ */
+ synchronized boolean createNonExistentDirs(FileContext localFs,
+ FsPermission perm) {
+ boolean failed = false;
+ for (final String dir : localDirs) {
+ try {
+ createDir(localFs, new Path(dir), perm);
+ } catch (IOException e) {
+ LOG.warn("Unable to create directory " + dir + " error " +
+ e.getMessage() + ", removing from the list of valid directories.");
+ localDirs.remove(dir);
+ failedDirs.add(dir);
+ numFailures++;
+ failed = true;
+ }
+ }
+ return !failed;
+ }
+
+ /**
* Check the health of current set of local directories, updating the list
* of valid directories if necessary.
* @return <em>true</em> if there is a new disk-failure identified in
@@ -86,4 +116,20 @@
}
return numFailures > oldNumFailures;
}
+
+ private void createDir(FileContext localFs, Path dir, FsPermission perm)
+ throws IOException {
+ if (dir == null) {
+ return;
+ }
+ try {
+ localFs.getFileStatus(dir);
+ } catch (FileNotFoundException e) {
+ createDir(localFs, dir.getParent(), perm);
+ localFs.mkdir(dir, perm, false);
+ if (!perm.equals(perm.applyUMask(localFs.getUMask()))) {
+ localFs.setPermission(dir, perm);
+ }
+ }
+ }
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LocalDirsHandlerService.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LocalDirsHandlerService.java
index 4e07b70..0c7e01d 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LocalDirsHandlerService.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LocalDirsHandlerService.java
@@ -26,9 +26,12 @@
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileContext;
import org.apache.hadoop.fs.LocalDirAllocator;
import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.permission.FsPermission;
import org.apache.hadoop.util.StringUtils;
+import org.apache.hadoop.yarn.YarnException;
import org.apache.hadoop.yarn.conf.YarnConfiguration;
import org.apache.hadoop.yarn.service.AbstractService;
@@ -120,6 +123,19 @@
lastDisksCheckTime = System.currentTimeMillis();
super.init(conf);
+ FileContext localFs;
+ try {
+ localFs = FileContext.getLocalFSFileContext(config);
+ } catch (IOException e) {
+ throw new YarnException("Unable to get the local filesystem", e);
+ }
+ FsPermission perm = new FsPermission((short)0755);
+ boolean createSucceeded = localDirs.createNonExistentDirs(localFs, perm);
+ createSucceeded &= logDirs.createNonExistentDirs(localFs, perm);
+ if (!createSucceeded) {
+ updateDirsAfterFailure();
+ }
+
// Check the disk health immediately to weed out bad directories
// before other init code attempts to use them.
checkDirs();
@@ -229,7 +245,8 @@
* Set good local dirs and good log dirs in the configuration so that the
* LocalDirAllocator objects will use this updated configuration only.
*/
- private void updateDirsInConfiguration() {
+ private void updateDirsAfterFailure() {
+ LOG.info("Disk(s) failed. " + getDisksHealthReport());
Configuration conf = getConfig();
List<String> localDirs = getLocalDirs();
conf.setStrings(YarnConfiguration.NM_LOCAL_DIRS,
@@ -237,6 +254,10 @@
List<String> logDirs = getLogDirs();
conf.setStrings(YarnConfiguration.NM_LOG_DIRS,
logDirs.toArray(new String[logDirs.size()]));
+ if (!areDisksHealthy()) {
+ // Just log.
+ LOG.error("Most of the disks failed. " + getDisksHealthReport());
+ }
}
private void checkDirs() {
@@ -249,12 +270,7 @@
}
if (newFailure) {
- LOG.info("Disk(s) failed. " + getDisksHealthReport());
- updateDirsInConfiguration();
- if (!areDisksHealthy()) {
- // Just log.
- LOG.error("Most of the disks failed. " + getDisksHealthReport());
- }
+ updateDirsAfterFailure();
}
lastDisksCheckTime = System.currentTimeMillis();
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
index f9650fb..7ca6a2c 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
@@ -329,7 +329,6 @@
+ remoteUgi.getTokenIdentifiers().size());
}
-
// Get the tokenId from the remote user ugi
ContainerTokenIdentifier tokenId =
selectContainerTokenIdentifier(remoteUgi);
@@ -341,8 +340,16 @@
+ containerIDStr);
} else {
+ // Is the container coming in with correct user-name?
+ if (!tokenId.getApplicationSubmitter().equals(launchContext.getUser())) {
+ unauthorized = true;
+ messageBuilder.append("\n Expected user-name "
+ + tokenId.getApplicationSubmitter() + " but found "
+ + launchContext.getUser());
+ }
+
// Is the container being relaunched? Or RPC layer let startCall with
- // tokens generated off old-secret through
+ // tokens generated off old-secret through?
if (!this.context.getContainerTokenSecretManager()
.isValidStartContainerRequest(tokenId)) {
unauthorized = true;
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
index 65bcfbd..74d0227 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
@@ -209,6 +209,7 @@
}
switch (rsrc.getType()) {
case ARCHIVE:
+ case PATTERN:
return 5 * rsrc.getSize();
case FILE:
default:
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java
index 7754baa..3d0e0fd 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalResourceRequest.java
@@ -34,6 +34,7 @@
private final long timestamp;
private final LocalResourceType type;
private final LocalResourceVisibility visibility;
+ private final String pattern;
/**
* Wrap API resource to match against cache of localized resources.
@@ -45,22 +46,28 @@
this(ConverterUtils.getPathFromYarnURL(resource.getResource()),
resource.getTimestamp(),
resource.getType(),
- resource.getVisibility());
+ resource.getVisibility(),
+ resource.getPattern());
}
LocalResourceRequest(Path loc, long timestamp, LocalResourceType type,
- LocalResourceVisibility visibility) {
+ LocalResourceVisibility visibility, String pattern) {
this.loc = loc;
this.timestamp = timestamp;
this.type = type;
this.visibility = visibility;
+ this.pattern = pattern;
}
@Override
public int hashCode() {
- return loc.hashCode() ^
+ int hash = loc.hashCode() ^
(int)((timestamp >>> 32) ^ timestamp) *
type.hashCode();
+ if(pattern != null) {
+ hash = hash ^ pattern.hashCode();
+ }
+ return hash;
}
@Override
@@ -72,9 +79,14 @@
return false;
}
final LocalResourceRequest other = (LocalResourceRequest) o;
+ String pattern = getPattern();
+ String otherPattern = other.getPattern();
+ boolean patternEquals = (pattern == null && otherPattern == null) ||
+ (pattern != null && otherPattern != null && pattern.equals(otherPattern));
return getPath().equals(other.getPath()) &&
getTimestamp() == other.getTimestamp() &&
- getType() == other.getType();
+ getType() == other.getType() &&
+ patternEquals;
}
@Override
@@ -87,6 +99,19 @@
ret = (int)(getTimestamp() - other.getTimestamp());
if (0 == ret) {
ret = getType().ordinal() - other.getType().ordinal();
+ if (0 == ret) {
+ String pattern = getPattern();
+ String otherPattern = other.getPattern();
+ if (pattern == null && otherPattern == null) {
+ ret = 0;
+ } else if (pattern == null) {
+ ret = -1;
+ } else if (otherPattern == null) {
+ ret = 1;
+ } else {
+ ret = pattern.compareTo(otherPattern);
+ }
+ }
}
}
return ret;
@@ -122,6 +147,11 @@
}
@Override
+ public String getPattern() {
+ return pattern;
+ }
+
+ @Override
public void setResource(URL resource) {
throw new UnsupportedOperationException();
}
@@ -145,14 +175,20 @@
public void setVisibility(LocalResourceVisibility visibility) {
throw new UnsupportedOperationException();
}
-
+
+ @Override
+ public void setPattern(String pattern) {
+ throw new UnsupportedOperationException();
+ }
+
@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("{ ");
sb.append(getPath().toString()).append(", ");
sb.append(getTimestamp()).append(", ");
- sb.append(getType()).append(" }");
+ sb.append(getType()).append(", ");
+ sb.append(getPattern()).append(" }");
return sb.toString();
}
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalizedResource.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalizedResource.java
index bee9c2d..00709fd 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalizedResource.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/LocalizedResource.java
@@ -219,7 +219,8 @@
ContainerId container = ctxt.getContainerId();
rsrc.ref.add(container);
rsrc.dispatcher.getEventHandler().handle(
- new LocalizerResourceRequestEvent(rsrc, req.getVisibility(), ctxt));
+ new LocalizerResourceRequestEvent(rsrc, req.getVisibility(), ctxt,
+ req.getLocalResourceRequest().getPattern()));
}
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
index 71ad968..c21ef51 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
@@ -752,6 +752,7 @@
next.setTimestamp(nextRsrc.getTimestamp());
next.setType(nextRsrc.getType());
next.setVisibility(evt.getVisibility());
+ next.setPattern(evt.getPattern());
scheduled.put(nextRsrc, evt);
return next;
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizerResourceRequestEvent.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizerResourceRequestEvent.java
index f316871..2e05dd7 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizerResourceRequestEvent.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizerResourceRequestEvent.java
@@ -32,14 +32,16 @@
private final LocalizerContext context;
private final LocalizedResource resource;
private final LocalResourceVisibility vis;
+ private final String pattern;
public LocalizerResourceRequestEvent(LocalizedResource resource,
- LocalResourceVisibility vis, LocalizerContext context) {
+ LocalResourceVisibility vis, LocalizerContext context, String pattern) {
super(LocalizerEventType.REQUEST_RESOURCE_LOCALIZATION,
ConverterUtils.toString(context.getContainerId()));
this.vis = vis;
this.context = context;
this.resource = resource;
+ this.pattern = pattern;
}
public LocalizedResource getResource() {
@@ -54,4 +56,8 @@
return vis;
}
+ public String getPattern() {
+ return pattern;
+ }
+
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDefaultContainerExecutor.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDefaultContainerExecutor.java
index 5af7b43..e67fc3e 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDefaultContainerExecutor.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDefaultContainerExecutor.java
@@ -18,6 +18,7 @@
package org.apache.hadoop.yarn.server.nodemanager;
+import java.io.FileNotFoundException;
import java.io.InputStream;
import java.io.IOException;
import java.net.InetSocketAddress;
@@ -26,8 +27,11 @@
import java.util.List;
import java.util.Random;
+import junit.framework.Assert;
+
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.AbstractFileSystem;
+import org.apache.hadoop.fs.CommonConfigurationKeys;
import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hadoop.fs.FSDataOutputStream;
import org.apache.hadoop.fs.FileContext;
@@ -38,6 +42,7 @@
import org.apache.hadoop.io.DataInputBuffer;
import org.apache.hadoop.io.DataOutputBuffer;
import org.apache.hadoop.util.Progressable;
+import org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer;
import org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.FakeFSDataInputStream;
import static org.apache.hadoop.fs.CreateFlag.*;
@@ -91,11 +96,16 @@
}
*/
+ private static final Path BASE_TMP_PATH = new Path("target",
+ TestDefaultContainerExecutor.class.getSimpleName());
+
@AfterClass
public static void deleteTmpFiles() throws IOException {
FileContext lfs = FileContext.getLocalFSFileContext();
- lfs.delete(new Path("target",
- TestDefaultContainerExecutor.class.getSimpleName()), true);
+ try {
+ lfs.delete(BASE_TMP_PATH, true);
+ } catch (FileNotFoundException e) {
+ }
}
byte[] createTmpFile(Path dst, Random r, int len)
@@ -116,6 +126,71 @@
return bytes;
}
+ @Test
+ public void testDirPermissions() throws Exception {
+ deleteTmpFiles();
+
+ final String user = "somebody";
+ final String appId = "app_12345_123";
+ final FsPermission userCachePerm = new FsPermission(
+ DefaultContainerExecutor.USER_PERM);
+ final FsPermission appCachePerm = new FsPermission(
+ DefaultContainerExecutor.APPCACHE_PERM);
+ final FsPermission fileCachePerm = new FsPermission(
+ DefaultContainerExecutor.FILECACHE_PERM);
+ final FsPermission appDirPerm = new FsPermission(
+ DefaultContainerExecutor.APPDIR_PERM);
+ final FsPermission logDirPerm = new FsPermission(
+ DefaultContainerExecutor.LOGDIR_PERM);
+ List<String> localDirs = new ArrayList<String>();
+ localDirs.add(new Path(BASE_TMP_PATH, "localDirA").toString());
+ localDirs.add(new Path(BASE_TMP_PATH, "localDirB").toString());
+ List<String> logDirs = new ArrayList<String>();
+ logDirs.add(new Path(BASE_TMP_PATH, "logDirA").toString());
+ logDirs.add(new Path(BASE_TMP_PATH, "logDirB").toString());
+
+ Configuration conf = new Configuration();
+ conf.set(CommonConfigurationKeys.FS_PERMISSIONS_UMASK_KEY, "077");
+ FileContext lfs = FileContext.getLocalFSFileContext(conf);
+ DefaultContainerExecutor executor = new DefaultContainerExecutor(lfs);
+ executor.init();
+
+ try {
+ executor.createUserLocalDirs(localDirs, user);
+ executor.createUserCacheDirs(localDirs, user);
+ executor.createAppDirs(localDirs, user, appId);
+
+ for (String dir : localDirs) {
+ FileStatus stats = lfs.getFileStatus(
+ new Path(new Path(dir, ContainerLocalizer.USERCACHE), user));
+ Assert.assertEquals(userCachePerm, stats.getPermission());
+ }
+
+ for (String dir : localDirs) {
+ Path userCachePath = new Path(
+ new Path(dir, ContainerLocalizer.USERCACHE), user);
+ Path appCachePath = new Path(userCachePath,
+ ContainerLocalizer.APPCACHE);
+ FileStatus stats = lfs.getFileStatus(appCachePath);
+ Assert.assertEquals(appCachePerm, stats.getPermission());
+ stats = lfs.getFileStatus(
+ new Path(userCachePath, ContainerLocalizer.FILECACHE));
+ Assert.assertEquals(fileCachePerm, stats.getPermission());
+ stats = lfs.getFileStatus(new Path(appCachePath, appId));
+ Assert.assertEquals(appDirPerm, stats.getPermission());
+ }
+
+ executor.createAppLogDirs(appId, logDirs);
+
+ for (String dir : logDirs) {
+ FileStatus stats = lfs.getFileStatus(new Path(dir, appId));
+ Assert.assertEquals(logDirPerm, stats.getPermission());
+ }
+ } finally {
+ deleteTmpFiles();
+ }
+ }
+
// @Test
// public void testInit() throws IOException, InterruptedException {
// Configuration conf = new Configuration();
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDirectoryCollection.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDirectoryCollection.java
index 9f6fcf7..4ab61c9 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDirectoryCollection.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDirectoryCollection.java
@@ -23,7 +23,13 @@
import java.util.List;
import java.util.ListIterator;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.CommonConfigurationKeys;
+import org.apache.hadoop.fs.FileContext;
+import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileUtil;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.permission.FsPermission;
import org.junit.AfterClass;
import org.junit.Assert;
import org.junit.BeforeClass;
@@ -65,4 +71,37 @@
// Verify no ConcurrentModification is thrown
li.next();
}
+
+ @Test
+ public void testCreateDirectories() throws IOException {
+ Configuration conf = new Configuration();
+ conf.set(CommonConfigurationKeys.FS_PERMISSIONS_UMASK_KEY, "077");
+ FileContext localFs = FileContext.getLocalFSFileContext(conf);
+
+ String dirA = new File(testDir, "dirA").getPath();
+ String dirB = new File(dirA, "dirB").getPath();
+ String dirC = new File(testDir, "dirC").getPath();
+ Path pathC = new Path(dirC);
+ FsPermission permDirC = new FsPermission((short)0710);
+
+ localFs.mkdir(pathC, null, true);
+ localFs.setPermission(pathC, permDirC);
+
+ String[] dirs = { dirA, dirB, dirC };
+ DirectoryCollection dc = new DirectoryCollection(dirs);
+ FsPermission defaultPerm = FsPermission.getDefault()
+ .applyUMask(new FsPermission((short)FsPermission.DEFAULT_UMASK));
+ boolean createResult = dc.createNonExistentDirs(localFs, defaultPerm);
+ Assert.assertTrue(createResult);
+
+ FileStatus status = localFs.getFileStatus(new Path(dirA));
+ Assert.assertEquals("local dir parent not created with proper permissions",
+ defaultPerm, status.getPermission());
+ status = localFs.getFileStatus(new Path(dirB));
+ Assert.assertEquals("local dir not created with proper permissions",
+ defaultPerm, status.getPermission());
+ status = localFs.getFileStatus(pathC);
+ Assert.assertEquals("existing local directory permissions modified",
+ permDirC, status.getPermission());
+ }
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResource.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResource.java
index edfe7d7..81446f5 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResource.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResource.java
@@ -37,7 +37,7 @@
public class TestLocalResource {
static org.apache.hadoop.yarn.api.records.LocalResource getYarnResource(Path p, long size,
- long timestamp, LocalResourceType type, LocalResourceVisibility state)
+ long timestamp, LocalResourceType type, LocalResourceVisibility state, String pattern)
throws URISyntaxException {
org.apache.hadoop.yarn.api.records.LocalResource ret = RecordFactoryProvider.getRecordFactory(null).newRecordInstance(org.apache.hadoop.yarn.api.records.LocalResource.class);
ret.setResource(ConverterUtils.getYarnUrlFromURI(p.toUri()));
@@ -45,6 +45,7 @@
ret.setTimestamp(timestamp);
ret.setType(type);
ret.setVisibility(state);
+ ret.setPattern(pattern);
return ret;
}
@@ -72,9 +73,9 @@
long basetime = r.nextLong() >>> 2;
org.apache.hadoop.yarn.api.records.LocalResource yA = getYarnResource(
- new Path("http://yak.org:80/foobar"), -1, basetime, FILE, PUBLIC);
+ new Path("http://yak.org:80/foobar"), -1, basetime, FILE, PUBLIC, null);
org.apache.hadoop.yarn.api.records.LocalResource yB = getYarnResource(
- new Path("http://yak.org:80/foobar"), -1, basetime, FILE, PUBLIC);
+ new Path("http://yak.org:80/foobar"), -1, basetime, FILE, PUBLIC, null);
final LocalResourceRequest a = new LocalResourceRequest(yA);
LocalResourceRequest b = new LocalResourceRequest(yA);
checkEqual(a, b);
@@ -83,31 +84,37 @@
// ignore visibility
yB = getYarnResource(
- new Path("http://yak.org:80/foobar"), -1, basetime, FILE, PRIVATE);
+ new Path("http://yak.org:80/foobar"), -1, basetime, FILE, PRIVATE, null);
b = new LocalResourceRequest(yB);
checkEqual(a, b);
// ignore size
yB = getYarnResource(
- new Path("http://yak.org:80/foobar"), 0, basetime, FILE, PRIVATE);
+ new Path("http://yak.org:80/foobar"), 0, basetime, FILE, PRIVATE, null);
b = new LocalResourceRequest(yB);
checkEqual(a, b);
// note path
yB = getYarnResource(
- new Path("hdfs://dingo.org:80/foobar"), 0, basetime, ARCHIVE, PUBLIC);
+ new Path("hdfs://dingo.org:80/foobar"), 0, basetime, ARCHIVE, PUBLIC, null);
b = new LocalResourceRequest(yB);
checkNotEqual(a, b);
// note type
yB = getYarnResource(
- new Path("http://yak.org:80/foobar"), 0, basetime, ARCHIVE, PUBLIC);
+ new Path("http://yak.org:80/foobar"), 0, basetime, ARCHIVE, PUBLIC, null);
b = new LocalResourceRequest(yB);
checkNotEqual(a, b);
// note timestamp
yB = getYarnResource(
- new Path("http://yak.org:80/foobar"), 0, basetime + 1, FILE, PUBLIC);
+ new Path("http://yak.org:80/foobar"), 0, basetime + 1, FILE, PUBLIC, null);
+ b = new LocalResourceRequest(yB);
+ checkNotEqual(a, b);
+
+ // note pattern
+ yB = getYarnResource(
+ new Path("http://yak.org:80/foobar"), 0, basetime + 1, FILE, PUBLIC, "^/foo/.*");
b = new LocalResourceRequest(yB);
checkNotEqual(a, b);
}
@@ -120,24 +127,35 @@
System.out.println("SEED: " + seed);
long basetime = r.nextLong() >>> 2;
org.apache.hadoop.yarn.api.records.LocalResource yA = getYarnResource(
- new Path("http://yak.org:80/foobar"), -1, basetime, FILE, PUBLIC);
+ new Path("http://yak.org:80/foobar"), -1, basetime, FILE, PUBLIC, "^/foo/.*");
final LocalResourceRequest a = new LocalResourceRequest(yA);
// Path primary
org.apache.hadoop.yarn.api.records.LocalResource yB = getYarnResource(
- new Path("http://yak.org:80/foobaz"), -1, basetime, FILE, PUBLIC);
+ new Path("http://yak.org:80/foobaz"), -1, basetime, FILE, PUBLIC, "^/foo/.*");
LocalResourceRequest b = new LocalResourceRequest(yB);
assertTrue(0 > a.compareTo(b));
// timestamp secondary
yB = getYarnResource(
- new Path("http://yak.org:80/foobar"), -1, basetime + 1, FILE, PUBLIC);
+ new Path("http://yak.org:80/foobar"), -1, basetime + 1, FILE, PUBLIC, "^/foo/.*");
b = new LocalResourceRequest(yB);
assertTrue(0 > a.compareTo(b));
// type tertiary
yB = getYarnResource(
- new Path("http://yak.org:80/foobar"), -1, basetime, ARCHIVE, PUBLIC);
+ new Path("http://yak.org:80/foobar"), -1, basetime, ARCHIVE, PUBLIC, "^/foo/.*");
+ b = new LocalResourceRequest(yB);
+ assertTrue(0 != a.compareTo(b)); // don't care about order, just ne
+
+ // path 4th
+ yB = getYarnResource(
+ new Path("http://yak.org:80/foobar"), -1, basetime, ARCHIVE, PUBLIC, "^/food/.*");
+ b = new LocalResourceRequest(yB);
+ assertTrue(0 != a.compareTo(b)); // don't care about order, just ne
+
+ yB = getYarnResource(
+ new Path("http://yak.org:80/foobar"), -1, basetime, ARCHIVE, PUBLIC, null);
b = new LocalResourceRequest(yB);
assertTrue(0 != a.compareTo(b)); // don't care about order, just ne
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResourcesTrackerImpl.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResourcesTrackerImpl.java
index 3ee623c..a972945 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResourcesTrackerImpl.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResourcesTrackerImpl.java
@@ -230,7 +230,7 @@
long ts, LocalResourceVisibility vis) {
final LocalResourceRequest req =
new LocalResourceRequest(new Path("file:///tmp/" + user + "/rsrc" + i),
- ts + i * 2000, LocalResourceType.FILE, vis);
+ ts + i * 2000, LocalResourceType.FILE, vis, null);
return req;
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java
index c425eb5..ee24548 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceRetention.java
@@ -83,7 +83,7 @@
for (int i = 0; i < nRsrcs; ++i) {
final LocalResourceRequest req = new LocalResourceRequest(
new Path("file:///" + user + "/rsrc" + i), timestamp + i * tsstep,
- LocalResourceType.FILE, LocalResourceVisibility.PUBLIC);
+ LocalResourceType.FILE, LocalResourceVisibility.PUBLIC, null);
final long ts = timestamp + i * tsstep;
final Path p = new Path("file:///local/" + user + "/rsrc" + i);
LocalizedResource rsrc = new LocalizedResource(req, null) {
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
index 28985f5..eeeb31b 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
@@ -43,8 +43,9 @@
@Test
public void testContainerLogDirs() throws IOException {
- String logdirwithFile = "file:///target/"
- + TestNMWebServer.class.getSimpleName() + "LogDir";
+ String absLogDir = new File("target",
+ TestNMWebServer.class.getSimpleName() + "LogDir").getAbsolutePath();
+ String logdirwithFile = "file://" + absLogDir;
Configuration conf = new Configuration();
conf.set(YarnConfiguration.NM_LOG_DIRS, logdirwithFile);
NodeHealthCheckerService healthChecker = new NodeHealthCheckerService();
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java
index ad28d6c..8f631b0 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java
@@ -231,7 +231,6 @@
RMApp application = null;
try {
String clientTokenStr = null;
- String user = UserGroupInformation.getCurrentUser().getShortUserName();
if (UserGroupInformation.isSecurityEnabled()) {
Token<ClientTokenIdentifier> clientToken = new
Token<ClientTokenIdentifier>(
@@ -256,11 +255,12 @@
submissionContext);
// Create RMApp
- application = new RMAppImpl(applicationId, rmContext,
- this.conf, submissionContext.getApplicationName(), user,
- submissionContext.getQueue(), submissionContext, clientTokenStr,
- appStore, this.scheduler,
- this.masterService, submitTime);
+ application =
+ new RMAppImpl(applicationId, rmContext, this.conf,
+ submissionContext.getApplicationName(),
+ submissionContext.getUser(), submissionContext.getQueue(),
+ submissionContext, clientTokenStr, appStore, this.scheduler,
+ this.masterService, submitTime);
// Sanity check - duplicate?
if (rmContext.getRMApps().putIfAbsent(applicationId, application) !=
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
index e9e5340..552f959 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
@@ -64,7 +64,6 @@
import org.apache.hadoop.yarn.server.resourcemanager.scheduler.ResourceScheduler;
import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent;
import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType;
-import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler;
import org.apache.hadoop.yarn.server.resourcemanager.security.ApplicationTokenSecretManager;
import org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer;
import org.apache.hadoop.yarn.server.resourcemanager.security.RMContainerTokenSecretManager;
@@ -256,10 +255,22 @@
}
protected ResourceScheduler createScheduler() {
- return ReflectionUtils.newInstance(this.conf.getClass(
- YarnConfiguration.RM_SCHEDULER, FifoScheduler.class,
- ResourceScheduler.class), this.conf);
- }
+ String schedulerClassName = conf.get(YarnConfiguration.RM_SCHEDULER,
+ YarnConfiguration.DEFAULT_RM_SCHEDULER);
+ LOG.info("Using Scheduler: " + schedulerClassName);
+ try {
+ Class<?> schedulerClazz = Class.forName(schedulerClassName);
+ if (ResourceScheduler.class.isAssignableFrom(schedulerClazz)) {
+ return (ResourceScheduler) ReflectionUtils.newInstance(schedulerClazz,
+ this.conf);
+ } else {
+ throw new YarnException("Class: " + schedulerClassName
+ + " not instance of " + ResourceScheduler.class.getCanonicalName());
+ }
+ } catch (ClassNotFoundException e) {
+ throw new YarnException("Could not instantiate Scheduler: "
+ + schedulerClassName, e);
+ } }
protected ApplicationMasterLauncher createAMLauncher() {
return new ApplicationMasterLauncher(this.clientToAMSecretManager,
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
index 7e7bbee..d222b90 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
@@ -1197,7 +1197,7 @@
if (UserGroupInformation.isSecurityEnabled()) {
containerToken =
containerTokenSecretManager.createContainerToken(containerId, nodeId,
- capability);
+ application.getUser(), capability);
if (containerToken == null) {
return null; // Try again later.
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/AppSchedulable.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/AppSchedulable.java
index 7b46d84..3f97c96 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/AppSchedulable.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/AppSchedulable.java
@@ -161,7 +161,7 @@
if (UserGroupInformation.isSecurityEnabled()) {
containerToken =
containerTokenSecretManager.createContainerToken(containerId, nodeId,
- capability);
+ application.getUser(), capability);
if (containerToken == null) {
return null; // Try again later.
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
index aebf989a..a69374c4 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
@@ -539,7 +539,8 @@
if (UserGroupInformation.isSecurityEnabled()) {
containerToken =
this.rmContext.getContainerTokenSecretManager()
- .createContainerToken(containerId, nodeId, capability);
+ .createContainerToken(containerId, nodeId,
+ application.getUser(), capability);
if (containerToken == null) {
return i; // Try again later.
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
index accba56..b097ea3 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockAM.java
@@ -112,7 +112,7 @@
ResourceRequest hostReq = createResourceReq(host, memory, priority,
containers);
reqs.add(hostReq);
- ResourceRequest rackReq = createResourceReq("default-rack", memory,
+ ResourceRequest rackReq = createResourceReq("/default-rack", memory,
priority, containers);
reqs.add(rackReq);
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
index 9f8633d..ef935e6 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
@@ -36,6 +36,7 @@
import org.apache.hadoop.yarn.api.records.NodeId;
import org.apache.hadoop.yarn.api.records.Resource;
import org.apache.hadoop.yarn.api.records.NodeState;
+import org.apache.hadoop.yarn.conf.YarnConfiguration;
import org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEvent;
import org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher;
import org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory;
@@ -58,12 +59,12 @@
public class MockRM extends ResourceManager {
public MockRM() {
- this(new Configuration());
+ this(new YarnConfiguration());
}
public MockRM(Configuration conf) {
- super(StoreFactory.getStore(conf));
- init(conf);
+ super(StoreFactory.getStore(conf));
+ init(conf instanceof YarnConfiguration ? conf : new YarnConfiguration(conf));
Logger rootLogger = LogManager.getRootLogger();
rootLogger.setLevel(Level.DEBUG);
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
index d291b38..968b709 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
@@ -52,18 +52,27 @@
import org.apache.log4j.Level;
import org.apache.log4j.LogManager;
import org.apache.log4j.Logger;
+import org.junit.BeforeClass;
import org.junit.Test;
public class TestFifoScheduler {
private static final Log LOG = LogFactory.getLog(TestFifoScheduler.class);
private final int GB = 1024;
+ private static YarnConfiguration conf;
+
+ @BeforeClass
+ public static void setup() {
+ conf = new YarnConfiguration();
+ conf.setClass(YarnConfiguration.RM_SCHEDULER,
+ FifoScheduler.class, ResourceScheduler.class);
+ }
@Test
public void test() throws Exception {
Logger rootLogger = LogManager.getRootLogger();
rootLogger.setLevel(Level.DEBUG);
- MockRM rm = new MockRM();
+ MockRM rm = new MockRM(conf);
rm.start();
MockNM nm1 = rm.registerNode("h1:1234", 6 * GB);
MockNM nm2 = rm.registerNode("h2:5678", 4 * GB);
@@ -178,15 +187,15 @@
public void testDefaultMinimumAllocation() throws Exception {
// Test with something lesser than default
testMinimumAllocation(
- new YarnConfiguration(),
+ new YarnConfiguration(TestFifoScheduler.conf),
YarnConfiguration.DEFAULT_RM_SCHEDULER_MINIMUM_ALLOCATION_MB / 2);
}
@Test
public void testNonDefaultMinimumAllocation() throws Exception {
// Set custom min-alloc to test tweaking it
- int allocMB = 512;
- YarnConfiguration conf = new YarnConfiguration();
+ int allocMB = 1536;
+ YarnConfiguration conf = new YarnConfiguration(TestFifoScheduler.conf);
conf.setInt(YarnConfiguration.RM_SCHEDULER_MINIMUM_ALLOCATION_MB, allocMB);
// Test for something lesser than this.
testMinimumAllocation(conf, allocMB / 2);
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManager.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManager.java
index 536aa67..ce64065 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManager.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManager.java
@@ -30,6 +30,7 @@
import org.apache.hadoop.yarn.api.records.NodeHealthStatus;
import org.apache.hadoop.yarn.api.records.Priority;
import org.apache.hadoop.yarn.api.records.Resource;
+import org.apache.hadoop.yarn.conf.YarnConfiguration;
import org.apache.hadoop.yarn.server.resourcemanager.recovery.Store;
import org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory;
import org.apache.hadoop.yarn.server.resourcemanager.resource.Resources;
@@ -45,7 +46,7 @@
@Before
public void setUp() throws Exception {
- Configuration conf = new Configuration();
+ Configuration conf = new YarnConfiguration();
Store store = StoreFactory.getStore(conf);
resourceManager = new ResourceManager(store);
resourceManager.init(conf);
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
index d785255..1e92b7e 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java
@@ -268,10 +268,12 @@
MockNM nm2 = rm.registerNode("host2:5678", 5120);
nm1.nodeHeartbeat(true);
nm2.nodeHeartbeat(false);
+ dispatcher.await();
checkUnealthyNMCount(rm, nm2, true, 1);
final int expectedNMs = ClusterMetrics.getMetrics().getNumActiveNMs();
QueueMetrics metrics = rm.getResourceScheduler().getRootQueueMetrics();
- Assert.assertEquals(5120 + 5120, metrics.getAvailableMB());
+ // TODO Metrics incorrect in case of the FifoScheduler
+ Assert.assertEquals(5120, metrics.getAvailableMB());
// reconnect of healthy node
nm1 = rm.registerNode("host1:1234", 5120);
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/TestAMRMRPCNodeUpdates.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/TestAMRMRPCNodeUpdates.java
index d709bee..d607c01 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/TestAMRMRPCNodeUpdates.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/TestAMRMRPCNodeUpdates.java
@@ -90,10 +90,10 @@
@Test
public void testAMRMUnusableNodes() throws Exception {
- MockNM nm1 = rm.registerNode("h1:1234", 5000);
- MockNM nm2 = rm.registerNode("h2:1234", 5000);
- MockNM nm3 = rm.registerNode("h3:1234", 5000);
- MockNM nm4 = rm.registerNode("h4:1234", 5000);
+ MockNM nm1 = rm.registerNode("h1:1234", 10000);
+ MockNM nm2 = rm.registerNode("h2:1234", 10000);
+ MockNM nm3 = rm.registerNode("h3:1234", 10000);
+ MockNM nm4 = rm.registerNode("h4:1234", 10000);
RMApp app1 = rm.submitApp(2000);
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
index 8419eb4..4a26920 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
@@ -52,6 +52,7 @@
import org.apache.hadoop.yarn.server.resourcemanager.resource.Resources;
import org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptState;
import org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNode;
+import org.apache.hadoop.yarn.server.resourcemanager.scheduler.ResourceScheduler;
import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAddedSchedulerEvent;
import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppRemovedSchedulerEvent;
import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.NodeAddedSchedulerEvent;
@@ -93,7 +94,7 @@
@Before
public void setUp() throws IOException {
scheduler = new FairScheduler();
- Configuration conf = new Configuration();
+ Configuration conf = createConfiguration();
// All tests assume only one assignment per node update
conf.set(FairSchedulerConfiguration.ASSIGN_MULTIPLE, "false");
Store store = StoreFactory.getStore(conf);
@@ -109,6 +110,13 @@
resourceManager = null;
}
+ private Configuration createConfiguration() {
+ Configuration conf = new YarnConfiguration();
+ conf.setClass(YarnConfiguration.RM_SCHEDULER, FairScheduler.class,
+ ResourceScheduler.class);
+ return conf;
+ }
+
private ApplicationAttemptId createAppAttemptId(int appId, int attemptId) {
ApplicationAttemptId attId = recordFactory.newRecordInstance(ApplicationAttemptId.class);
ApplicationId appIdImpl = recordFactory.newRecordInstance(ApplicationId.class);
@@ -217,7 +225,9 @@
new ArrayList<ContainerStatus>(), new ArrayList<ContainerStatus>());
scheduler.handle(updateEvent);
- assertEquals(512, scheduler.getQueueManager().getQueue("queue1").
+ // Asked for less than min_allocation.
+ assertEquals(YarnConfiguration.DEFAULT_RM_SCHEDULER_MINIMUM_ALLOCATION_MB,
+ scheduler.getQueueManager().getQueue("queue1").
getQueueSchedulable().getResourceUsage().getMemory());
NodeUpdateSchedulerEvent updateEvent2 = new NodeUpdateSchedulerEvent(node2,
@@ -278,7 +288,7 @@
@Test
public void testUserAsDefaultQueue() throws Exception {
- Configuration conf = new Configuration();
+ Configuration conf = createConfiguration();
conf.set(FairSchedulerConfiguration.USER_AS_DEFAULT_QUEUE, "true");
scheduler.reinitialize(conf, resourceManager.getRMContext());
AppAddedSchedulerEvent appAddedEvent = new AppAddedSchedulerEvent(
@@ -299,7 +309,7 @@
@Test
public void testFairShareWithMinAlloc() throws Exception {
- Configuration conf = new Configuration();
+ Configuration conf = createConfiguration();
conf.set(FairSchedulerConfiguration.ALLOCATION_FILE, ALLOC_FILE);
scheduler.reinitialize(conf, resourceManager.getRMContext());
@@ -354,31 +364,35 @@
ApplicationAttemptId id22 = createAppAttemptId(2, 2);
scheduler.addApplication(id22, "queue2", "user1");
- // First ask, queue1 requests 1024
+ int minReqSize = YarnConfiguration.DEFAULT_RM_SCHEDULER_MINIMUM_ALLOCATION_MB;
+
+ // First ask, queue1 requests 1 large (minReqSize * 2).
List<ResourceRequest> ask1 = new ArrayList<ResourceRequest>();
- ResourceRequest request1 = createResourceRequest(1024, "*", 1, 1);
+ ResourceRequest request1 = createResourceRequest(minReqSize * 2, "*", 1, 1);
ask1.add(request1);
scheduler.allocate(id11, ask1, new ArrayList<ContainerId>());
- // Second ask, queue2 requests 1024 + (2 * 512)
+ // Second ask, queue2 requests 1 large + (2 * minReqSize)
List<ResourceRequest> ask2 = new ArrayList<ResourceRequest>();
- ResourceRequest request2 = createResourceRequest(1024, "foo", 1, 1);
- ResourceRequest request3 = createResourceRequest(512, "bar", 1, 2);
+ ResourceRequest request2 = createResourceRequest(2 * minReqSize, "foo", 1, 1);
+ ResourceRequest request3 = createResourceRequest(minReqSize, "bar", 1, 2);
ask2.add(request2);
ask2.add(request3);
scheduler.allocate(id21, ask2, new ArrayList<ContainerId>());
- // Third ask, queue2 requests 1024
+ // Third ask, queue2 requests 1 large
List<ResourceRequest> ask3 = new ArrayList<ResourceRequest>();
- ResourceRequest request4 = createResourceRequest(1024, "*", 1, 1);
+ ResourceRequest request4 = createResourceRequest(2 * minReqSize, "*", 1, 1);
ask3.add(request4);
scheduler.allocate(id22, ask3, new ArrayList<ContainerId>());
scheduler.update();
- assertEquals(1024, scheduler.getQueueManager().getQueue("queue1").getQueueSchedulable().getDemand().getMemory());
- assertEquals(1024 + 1024 + (2 * 512), scheduler.getQueueManager().getQueue("queue2").getQueueSchedulable().getDemand().getMemory());
-
+ assertEquals(2 * minReqSize, scheduler.getQueueManager().getQueue("queue1")
+ .getQueueSchedulable().getDemand().getMemory());
+ assertEquals(2 * minReqSize + 2 * minReqSize + (2 * minReqSize), scheduler
+ .getQueueManager().getQueue("queue2").getQueueSchedulable().getDemand()
+ .getMemory());
}
@Test
@@ -405,7 +419,7 @@
@Test
public void testAllocationFileParsing() throws Exception {
- Configuration conf = new Configuration();
+ Configuration conf = createConfiguration();
conf.set(FairSchedulerConfiguration.ALLOCATION_FILE, ALLOC_FILE);
scheduler.reinitialize(conf, resourceManager.getRMContext());
@@ -508,7 +522,7 @@
@Test
public void testBackwardsCompatibleAllocationFileParsing() throws Exception {
- Configuration conf = new Configuration();
+ Configuration conf = createConfiguration();
conf.set(FairSchedulerConfiguration.ALLOCATION_FILE, ALLOC_FILE);
scheduler.reinitialize(conf, resourceManager.getRMContext());
@@ -611,7 +625,7 @@
@Test
public void testIsStarvedForMinShare() throws Exception {
- Configuration conf = new Configuration();
+ Configuration conf = createConfiguration();
conf.set(FairSchedulerConfiguration.ALLOCATION_FILE, ALLOC_FILE);
scheduler.reinitialize(conf, resourceManager.getRMContext());
@@ -670,7 +684,7 @@
@Test
public void testIsStarvedForFairShare() throws Exception {
- Configuration conf = new Configuration();
+ Configuration conf = createConfiguration();
conf.set(FairSchedulerConfiguration.ALLOCATION_FILE, ALLOC_FILE);
scheduler.reinitialize(conf, resourceManager.getRMContext());
@@ -734,7 +748,7 @@
* now this means decreasing order of priority.
*/
public void testChoiceOfPreemptedContainers() throws Exception {
- Configuration conf = new Configuration();
+ Configuration conf = createConfiguration();
conf.set(FairSchedulerConfiguration.ALLOCATION_FILE + ".allocation.file", ALLOC_FILE);
scheduler.reinitialize(conf, resourceManager.getRMContext());
@@ -867,7 +881,7 @@
* Tests the timing of decision to preempt tasks.
*/
public void testPreemptionDecision() throws Exception {
- Configuration conf = new Configuration();
+ Configuration conf = createConfiguration();
conf.set(FairSchedulerConfiguration.ALLOCATION_FILE, ALLOC_FILE);
MockClock clock = new MockClock();
scheduler.setClock(clock);
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
index 317f892..2b21b2d 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
@@ -31,6 +31,7 @@
import org.apache.hadoop.yarn.api.records.Priority;
import org.apache.hadoop.yarn.api.records.QueueInfo;
import org.apache.hadoop.yarn.api.records.Resource;
+import org.apache.hadoop.yarn.conf.YarnConfiguration;
import org.apache.hadoop.yarn.event.AsyncDispatcher;
import org.apache.hadoop.yarn.server.resourcemanager.Application;
import org.apache.hadoop.yarn.server.resourcemanager.RMContext;
@@ -43,6 +44,7 @@
import org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.InlineDispatcher;
import org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNode;
import org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics;
+import org.apache.hadoop.yarn.server.resourcemanager.scheduler.ResourceScheduler;
import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.AppAddedSchedulerEvent;
import org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEvent;
import org.apache.hadoop.yarn.util.BuilderUtils;
@@ -59,7 +61,10 @@
public void setUp() throws Exception {
Store store = StoreFactory.getStore(new Configuration());
resourceManager = new ResourceManager(store);
- resourceManager.init(new Configuration());
+ Configuration conf = new Configuration();
+ conf.setClass(YarnConfiguration.RM_SCHEDULER,
+ FifoScheduler.class, ResourceScheduler.class);
+ resourceManager.init(conf);
}
@After
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
index bcbdd07..c410478 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServices.java
@@ -38,6 +38,7 @@
import org.apache.hadoop.yarn.server.resourcemanager.ResourceManager;
import org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics;
import org.apache.hadoop.yarn.server.resourcemanager.scheduler.ResourceScheduler;
+import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler;
import org.apache.hadoop.yarn.server.security.ApplicationACLsManager;
import org.apache.hadoop.yarn.service.Service.STATE;
import org.apache.hadoop.yarn.util.YarnVersionInfo;
@@ -74,7 +75,10 @@
bind(JAXBContextResolver.class);
bind(RMWebServices.class);
bind(GenericExceptionHandler.class);
- rm = new MockRM(new Configuration());
+ Configuration conf = new Configuration();
+ conf.setClass(YarnConfiguration.RM_SCHEDULER, FifoScheduler.class,
+ ResourceScheduler.class);
+ rm = new MockRM(conf);
bind(ResourceManager.class).toInstance(rm);
bind(RMContext.class).toInstance(rm.getRMContext());
bind(ApplicationACLsManager.class).toInstance(
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
index bd4399f..3bb9e56 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java
@@ -45,6 +45,8 @@
import org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppFailedAttemptEvent;
import org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppState;
import org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttempt;
+import org.apache.hadoop.yarn.server.resourcemanager.scheduler.ResourceScheduler;
+import org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler;
import org.apache.hadoop.yarn.server.security.ApplicationACLsManager;
import org.apache.hadoop.yarn.webapp.GenericExceptionHandler;
import org.apache.hadoop.yarn.webapp.WebServicesTestUtils;
@@ -82,6 +84,8 @@
bind(GenericExceptionHandler.class);
Configuration conf = new Configuration();
conf.setInt(YarnConfiguration.RM_AM_MAX_RETRIES, 2);
+ conf.setClass(YarnConfiguration.RM_SCHEDULER, FifoScheduler.class,
+ ResourceScheduler.class);
rm = new MockRM(conf);
bind(ResourceManager.class).toInstance(rm);
bind(RMContext.class).toInstance(rm.getRMContext());
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java
index 1a0885f..9a7a978 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/MiniYARNCluster.java
@@ -102,6 +102,12 @@
nodeManagers[index] = new CustomNodeManager();
}
}
+
+ @Override
+ public void init(Configuration conf) {
+ super.init(conf instanceof YarnConfiguration ? conf
+ : new YarnConfiguration(conf));
+ }
public File getTestWorkDir() {
return testWorkDir;
@@ -201,7 +207,7 @@
}
public synchronized void init(Configuration conf) {
- Configuration config = new Configuration(conf);
+ Configuration config = new YarnConfiguration(conf);
super.init(config);
}
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
index 1c7933a..0523a3f 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/TestContainerManagerSecurity.java
@@ -219,9 +219,10 @@
// Malice user modifies the resource amount
Resource modifiedResource = BuilderUtils.newResource(2048);
- ContainerTokenIdentifier modifiedIdentifier = new ContainerTokenIdentifier(
- dummyIdentifier.getContainerID(), dummyIdentifier.getNmHostAddress(),
- modifiedResource, Long.MAX_VALUE, dummyIdentifier.getMasterKeyId());
+ ContainerTokenIdentifier modifiedIdentifier =
+ new ContainerTokenIdentifier(dummyIdentifier.getContainerID(),
+ dummyIdentifier.getNmHostAddress(), "testUser", modifiedResource,
+ Long.MAX_VALUE, dummyIdentifier.getMasterKeyId());
Token<ContainerTokenIdentifier> modifiedToken = new Token<ContainerTokenIdentifier>(
modifiedIdentifier.getBytes(), containerToken.getPassword().array(),
new Text(containerToken.getKind()), new Text(containerToken
@@ -320,12 +321,14 @@
callWithIllegalContainerID(client, tokenId);
callWithIllegalResource(client, tokenId);
+ callWithIllegalUserName(client, tokenId);
return client;
}
});
- /////////// End of testing for illegal containerIDs and illegal Resources
+ // ///////// End of testing for illegal containerIDs, illegal Resources and
+ // illegal users
/////////// Test calls with expired tokens
RPC.stopProxy(client);
@@ -336,7 +339,7 @@
resourceManager.getRMContainerTokenSecretManager();
final ContainerTokenIdentifier newTokenId =
new ContainerTokenIdentifier(tokenId.getContainerID(),
- tokenId.getNmHostAddress(), tokenId.getResource(),
+ tokenId.getNmHostAddress(), "testUser", tokenId.getResource(),
System.currentTimeMillis() - 1,
containerTokenSecreteManager.getCurrentKey().getKeyId());
byte[] passowrd =
@@ -346,9 +349,7 @@
token = new Token<ContainerTokenIdentifier>(
newTokenId.getBytes(), passowrd, new Text(
containerToken.getKind()), new Text(containerToken.getService()));
-
-
-
+
unauthorizedUser.addToken(token);
unauthorizedUser.doAs(new PrivilegedAction<Void>() {
@Override
@@ -567,6 +568,29 @@
}
}
+ void callWithIllegalUserName(ContainerManager client,
+ ContainerTokenIdentifier tokenId) {
+ StartContainerRequest request = recordFactory
+ .newRecordInstance(StartContainerRequest.class);
+ // Authenticated but unauthorized, due to wrong resource
+ ContainerLaunchContext context =
+ createContainerLaunchContextForTest(tokenId);
+ context.setUser("Saruman"); // Set a different user-name.
+ request.setContainerLaunchContext(context);
+ try {
+ client.startContainer(request);
+ fail("Connection initiation with unauthorized "
+ + "access is expected to fail.");
+ } catch (YarnRemoteException e) {
+ LOG.info("Got exception : ", e);
+ Assert.assertTrue(e.getMessage().contains(
+ "Unauthorized request to start container. "));
+ Assert.assertTrue(e.getMessage().contains(
+ "Expected user-name " + tokenId.getApplicationSubmitter()
+ + " but found " + context.getUser()));
+ }
+ }
+
private ContainerLaunchContext createContainerLaunchContextForTest(
ContainerTokenIdentifier tokenId) {
ContainerLaunchContext context =
