Apache Hadoop Changelog

Release 0.23.6 - 2013-02-06

INCOMPATIBLE CHANGES:

JIRASummaryPriorityComponentReporterContributor

IMPORTANT ISSUES:

JIRASummaryPriorityComponentReporterContributor

NEW FEATURES:

JIRASummaryPriorityComponentReporterContributor

IMPROVEMENTS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-8992Enhance unit-test coverage of class HarFileSystemMinor.Ivan A. VeselovskyIvan A. Veselovsky
HADOOP-8931Add Java version to startup messageTrivial.Eli CollinsEli Collins
HADOOP-8561Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processesMajorsecurityLuke LuYu Gao
MAPREDUCE-4899Provide a plugin to the Yarn Web App Proxy to generate tracking links for M/R appllications given the IDMajor.Derek DagitDerek Dagit
MAPREDUCE-4845ClusterStatus.getMaxMemory() and getUsedMemory() exist in MR1 but not MR2MajorclientSandy RyzaSandy Ryza
MAPREDUCE-4811JobHistoryServer should show when it was started in WebUI About pageMinorjobhistoryserver, mrv2Ravi PrakashRavi Prakash
MAPREDUCE-4810Add admin command options for ApplicationMasterMinorapplicationmasterJason LoweJerry Chen
MAPREDUCE-4764repair test org.apache.hadoop.mapreduce.security.TestBinaryTokenFileMajor.Ivan A. Veselovsky
YARN-285RM should be able to provide a tracking link for apps that have already been purgedMajor.Derek DagitDerek Dagit
YARN-80Support delay scheduling for node locality in MR2's capacity schedulerMajorcapacityschedulerTodd LipconArun C Murthy

BUG FIXES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-9255relnotes.py missing last jiraCriticalscriptsThomas GravesThomas Graves
HADOOP-9242Duplicate surefire plugin config in hadoop-commonMajortestAndrey KlochkovAndrey Klochkov
HADOOP-9181Set daemon flag for HttpServer's QueuedThreadPoolMajor.Liang XieLiang Xie
HADOOP-9169Bring branch-0.23 ExitUtil up to same level as branch-2Major.Robert Joseph EvansRobert Joseph Evans
HADOOP-9152HDFS can report negative DFS Used on clusters with very small amounts of dataMinorfsBrock NolandBrock Noland
HADOOP-9135JniBasedUnixGroupsMappingWithFallback should log at debug rather than info during fallbackTrivialsecurityColin Patrick McCabeColin Patrick McCabe
HADOOP-9108Add a method to clear terminateCalled to ExitUtil for test casesMajorutilKihwal LeeKihwal Lee
HADOOP-9105FsShell -moveFromLocal erroneously failsMajorfsDaryn SharpDaryn Sharp
HADOOP-9097Maven RAT plugin is not checking all source filesCriticalbuildTom WhiteThomas Graves
HADOOP-9072Hadoop-Common-0.23-Build Fails to build in JenkinsMajor.Robert ParkerRobert Parker
HADOOP-7868Hadoop native fails to compile when default linker option is -Wl,--as-neededMajornativeJames PageTrevor Robinson
HDFS-4426Secondary namenode shuts down immediately after startupBlockernamenodeJason LoweArpit Agarwal
HDFS-4385Maven RAT plugin is not checking all source filesCriticalbuildThomas GravesThomas Graves
HDFS-4315DNs with multiple BPs can have BPOfferServices fail to start due to unsynchronized map accessMajordatanodeAaron T. MyersAaron T. Myers
HDFS-4270Replications of the highest priority should be allowed to choose a source datanode that has reached its max replication limitMinornamenodeDerek DagitDerek Dagit
HDFS-4254testAllEditsDirsFailOnFlush makes subsequent test cases fail (0.23.6 only)MajortestKihwal LeeKihwal Lee
HDFS-4242Map.Entry is incorrectly used in LeaseManagerMajornamenodeTsz Wo Nicholas SzeTsz Wo Nicholas Sze
HDFS-4233NN keeps serving even after no journals started while rolling editBlockernamenodeKihwal LeeKihwal Lee
HDFS-4232NN fails to write a fsimage with stale leasesBlockernamenodeDaryn SharpDaryn Sharp
HDFS-4186logSync() is called with the write lock held while releasing leaseCriticalnamenodeKihwal LeeKihwal Lee
HDFS-2757Cannot read a local block that's being written to when using the local read short circuitMajor.Jean-Daniel CryansJean-Daniel Cryans
MAPREDUCE-4934Maven RAT plugin is not checking all source filesCriticalbuildThomas GravesThomas Graves
MAPREDUCE-4925The pentomino option parser may be buggyMajorexamplesKarthik KambatlaKarthik Kambatla
MAPREDUCE-4921JobClient should acquire HS token with RM principalBlockerclientDaryn SharpDaryn Sharp
MAPREDUCE-4913TestMRAppMaster#testMRAppMasterMissingStaging occasionally exitsMajormr-amJason LoweJason Lowe
MAPREDUCE-4902Fix typo “receievd” should be “received” in log outputTrivial.Albert ChuAlbert Chu
MAPREDUCE-4895Fix compilation failure of org.apache.hadoop.mapred.gridmix.TestResourceUsageEmulatorsMajor.Dennis YDennis Y
MAPREDUCE-4894Renewal / cancellation of JobHistory tokensBlockerjobhistoryserver, mrv2Siddharth SethSiddharth Seth
MAPREDUCE-4890Invalid TaskImpl state transitions when task fails while speculatingCriticalmr-amJason LoweJason Lowe
MAPREDUCE-4861Cleanup: Remove unused mapreduce.security.token.DelegationTokenRenewalMajor.Karthik KambatlaKarthik Kambatla
MAPREDUCE-4856TestJobOutputCommitter uses same directory as TestJobCleanupMajortestSandy RyzaSandy Ryza
MAPREDUCE-4848TaskAttemptContext cast error during AM recoveryMajormr-amJason LoweJerry Chen
MAPREDUCE-4842Shuffle race can hang reducerBlockermrv2Jason LoweMariappan Asokan
MAPREDUCE-4836Elapsed time for running tasks on AM web UI tasks page is 0Major.Ravi PrakashRavi Prakash
MAPREDUCE-4833Task can get stuck in FAIL_CONTAINER_CLEANUPCriticalapplicationmaster, mrv2Robert Joseph EvansRobert Parker
MAPREDUCE-4832MR AM can get in a split brain situationCriticalapplicationmasterRobert Joseph EvansJason Lowe
MAPREDUCE-4825JobImpl.finished doesn't expect ERROR as a final job stateMajormr-amJason LoweJason Lowe
MAPREDUCE-4819AM can rerun job after reporting final job status to the clientBlockermr-amJason LoweBikas Saha
MAPREDUCE-4817Hardcoded task ping timeout kills tasks localizing large amounts of dataCriticalapplicationmaster, mr-amJason LoweThomas Graves
MAPREDUCE-4813AM timing out during job commitCriticalapplicationmasterJason LoweJason Lowe
MAPREDUCE-4678Running the Pentomino example with defaults throws java.lang.NegativeArraySizeExceptionMinorexamplesChris McConnellChris McConnell
MAPREDUCE-4279getClusterStatus() fails with null pointer exception when running jobs in local modeMajorjobtrackerRahul JainDevaraj K
YARN-354WebAppProxyServer exits immediately after startupBlocker.Liang XieLiang Xie
YARN-334Maven RAT plugin is not checking all source filesCritical.Thomas GravesThomas Graves
YARN-325RM CapacityScheduler can deadlock when getQueueInfo() is called and a container is completingBlockercapacityschedulerJason LoweArun C Murthy
YARN-320RM should always be able to renew its own tokensBlockerresourcemanagerDaryn SharpDaryn Sharp
YARN-293Node Manager leaks LocalizerRunner object for every ContainerCriticalnodemanagerDevaraj KRobert Joseph Evans
YARN-266RM and JHS Web UIs are blank because AppsBlock is not escaping string properlyCriticalresourcemanagerRavi PrakashRavi Prakash
YARN-258RM web page UI shows Invalid Date for start and finish timesMajorresourcemanagerRavi PrakashRavi Prakash
YARN-251Proxy URI generation fails for blank tracking URIsMajorresourcemanagerTom WhiteTom White
YARN-225Proxy Link in RM UI thows NPE in Secure modeCriticalresourcemanagerDevaraj KDevaraj K
YARN-223Change processTree interface to work better with native codeCritical.Radim KolarRadim Kolar
YARN-170NodeManager stop() gets called twice on shutdownMajornodemanagerSandy RyzaSandy Ryza
YARN-72NM should handle cleaning up containers when it shuts downMajornodemanagerHitesh ShahSandy Ryza

TESTS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-9217Print thread dumps when hadoop-common tests failMajortestAndrey KlochkovAndrey Klochkov
HADOOP-9038provide unit-test coverage of class org.apache.hadoop.fs.LocalDirAllocator.AllocatorPerContext.PathIteratorMinor.Ivan A. VeselovskyIvan A. Veselovsky

SUB-TASKS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-9020Add a SASL PLAIN serverMajoripc, securityDaryn SharpDaryn Sharp
HDFS-4248Renames may remove file leasesBlockernamenodeDaryn SharpDaryn Sharp
HDFS-4247saveNamespace should be tolerant of dangling leaseBlockernamenodeDaryn SharpDaryn Sharp
YARN-280RM does not reject app submission with invalid tokensMajorresourcemanagerDaryn SharpDaryn Sharp
YARN-204test coverage for org.apache.hadoop.toolsMajorapplicationsAleksey GorshkovAleksey Gorshkov
YARN-50Implement renewal / cancellation of Delegation TokensBlocker.Siddharth SethSiddharth Seth

OTHER:

JIRASummaryPriorityComponentReporterContributor
HADOOP-8427Convert Forrest docs to APT, incrementalMajordocumentationEli CollinsAndy Isaacson