Apache Hadoop Changelog

Release 2.4.1 - 2014-06-30

INCOMPATIBLE CHANGES:

JIRASummaryPriorityComponentReporterContributor

IMPORTANT ISSUES:

JIRASummaryPriorityComponentReporterContributor

NEW FEATURES:

JIRASummaryPriorityComponentReporterContributor

IMPROVEMENTS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-10466Lower the log level in UserGroupInformationMinorsecurityNicolas LiochonNicolas Liochon
HDFS-4052BlockManager#invalidateWork should print logs outside the lockMinor.Jing ZhaoJing Zhao
YARN-1892Excessive logging in RMMinorschedulerSiddharth SethJian He

BUG FIXES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-11273TestMiniKdc failure: login options not compatible with IBM JDKMajortestGao Zhong LiangGao Zhong Liang
HADOOP-10612NFS failed to refresh the user group id mapping tableMajornfsBrandon LiBrandon Li
HADOOP-10562Namenode exits on exception without printing stack trace in AbstractDelegationTokenSecretManagerCritical.Suresh SrinivasSuresh Srinivas
HADOOP-10527Fix incorrect return code and allow more retries on EINTRMajor.Kihwal LeeKihwal Lee
HADOOP-10522JniBasedUnixGroupMapping mishandles errorsCritical.Kihwal LeeKihwal Lee
HADOOP-10490TestMapFile and TestBloomMapFile leak file descriptors.MinortestChris NaurothChris Nauroth
HADOOP-10473TestCallQueueManager is still flakyMinortestTsz Wo Nicholas SzeTsz Wo Nicholas Sze
HADOOP-10456Bug in Configuration.java exposed by Spark (ConcurrentModificationException)MajorconfNishkam RaviNishkam Ravi
HADOOP-10455When there is an exception, ipc.Server should first check whether it is an terse exceptionMajoripcTsz Wo Nicholas SzeTsz Wo Nicholas Sze
HADOOP-8826Docs still refer to 0.20.205 as stable lineMinor.Robert Joseph EvansMit Desai
HDFS-6411nfs-hdfs-gateway mount raises I/O error and hangs when a unauthorized user attempts to access itMajornfsZhongyi XieBrandon Li
HDFS-6402Suppress findbugs warning for failure to override equals and hashCode in FsAclPermission.TrivialnamenodeChris NaurothChris Nauroth
HDFS-6397NN shows inconsistent value in deadnode countCritical.Mohammad Kamrul IslamMohammad Kamrul Islam
HDFS-6362InvalidateBlocks is inconsistent in usage of DatanodeUuid and StorageIDBlockernamenodeArpit AgarwalArpit Agarwal
HDFS-6361TestIdUserGroup.testUserUpdateSetting failed due to out of range nfsnobody IdMajornfsYongjun ZhangYongjun Zhang
HDFS-6340DN can't finalize upgradeBlockerdatanodeRahul SinghalRahul Singhal
HDFS-6329WebHdfs does not work if HA is enabled on NN but logical URI is not configured.Blocker.Kihwal LeeKihwal Lee
HDFS-6326WebHdfs ACL compatibility is brokenBlockerwebhdfsDaryn SharpChris Nauroth
HDFS-6325Append should fail if the last block has insufficient number of replicasMajornamenodeKonstantin ShvachkoKeith Pak
HDFS-6313WebHdfs may use the wrong NN when configured for multiple HA NNsBlockerwebhdfsDaryn SharpKihwal Lee
HDFS-6245datanode fails to start with a bad disk even when failed volumes is setMajor.Arpit GuptaArpit Agarwal
HDFS-6236ImageServlet should use Time#monotonicNow to measure latency.MinornamenodeChris NaurothChris Nauroth
HDFS-6235TestFileJournalManager can fail on Windows due to file locking if tests run out of order.Trivialnamenode, testChris NaurothChris Nauroth
HDFS-6234TestDatanodeConfig#testMemlockLimit fails on Windows due to invalid file path.Trivialdatanode, testChris NaurothChris Nauroth
HDFS-6232OfflineEditsViewer throws a NPE on edits containing ACL modificationsMajortoolsStephen ChuAkira AJISAKA
HDFS-6231DFSClient hangs infinitely if using hedged reads and all eligible datanodes die.Majorhdfs-clientChris NaurothChris Nauroth
HDFS-6229Race condition in failover can cause RetryCache fail to workMajorhaJing ZhaoJing Zhao
HDFS-6215Wrong error message for upgradeMinor.Kihwal LeeKihwal Lee
HDFS-6209Fix flaky test TestValidateConfigurationSettings.testThatDifferentRPCandHttpPortsAreOKMinortestArpit AgarwalArpit Agarwal
HDFS-6208DataNode caching can leak file descriptors.MajordatanodeChris NaurothChris Nauroth
HDFS-6206DFSUtil.substituteForWildcardAddress may throw NPEMajor.Tsz Wo Nicholas SzeTsz Wo Nicholas Sze
HDFS-6204TestRBWBlockInvalidation may failMinortestTsz Wo Nicholas SzeTsz Wo Nicholas Sze
HDFS-6198DataNode rolling upgrade does not correctly identify current block pool directory and replace with trash on Windows.MajordatanodeChris NaurothChris Nauroth
HDFS-6197Rolling upgrade rollback on Windows can fail attempting to rename edit log segment files to a destination that already exists.MinornamenodeChris NaurothChris Nauroth
HDFS-2882DN continues to start up, even if block pool fails to initializeMajordatanodeTodd LipconVinayakumar B
MAPREDUCE-5841uber job doesn't terminate on getting mapred job killMajormrv2Sangjin LeeSangjin Lee
MAPREDUCE-5835Killing Task might cause the job to go to ERROR stateCritical.Ming MaMing Ma
MAPREDUCE-5832Few tests in TestJobClient fail on WindowsMajor.Jian HeVinod Kumar Vavilapalli
MAPREDUCE-5830HostUtil.getTaskLogUrl is not backwards binary compatible with 2.3Blocker.Jason LoweAkira AJISAKA
MAPREDUCE-5828TestMapReduceJobControl fails on JDK 7 + WindowsMajor.Vinod Kumar VavilapalliVinod Kumar Vavilapalli
MAPREDUCE-5827TestSpeculativeExecutionWithMRApp failsMajor.Zhijie ShenZhijie Shen
MAPREDUCE-5826TestHistoryServerFileSystemStateStoreService.testTokenStore fails in windowsMajor.Varun VasudevVarun Vasudev
MAPREDUCE-5824TestPipesNonJavaInputFormat.testFormat fails in windowsMajor.Xuan GongXuan Gong
MAPREDUCE-5821IFile merge allocates new byte array for every valueMajorperformance, taskTodd LipconTodd Lipcon
MAPREDUCE-5818hsadmin cmd is missing in mapred.cmdMajor.Jian HeJian He
MAPREDUCE-5815Fix NPE in TestMRAppMasterBlockerclient, mrv2Gera ShegalovAkira AJISAKA
MAPREDUCE-5714TestMRAppComponentDependencies causes surefire to exit without saying proper goodbyeMajortestJinghui WangJinghui Wang
MAPREDUCE-3191docs for map output compression incorrectly reference SequenceFileTrivial.Todd LipconChen He
YARN-2081TestDistributedShell fails after YARN-1962Minorapplications/distributed-shellHong ZhiguoHong Zhiguo
YARN-2066Wrong field is referenced in GetApplicationsRequestPBImpl#mergeLocalToBuilder()Minor.Ted YuHong Zhiguo
YARN-2016Yarn getApplicationRequest start time range is not honoredMajorresourcemanagerVenkat RanganathanJunping Du
YARN-1986In Fifo Scheduler, node heartbeat in between creating app and attempt causes NPECritical.Jon BringhurstHong Zhiguo
YARN-1976Tracking url missing http protocol for FAILED applicationMajor.Yesha VoraJunping Du
YARN-1975Used resources shows escaped html in CapacityScheduler and FairScheduler pageMajorresourcemanagerNathan RobertsMit Desai
YARN-1934Potential NPE in ZKRMStateStore caused by handling Disconnected event from ZK.BlockerresourcemanagerRohith Sharma K SKarthik Kambatla
YARN-1933TestAMRestart and TestNodeHealthService failing sometimes on WindowsMajor.Jian HeJian He
YARN-1932Javascript injection on the job status pageBlocker.Mit DesaiMit Desai
YARN-1931Private API change in YARN-1824 in 2.4 broke compatibility with previous releasesBlockerapplicationsThomas GravesSandy Ryza
YARN-1929DeadLock in RM when automatic failover is enabled.BlockerresourcemanagerRohith Sharma K SKarthik Kambatla
YARN-1928TestAMRMRPCNodeUpdates fails ocassionallyMajor.Zhijie ShenZhijie Shen
YARN-1926DistributedShell unit tests fail on WindowsMajor.Varun VasudevVarun Vasudev
YARN-1924STATE_STORE_OP_FAILED happens when ZKRMStateStore tries to update app(attempt) before storing itCritical.Arpit GuptaJian He
YARN-1920TestFileSystemApplicationHistoryStore.testMissingApplicationAttemptHistoryData fails in windowsMajor.Vinod Kumar VavilapalliVinod Kumar Vavilapalli
YARN-1914Test TestFSDownload.testDownloadPublicWithStatCache fails on WindowsMajor.Varun VasudevVarun Vasudev
YARN-1910TestAMRMTokens fails on windowsMajor.Xuan GongXuan Gong
YARN-1908Distributed shell with custom script has permission error.Majorapplications/distributed-shellTassapol AthiapinyaVinod Kumar Vavilapalli
YARN-1907TestRMApplicationHistoryWriter#testRMWritingMassiveHistory runs slow and intermittently failsMajor.Mit DesaiMit Desai
YARN-1903Killing Container on NEW and LOCALIZING will result in exitCode and diagnostics not setMajor.Zhijie ShenZhijie Shen
YARN-1883TestRMAdminService fails due to inconsistent entries in UserGroupsMajor.Mit DesaiMit Desai
YARN-1837TestMoveApplication.testMoveRejectedByScheduler randomly failsMajor.Tsuyoshi OzawaHong Zhiguo
YARN-1201TestAMAuthorization fails with local hostname cannot be resolvedMinorresourcemanagerNemon LouWangda Tan

TESTS:

JIRASummaryPriorityComponentReporterContributor
HDFS-6189Multiple HDFS tests fail on Windows attempting to use a test root path containing a colon.MajortestChris NaurothChris Nauroth
MAPREDUCE-5843TestMRKeyValueTextInputFormat failing on WindowsMajor.Varun VasudevVarun Vasudev
MAPREDUCE-5833TestRMContainerAllocator fails ocassionallyMajor.Zhijie ShenZhijie Shen
YARN-1947TestRMDelegationTokens#testRMDTMasterKeyStateOnRollingMasterKey is failing intermittentlyMajor.Jian HeJian He
YARN-1905TestProcfsBasedProcessTree must only run on Linux.TrivialnodemanagerChris NaurothChris Nauroth
YARN-1750TestNodeStatusUpdater#testNMRegistration is incorrect in test caseMajornodemanagerMing MaWangda Tan
YARN-1281TestZKRMStateStoreZKClientConnections fails intermittentlyMajorresourcemanagerKarthik KambatlaTsuyoshi Ozawa

SUB-TASKS:

JIRASummaryPriorityComponentReporterContributor
YARN-2053Slider AM fails to restart: NPE in RegisterApplicationMasterResponseProto$Builder.addAllNmTokensFromPreviousAttemptsMajorresourcemanagerSumit MohantyWangda Tan
YARN-1962Timeline server is enabled by defaultMajor.Mohammad Kamrul IslamMohammad Kamrul Islam
YARN-1957ProportionalCapacitPreemptionPolicy handling of corner cases...MajorresourcemanagerCarlo CurinoCarlo Curino
YARN-1898Standby RM's conf, stacks, logLevel, metrics, jmx and logs links are redirecting to Active RMMajorresourcemanagerYesha VoraXuan Gong
YARN-1861Both RM stuck in standby mode when automatic failover is enabledBlockerresourcemanagerArpit GuptaKarthik Kambatla
YARN-1701Improve default paths of timeline store and generic history storeMajor.Gera ShegalovTsuyoshi Ozawa
YARN-1696Document RM HABlockerresourcemanagerKarthik KambatlaTsuyoshi Ozawa

OTHER:

JIRASummaryPriorityComponentReporterContributor