Apache Hadoop Changelog

Release 1.3.0 - Unreleased (as of 2016-03-04)

INCOMPATIBLE CHANGES:

JIRASummaryPriorityComponentReporterContributor
MAPREDUCE-5777Support utf-8 text with BOM (byte order marker)Major.bc Wongzhihai xu

IMPORTANT ISSUES:

JIRASummaryPriorityComponentReporterContributor

NEW FEATURES:

JIRASummaryPriorityComponentReporterContributor

IMPROVEMENTS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-10614CBZip2InputStream is not threadsafeMajor.Xiangrui MengXiangrui Meng
HADOOP-9855Backport HADOOP-6578 to branch-1Major.James KinleyJames Kinley
HADOOP-9450HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpendedMajorscriptsMitch WyleHarsh J
HADOOP-8873Port HADOOP-8175 (Add mkdir -p flag) to branch-1Major.Eli CollinsAkira AJISAKA
HDFS-7312Update DistCp v1 to optionally not use tmp location (branch-1 only)MinortoolsJoseph ProsserJoseph Prosser
HDFS-5367Restoring namenode storage locks namenode due to unnecessary fsimage writeMajor.zhaoyunjiongzhaoyunjiong
HDFS-5038Backport several branch-2 APIs to branch-1Minor.Jing ZhaoJing Zhao
HDFS-4963Improve multihoming support in namenodeMajornamenodeArpit AgarwalArpit Agarwal
HDFS-4903Print trash configuration and trash emptier state in namenode logMinornamenodeSuresh SrinivasArpit Agarwal
HDFS-4521invalid network topologies should not be cachedMinor.Colin Patrick McCabeColin Patrick McCabe
MAPREDUCE-6088TestTokenCache tests should use their own JobConf instancesMajormrv1, testzhihai xuzhihai xu
MAPREDUCE-5712Backport Fair Scheduler pool placement by secondary groupMajorschedulerTed MalaskaTed Malaska
MAPREDUCE-5651Backport Fair Scheduler queue placement policies to branch-1MajorschedulerSandy RyzaTed Malaska
MAPREDUCE-5609Add debug log message when sending job end notificationMajor.Robert KanterRobert Kanter
MAPREDUCE-5457Add a KeyOnlyTextOutputReader to enable streaming to write out text files without separatorsMajor.Sandy RyzaSandy Ryza
MAPREDUCE-5408CLONE - The logging level of the tasks should be configurable by the jobMajor.Owen O'MalleyArun C Murthy
MAPREDUCE-5406Improve logging around Task Tracker exiting with JVM manager inconsistent stateMajortasktrackerChelsey ChangChelsey Chang
MAPREDUCE-5367Local jobs all use same local working directoryMajor.Sandy RyzaSandy Ryza
MAPREDUCE-3310Custom grouping comparator cannot be set for CombinersMajorclientMathias HerbertsAlejandro Abdelnur
MAPREDUCE-2351mapred.job.tracker.history.completed.location should support an arbitrary filesystem URIMajor.Tom WhiteTom White

BUG FIXES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-11035distcp on mr1(branch-1) fails with NPE using a short relative source path.Majortoolszhihai xuzhihai xu
HADOOP-10562Namenode exits on exception without printing stack trace in AbstractDelegationTokenSecretManagerCritical.Suresh SrinivasSuresh Srinivas
HADOOP-10142Avoid groups lookup for unprivileged users such as “dr.who”Major.Vinayakumar BVinayakumar B
HADOOP-10090Jobtracker metrics not updated properly after execution of a mapreduce jobMajormetricsIvan MiticIvan Mitic
HADOOP-10009Backport HADOOP-7808 to branch-1Major.Haohui MaiHaohui Mai
HADOOP-9801Configuration#writeXml uses platform defaulting encoding, which may mishandle multi-byte characters.MajorconfChris NaurothChris Nauroth
HADOOP-9768chown and chgrp reject users and groups with spaces on platforms where spaces are otherwise acceptableMajorfsChris NaurothChris Nauroth
HADOOP-9678TestRPC#testStopsAllThreads intermittently fails on WindowsMajor.Ivan MiticIvan Mitic
HADOOP-9507LocalFileSystem rename() is broken in some cases when destination existsMinorfsMostafa ElhemaliChris Nauroth
HADOOP-9307BufferedFSInputStream.read returns wrong results after certain seeksMajorfsTodd LipconTodd Lipcon
HADOOP-7140IPC Reader threads do not stop when server stopsCritical.Todd LipconTodd Lipcon
HDFS-7503Namenode restart after large deletions can cause slow processReport (due to logging)MajornamenodeArpit AgarwalArpit Agarwal
HDFS-6822In branch-1, Namenode and datanode fails to replace “_HOST” to hostname for hadoop.http.authentication.kerberos.principalMajorsecurityJing ZhaoJing Zhao
HDFS-6649Documentation for setrep is wrongTrivialdocumentationAlexander FahlkeAkira AJISAKA
HDFS-6141WebHdfsFileSystem#toUrl does not perform character escaping.MajorwebhdfsChris NaurothChris Nauroth
HDFS-5944LeaseManager:findLeaseWithPrefixPath can't handle path like /a/b/ right and cause SecondaryNameNode failed do checkpointMajornamenodezhaoyunjiongzhaoyunjiong
HDFS-5685DistCp will fail to copy with -delete switchMajorhdfs-clientYongjun ZhangYongjun Zhang
HDFS-5516WebHDFS does not require user name when anonymous http requests are disallowed.MajorwebhdfsChris NaurothMiodrag Radulovic
HDFS-5245shouldRetry() in WebHDFSFileSystem generates excessive warningsMinorwebhdfsHaohui MaiHaohui Mai
HDFS-5211Race condition between DistributedFileSystem#close and FileSystem#close can cause return of a closed DistributedFileSystem instance from the FileSystem cache.Majorhdfs-clientChris NaurothChris Nauroth
HDFS-5003TestNNThroughputBenchmark failed caused by existing directoriesMinortestXi FangXi Fang
HDFS-4944WebHDFS cannot create a file path containing characters that must be URI-encoded, such as space.MajorwebhdfsChris NaurothChris Nauroth
HDFS-4898BlockPlacementPolicyWithNodeGroup.chooseRemoteRack() fails to properly fallback to local rackMinornamenodeEric SirianniTsz Wo Nicholas Sze
HDFS-4794Browsing filesystem via webui throws kerberos exception when NN service RPC is enabled in a secure clusterMajorsecurityBenoy AntonyBenoy Antony
HDFS-4622Remove redundant synchronized from FSNamesystem#rollEditLog in branch-1Trivial.Jing ZhaoJing Zhao
HDFS-2264NamenodeProtocol has the wrong value for clientPrincipal in KerberosInfo annotationMajornamenodeAaron T. MyersAaron T. Myers
MAPREDUCE-6198NPE from JobTracker#resolveAndAddToTopology in MR1 cause initJob and heartbeat failure.Majormrv1zhihai xuzhihai xu
MAPREDUCE-6196Fix BigDecimal ArithmeticException in PiEstimatorMinor.Ray ChiangRay Chiang
MAPREDUCE-6170TestUlimit failure on JDK8Majorcontrib/streamingbc Wongbc Wong
MAPREDUCE-6147Support mapreduce.input.fileinputformat.split.maxsizeMinormrv1zhihai xuzhihai xu
MAPREDUCE-6076Zero map split input length combine with none zero map split input length may cause MR1 job hung sometimes.Majormrv1zhihai xuzhihai xu
MAPREDUCE-6012DBInputSplit creates invalid ranges on OracleMajor.Julien SerdaruWei Yan
MAPREDUCE-6009Map-only job with new-api runs wrong OutputCommitter when cleanup scheduled in a reduce slotBlockerclient, job submissionGera ShegalovGera Shegalov
MAPREDUCE-5979FairScheduler: zero weight can cause sort failuresMajorschedulerAnubhav DhootAnubhav Dhoot
MAPREDUCE-5968Work directory is not deleted when downloadCacheObject throws IOExceptionMajormrv1zhihai xuzhihai xu
MAPREDUCE-5966MR1 FairScheduler use of custom weight adjuster is not thread safe for comparisonsMajorschedulerAnubhav DhootAnubhav Dhoot
MAPREDUCE-5877Inconsistency between JT/TT for tasks taking a long time to launchCriticaljobtracker, tasktrackerKarthik KambatlaKarthik Kambatla
MAPREDUCE-5822FairScheduler does not preempt due to fairshare-starvation when fairshare is 1MajorschedulerAnubhav DhootAnubhav Dhoot
MAPREDUCE-5808Port output replication factor configurable for terasort to Hadoop 1.xMinorexamplesChuan LiuChuan Liu
MAPREDUCE-5710Backport MAPREDUCE-1305 to branch-1Major.Yongjun ZhangYongjun Zhang
MAPREDUCE-5702TaskLogServlet#printTaskLog has spurious HTML closing tagsTrivialtaskKarthik KambatlaRobert Kanter
MAPREDUCE-5698Backport MAPREDUCE-1285 to branch-1Major.Yongjun ZhangYongjun Zhang
MAPREDUCE-5660Log info about possible thrashing (when using memory-based scheduling in Capacity Scheduler) is not printedTrivialcapacity-sched, mrv1, tasktrackerAdam KawaAdam Kawa
MAPREDUCE-5569FloatSplitter is not generating correct splitsMajor.Nathan RobertsNathan Roberts
MAPREDUCE-5556mapred docs have incorrect classpathTrivial.Allen WittenauerHarsh J
MAPREDUCE-5512TaskTracker hung after failed reconnect to the JobTrackerMajortasktrackerIvan MiticIvan Mitic
MAPREDUCE-5508JobTracker memory leak caused by unreleased FileSystem objects in JobInProgress#cleanupJobCriticaljobtrackerXi FangXi Fang
MAPREDUCE-5405Job recovery can fail if task log directory symlink from prior run still existsMajormrv1Chris NaurothChris Nauroth
MAPREDUCE-5288ResourceEstimator#getEstimatedTotalMapOutputSize suffers from divide by zero issuesMajormrv1Harsh JKarthik Kambatla
MAPREDUCE-5272A Minor Error in Javadoc of TestMRWithDistributedCache in Branch-1TrivialtestZhijie ShenZhijie Shen
MAPREDUCE-5250Searching for ‘;’ in JobTracker History throws ArrayOutOfBoundExceptionMinor.Karthik KambatlaKarthik Kambatla
MAPREDUCE-5218Annotate (comment) internal classes as PrivateMinor.Karthik KambatlaKarthik Kambatla
MAPREDUCE-5217DistCp fails when launched by Oozie in a secure clusterMajordistcp, securityVenkat RanganathanVenkat Ranganathan
MAPREDUCE-5183In, TaskTracker#reportProgress logging of 0.0-1.0 progress is followed by percent signMinormrv1, tasktrackerSandy RyzaNiranjan Singh
MAPREDUCE-5133TestSubmitJob.testSecureJobExecution is flaky due to job dir deletion raceMajortestSandy RyzaSandy Ryza
MAPREDUCE-5095TestShuffleExceptionCount#testCheckException fails occasionally with JDK7Major.Arpit AgarwalArpit Agarwal
MAPREDUCE-5073TestJobStatusPersistency.testPersistency fails on JDK7MajortestSandy RyzaSandy Ryza
MAPREDUCE-5072TestDelegationTokenRenewal.testDTRenewal fails in MR1 on jdk7MajortestSandy RyzaSandy Ryza
MAPREDUCE-5070TestClusterStatus.testClusterMetrics fails on JDK7MajortestSandy RyzaSandy Ryza
MAPREDUCE-5047keep.failed.task.files=true causes job failure on secure clustersMajortask, tasktrackerSandy RyzaSandy Ryza
MAPREDUCE-5046backport MAPREDUCE-1423 to mapred.lib.CombineFileInputFormatMajorclientSangjin Lee
MAPREDUCE-4366mapred metrics shows negative count of waiting maps and reducesMajorjobtrackerThomas GravesSandy Ryza
MAPREDUCE-2817MiniRMCluster hardcodes ‘mapred.local.dir’ configuration to ‘build/test/mapred/local’MinortestAlejandro AbdelnurRobert Kanter

TESTS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-9624TestFSMainOperationsLocalFileSystem failed when the Hadoop test root path has “X” in its nameMinortestXi FangXi Fang

SUB-TASKS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-9573Fix test-patch script to work with the enhanced PreCommit-Admin script.MajorbuildGiridharan KesavanGiridharan Kesavan
MAPREDUCE-4049plugin for generic shuffle serviceMajorperformance, task, tasktrackerAvner BenHanochAvner BenHanoch

OTHER:

JIRASummaryPriorityComponentReporterContributor