Apache Hadoop Changelog

Release 0.20.1 - 2009-09-01

INCOMPATIBLE CHANGES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-5881Simplify configuration related to task-memory-monitoring and memory-based schedulingMajor.Vinod Kumar VavilapalliVinod Kumar Vavilapalli
HADOOP-5726Remove pre-emption from the capacity scheduler code baseMajor.Hemanth Yamijalarahul k singh

IMPORTANT ISSUES:

JIRASummaryPriorityComponentReporterContributor

NEW FEATURES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-6080Handling of Trash with quotaMajorfsKoji NoguchiJakob Homan
HADOOP-5714Metric to show number of fs.exists (or number of getFileInfo) callsMinormetricsKoji NoguchiJakob Homan
HADOOP-3315New binary file formatMajorioOwen O'MalleyHong Tang

IMPROVEMENTS:

JIRASummaryPriorityComponentReporterContributor
HDFS-635HDFS Project page does not show 0.20.1 documentation/release information.MajordocumentationAndy Sautins
HDFS-527Refactor DFSClient constructorsMajorhdfs-clientTsz Wo Nicholas SzeTsz Wo Nicholas Sze
MAPREDUCE-767to remove mapreduce dependency on commons-cli2Majorcontrib/streamingGiridharan KesavanAmar Kamat
MAPREDUCE-465Deprecate org.apache.hadoop.mapred.lib.MultithreadedMapRunnerMinor.Amareshwari SriramadasuAmareshwari Sriramadasu

BUG FIXES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-6215fix GenericOptionParser to deal with -D with ‘=’ in the valueMajor.Owen O'MalleyAmar Kamat
HADOOP-6145No error message for deleting non-existant file or directory.MajorfsSuman SehgalJakob Homan
HADOOP-6141hadoop 0.20 branch “test-patch” is brokenMajorbuildHong TangHong Tang
HADOOP-6139Incomplete help message is displayed for rm and rmr options.Minor.Suman SehgalJakob Homan
HADOOP-6017NameNode and SecondaryNameNode fail to restart because of abnormal filenames.Blocker.Raghu AngadiTsz Wo Nicholas Sze
HADOOP-5951StorageInfo needs Apache license header.Major.Suresh SrinivasSuresh Srinivas
HADOOP-5937Correct info message “Use hadoop dfs -safemode option” to " Use hdfs dfsadmin -safemode option" .Minor.Ravi PhulariRavi Phulari
HADOOP-5932MemoryMatcher logs 0 as freeMemOnTT even though there are free slots available on TaskTrakerMajor.Karam SinghVinod Kumar Vavilapalli
HADOOP-5924JT fails to recover the jobs after restart after HADOOP:4372Major.Ramya SunilAmar Kamat
HADOOP-5921JobTracker does not come up because of NotReplicatedYetExceptionMajor.Amareshwari SriramadasuAmar Kamat
HADOOP-5920TestJobHistory fails some times.Major.Amareshwari SriramadasuAmar Kamat
HADOOP-5908ArithmeticException in heartbeats with zero map jobsMajor.Vinod Kumar VavilapalliAmar Kamat
HADOOP-5884Capacity scheduler should account high memory jobs as using more capacity of the queueMajor.Hemanth YamijalaVinod Kumar Vavilapalli
HADOOP-5883TaskMemoryMonitorThread might shoot down tasks even if their processes momentarily exceed the requested memoryMajor.Hemanth YamijalaHemanth Yamijala
HADOOP-5882Progress is not updated when the New Reducer is running reduce phaseBlocker.Jothi PadmanabhanAmareshwari Sriramadasu
HADOOP-5850map/reduce doesn't run jobs with 0 mapsCritical.Owen O'MalleyVinod Kumar Vavilapalli
HADOOP-5828Use absolute path for JobTracker's mapred.local.dir in MiniMRClusterMajortestHemanth YamijalaHemanth Yamijala
HADOOP-5746Errors encountered in MROutputThread after the last map/reduce call can go undetectedMajor.Devaraj DasAmar Kamat
HADOOP-5736Update CapacityScheduler documentation to reflect latest changesMajor.Sreekanth RamakrishnanSreekanth Ramakrishnan
HADOOP-5719Jobs failed during job initalization are never removed from Capacity Schedulers waiting listMajor.Sreekanth RamakrishnanSreekanth Ramakrishnan
HADOOP-5718Capacity Scheduler should not check for presence of default queue while starting up.Major.Sreekanth RamakrishnanSreekanth Ramakrishnan
HADOOP-5711Change Namenode file close log to infoMinor.Tsz Wo Nicholas SzeTsz Wo Nicholas Sze
HADOOP-5691org.apache.hadoop.mapreduce.Reducer should not be abstract.Major.Amareshwari SriramadasuAmareshwari Sriramadasu
HADOOP-5688HftpFileSystem.getChecksum(..) does not work for the paths with scheme and authorityMajor.Tsz Wo Nicholas SzeTsz Wo Nicholas Sze
HADOOP-5655TestMRServerPorts fails on java.net.BindExceptionMajor.Hairong KuangDevaraj Das
HADOOP-5654TestReplicationPolicy.<init> fails on java.net.BindExceptionMajortestHairong KuangHairong Kuang
HADOOP-5648Not able to generate gridmix.jar on already compiled version of hadoopMajorbenchmarksSuman SehgalGiridharan Kesavan
HADOOP-5646TestQueueCapacities is failing Hudson tests for the last few buildsMajor.Jothi PadmanabhanVinod Kumar Vavilapalli
HADOOP-5641Possible NPE in CapacityScheduler's MemoryMatcherMajor.Vinod Kumar VavilapalliHemanth Yamijala
HADOOP-5636Job is left in Running state after a killJobCritical.Amareshwari SriramadasuAmar Kamat
HADOOP-5539o.a.h.mapred.Merger not maintaining map out compression on intermediate filesBlocker.Billy PearsonJothi Padmanabhan
HADOOP-5533Recovery duration shown on the jobtracker webpage is inaccurateMajor.Amar KamatAmar Kamat
HADOOP-5349When the size required for a path is -1, LocalDirAllocator.getLocalPathForWrite fails with a DiskCheckerException when the disk it selects is bad.Major.Vinod Kumar VavilapalliVinod Kumar Vavilapalli
HADOOP-5213BZip2CompressionOutputStream NullPointerExceptionBlockerioZheng ShaoZheng Shao
HADOOP-5210Reduce Task Progress shows > 100% when the total size of map outputs (for a single reducer) is highMinor.Jothi PadmanabhanRavi Gummadi
HADOOP-4674hadoop fs -help should list detailed help info for the following commands: test, text, tail, stat & touchzTrivialfsDavid NeSmithRavi Phulari
HADOOP-4626API link in forrest doc should point to the same version of hadoop.MinordocumentationTsz Wo Nicholas SzeTsz Wo Nicholas Sze
HDFS-1022Merge under-10-min tests specs into one fileMajortestErik StefflErik Steffl
HDFS-525ListPathsServlet.java uses static SimpleDateFormat that has threading issuesMajornamenodeSuresh SrinivasSuresh Srinivas
HDFS-438Improve help message for quotasMinor.Raghu AngadiRaghu Angadi
HDFS-167DFSClient continues to retry indefinitelyMinorhdfs-clientDerek WollensteinBill Zeller
HDFS-26HADOOP-5862 for version .20 (Namespace quota exceeded message unclear)Major.Boris ShkolnikBoris Shkolnik
MAPREDUCE-924TestPipes must not directly invoke ‘main’ of pipes as an exit from main could cause the testcase to crash.MajorpipesAmareshwari SriramadasuAmareshwari Sriramadasu
MAPREDUCE-911TestTaskFail fail sometimesMajortestAmareshwari SriramadasuAmareshwari Sriramadasu
MAPREDUCE-838Task succeeds even when committer.commitTask fails with IOExceptionBlockertaskKoji NoguchiAmareshwari Sriramadasu
MAPREDUCE-834When TaskTracker config use old memory management values its memory monitoring is diabled.Major.Karam SinghSreekanth Ramakrishnan
MAPREDUCE-832Too many WARN messages about deprecated memorty config variables in JobTacker logMajor.Karam Singhrahul k singh
MAPREDUCE-818org.apache.hadoop.mapreduce.Counters.getGroup returns null if the group name doesnt exist.Minor.Amareshwari SriramadasuAmareshwari Sriramadasu
MAPREDUCE-807Stray user files in mapred.system.dir with permissions other than 777 can prevent the jobtracker from starting up.BlockerjobtrackerAmar KamatAmar Kamat
MAPREDUCE-805Deadlock in JobtrackerMajor.Michael TammAmar Kamat
MAPREDUCE-796Encountered “ClassCastException” on tasktracker while running wordcount with MultithreadedMapRunnerMajorexamplesSuman SehgalAmar Kamat
MAPREDUCE-745TestRecoveryManager fails sometimesMajorjobtrackerAmareshwari SriramadasuAmar Kamat
MAPREDUCE-735ArrayIndexOutOfBoundsException is thrown by KeyFieldBasedPartitionerMajor.Suman SehgalAmar Kamat
MAPREDUCE-687TestMiniMRMapRedDebugScript fails sometimesMajortestAmar KamatAmareshwari Sriramadasu
MAPREDUCE-657CompletedJobStatusStore hardcodes filesystem to hdfsMajorjobtrackerAmar KamatAmar Kamat
MAPREDUCE-565Partitioner does not work with new APIBlockertaskJothi PadmanabhanOwen O'Malley
MAPREDUCE-430Task stuck in cleanup with OutOfMemoryErrorsMajor.Amareshwari SriramadasuAmar Kamat
MAPREDUCE-421mapred pipes might return exit code 0 even when failingMajorpipesChristian KunzChristian Kunz
MAPREDUCE-383pipes combiner does not reset properly after a spillMajor.Christian KunzChristian Kunz
MAPREDUCE-179setProgress not called for new RecordReadersBlocker.Chris DouglasChris Douglas
MAPREDUCE-130Delete the jobconf copy from the log directory of the JobTracker when the job is retiredMajor.Devaraj DasAmar Kamat
MAPREDUCE-124When abortTask of OutputCommitter fails with an Exception for a map-only job, the task is marked as successMajor.Jothi PadmanabhanAmareshwari Sriramadasu
MAPREDUCE-40Memory management variables need a backwards compatibility option after HADOOP-5881Blocker.Hemanth Yamijalarahul k singh
MAPREDUCE-18Under load the shuffle sometimes gets incorrect dataBlocker.Owen O'MalleyRavi Gummadi
MAPREDUCE-2ArrayOutOfIndex error in KeyFieldBasedPartitioner on empty keyMajor.Amar KamatAmar Kamat

TESTS:

JIRASummaryPriorityComponentReporterContributor

SUB-TASKS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-6213Remove commons dependency on commons-cli2BlockerutilAmar KamatAmar Kamat

OTHER:

JIRASummaryPriorityComponentReporterContributor