Apache Hadoop Changelog

Release 0.14.0 - 2007-08-20

INCOMPATIBLE CHANGES:

JIRASummaryPriorityComponentReporterContributor

IMPORTANT ISSUES:

JIRASummaryPriorityComponentReporterContributor

NEW FEATURES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-1597Distributed upgrade status reporting and post upgrade features.Blocker.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1570Add a per-job configuration knob to control loading of native hadoop librariesMajorioArun C MurthyArun C Murthy
HADOOP-1568NameNode Schema for HttpFileSystemMajorfsChris DouglasChris Douglas
HADOOP-1562Report Java VM metricsMajormetricsDavid BowenDavid Bowen
HADOOP-1515MultiFileSplit, MultiFileInputFormatMajor.Enis SoztutarEnis Soztutar
HADOOP-1508ant Task for FsShell operationsMinorbuild, fsChris DouglasChris Douglas
HADOOP-1469Asynchronous table creationMinor.James Kennedystack
HADOOP-1447Support for textInputFormat in contrib/data_joinMinor.Senthil SubramanianSenthil Subramanian
HADOOP-1437Eclipse plugin for developing and executing MapReduce programs on HadoopMajor.Eugene HungChristophe Taton
HADOOP-1433Add job priorityMinor.Johan OskarssonJohan Oskarsson
HADOOP-1379Integrate Findbugs into nightly build processMajortestNigel DaleyNigel Daley
HADOOP-1377Creation time and modification time for hadoop files and directoriesMajor.dhruba borthakurdhruba borthakur
HADOOP-1134Block level CRCs in HDFSMajor.Raghu AngadiRaghu Angadi
HADOOP-234Hadoop Pipes for writing map/reduce jobs in C++ and pythonMajor.Sanjay DahiyaOwen O'Malley

IMPROVEMENTS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-1653FSDirectory class code cleanupTrivial.Christophe TatonChristophe Taton
HADOOP-1620FileSystem should have fewer abstract methodsMajor.Doug CuttingDoug Cutting
HADOOP-1585GenericWritable should use genericsMinorioEspen Amble KolstadEspen Amble Kolstad
HADOOP-1582hdfsRead and hdfsPread should return 0 instead of -1 at end-of-file.Minor.Christian KunzChristian Kunz
HADOOP-1580provide better error message when subprocesses fail in hadoop streamingMinor.John HeidemannJohn Heidemann
HADOOP-1547Provide examples for aggregate libraryMajor.Tom WhiteRunping Qi
HADOOP-1518Add session id to metric dataMajor.David BowenDavid Bowen
HADOOP-1485Metrics should be there for reporting shuffle failures/successesMajor.Devaraj DasDevaraj Das
HADOOP-1484Kill jobs from web interfaceMinor.Enis SoztutarEnis Soztutar
HADOOP-1478The blockStream of DFSClient.FSInputStream should not be bufferedMajor.Hairong KuangHairong Kuang
HADOOP-1473Make jobids unique across jobtracker restartsMajor.Owen O'MalleyOwen O'Malley
HADOOP-1470Rework FSInputChecker and FSOutputSummer to support checksum code sharing between ChecksumFileSystem and block level crc dfsMajorfsHairong KuangHairong Kuang
HADOOP-1467Remove redundant counters from WordCount exampleMajor.Owen O'MalleyOwen O'Malley
HADOOP-1462Better progress reporting from a TaskMajor.Vivek RatanVivek Ratan
HADOOP-1457Counters for monitoring task assignmentsMinor.Devaraj DasArun C Murthy
HADOOP-1455Allow any key-value pair on the command line of ‘hadoop pipes’ to be added to the JobConfMajor.Christian KunzDevaraj Das
HADOOP-1450checksums should be closer to data generation and consumptionMajorfsDoug CuttingDoug Cutting
HADOOP-1440JobClient should not sort input-splitsMajor.Milind BhandarkarSenthil Subramanian
HADOOP-1438Grammatical / wording / copy edits for Hadoop Distributed File System: Architecture and Design white paperTrivialdocumentationLuke Nezda
HADOOP-1429RPC Server won't go quietlyMinoripcstackstack
HADOOP-1417Exclude some Findbugs detectorsMinorbuildNigel DaleyNigel Daley
HADOOP-1413A new example to do tile placements using distributed dancing linksMajor.Owen O'MalleyOwen O'Malley
HADOOP-1408fix warning about cast of Map<String, Map<String, JobInfo>> in jobhistory.jspMajor.Owen O'MalleyOwen O'Malley
HADOOP-1376RandomWriter should be tweaked to generate input data for terasortMajor.Devaraj DasDevaraj Das
HADOOP-1343Deprecate the Configuration.set(String,Object) method and make Configuration IterableMajorconfOwen O'MalleyOwen O'Malley
HADOOP-1342A configurable limit on the number of unique values should be set on the UniqueValueCount and ValueHistogram aggregatorsMajor.Runping QiRunping Qi
HADOOP-1340md5 file in filecache should inherit replication factor from the file it belongs to.Major.Christian Kunzdhruba borthakur
HADOOP-1320Rewrite ‘random-writer’ to use ‘-reducer NONE’Minor.Arun C MurthyArun C Murthy
HADOOP-1292dfs -copyToLocal should guarantee file is completeMajor.eric baldeschwielerTsz Wo Nicholas Sze
HADOOP-1286Distributed cluster upgradeMajor.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1283Eliminate internal UTF8 to String and vice versa conversions in the name-node.Major.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1147remove all @author tags from sourceMinor.Doug CuttingDoug Cutting
HADOOP-1066http://lucene.apache.org/hadoop/ front page is not user-friendlyMinordocumentationMarco NicosiaDoug Cutting
HADOOP-1028Servers should log startup and shutdown messagesMajor.Nigel DaleyTsz Wo Nicholas Sze
HADOOP-1023better links to mailing list archivesMajordocumentationDaniel NaberTom White
HADOOP-1003Proposal to batch commits to edits log.Major.Raghu Angadidhruba borthakur
HADOOP-894dfs client protocol should allow asking for parts of the block mapMajor.Owen O'MalleyKonstantin Shvachko
HADOOP-800More improvements to DFS browsing WIMajor.arkady borkovskyEnis Soztutar

BUG FIXES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-1717TestDFSUpgradeFromImage fails on SolarisBlockertestNigel DaleyRaghu Angadi
HADOOP-1716TestPipes.testPipes failsBlocker.Nigel DaleyOwen O'Malley
HADOOP-1714TestDFSUpgradeFromImage fails on WindowsBlockertestNigel DaleyRaghu Angadi
HADOOP-1712Unhandled exception in Block CRC upgrade on datanode.Blocker.Raghu AngadiRaghu Angadi
HADOOP-16987500+ reducers/partitions causes job to hangBlocker.Srikanth KakaniDevaraj Das
HADOOP-1681Re organize StreamJob::submitAndMonitorJob() Exception handlingMinor.Lohit Vijayarenu
HADOOP-1680Improvements to Block CRC upgrade messagesBlocker.Raghu AngadiRaghu Angadi
HADOOP-1668add INCOMPATIBLE CHANGES section to CHANGES.txt for Hadoop 0.14BlockerdocumentationNigel DaleyNigel Daley
HADOOP-1666The FsShell Object cannot be used for multiple fs commands.Minorfsdhruba borthakurdhruba borthakur
HADOOP-1665DFS Trash feature bugsBlocker.Nigel Daleydhruba borthakur
HADOOP-1663streaming returning 0 when submitJob fails with ExceptionMajor.Koji NoguchiKoji Noguchi
HADOOP-1659job id / job name mix-upBlocker.Christian KunzArun C Murthy
HADOOP-1657NNBench benchmark hangs with trunkBlocker.Raghu AngadiRaghu Angadi
HADOOP-1649Performance regression with Block CRCsBlocker.Raghu AngadiRaghu Angadi
HADOOP-1647DistributedFileSystem.getFileStatus() fails for path “/”Blocker.Enis Soztutardhruba borthakur
HADOOP-1640TestDecommission fails on WindowsBlocker.Nigel Daleydhruba borthakur
HADOOP-1639TestSymLink is failing fairly often and is blocking the regressionMajor.Owen O'MalleyMahadev konar
HADOOP-1638Master node unable to bind to DNS hostnameMinorcontrib/cloudStu Hood
HADOOP-1635Keypair Name HardcodedMinorcontrib/cloudStu Hood
HADOOP-1634EC2 launch-hadoop-cluster awk ProblemMinorcontrib/cloudStu Hood
HADOOP-1632IllegalArgumentException in fsckBlocker.Konstantin ShvachkoHairong Kuang
HADOOP-1625“could not move files” exception in DataXceiverBlocker.Konstantin ShvachkoRaghu Angadi
HADOOP-1624Unknown op code exception in DataXceiver.Blocker.Konstantin ShvachkoRaghu Angadi
HADOOP-1623dfs -cp infinite loop creating sub-directoriesBlocker.Koji Noguchidhruba borthakur
HADOOP-1619FSInputChecker attempts to seek past EOFBlockerfsNigel DaleyHairong Kuang
HADOOP-1613The dfs webui (dfshealth) shows “Last Contact” as a negative numberMinor.dhruba borthakurdhruba borthakur
HADOOP-1603Replication gets set to 1 sometimes when Namenode restarted.Blocker.Raghu AngadiRaghu Angadi
HADOOP-1599TestCopyFiles with IllegalArgumentException on WindowsBlockerfsNigel DaleySenthil Subramanian
HADOOP-1596TestSymLink is failingBlocker.Doug CuttingOwen O'Malley
HADOOP-1590Jobtracker web interface contains several absolute href links instead of relative onesMajor.Thomas Friol
HADOOP-1587Tasks run by MiniMRCluster don't get sysprops from TestCasesBlockertestAlejandro AbdelnurDevaraj Das
HADOOP-1584Bug in readFields of GenericWritableMinorioEspen Amble KolstadEspen Amble Kolstad
HADOOP-1578Data-nodes should send storage ID to the name-node during registrationBlocker.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1576web interface inconsistencies when using speculative executionBlocker.Christian KunzArun C Murthy
HADOOP-1564Write unit tests to detect CRC corruptionMajor.dhruba borthakurdhruba borthakur
HADOOP-15569 unit test failures: file.out.index already existsMajor.Nigel DaleyDevaraj Das
HADOOP-1554Fix the JobHistory to display things like the number of nodes the job ran on, the number of killed/failed tasksMajor.Devaraj DasDevaraj Das
HADOOP-1553Extensive logging of C++ application can slow down task by an order of magnitudeBlocker.Christian KunzOwen O'Malley
HADOOP-1551libhdfs API is out of sync with Filesystem APIBlocker.Christian KunzSameer Paranjpye
HADOOP-1546The DFS WebUI shows an incorrect column for file Creatin TimeMajor.dhruba borthakurdhruba borthakur
HADOOP-1542Incorrect task/tip being scheduled (looks like speculative execution)Blocker.Nigel DaleyOwen O'Malley
HADOOP-1536libhdfs tests failingBlocker.Nigel Daleydhruba borthakur
HADOOP-1535Wrong comparator used to merge files in Reduce phaseMajor.Vivek RatanVivek Ratan
HADOOP-1524Task Logs userlogs don't show up for a whileMajor.Michael BieniosekMichael Bieniosek
HADOOP-1520IndexOutOfBoundsException in FSEditLog.processIOErrorBlocker.Nigel Daleydhruba borthakur
HADOOP-1517Three methods in FSNamesystem should not be synchronized.Critical.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1514Progress reporting not handled for the case where a reducer currently doesn't have anything to fetchBlocker.Devaraj DasVivek Ratan
HADOOP-1513A likely race condition between the creation of a directory and checking for its existence in the DiskChecker classCriticalfsDevaraj DasDevaraj Das
HADOOP-1512TestTextInputFormat fails on WindowsMajor.Konstantin Shvachko
HADOOP-1504terminate-hadoop-cluster may be overzealousBlockerfs/s3Doug CuttingTom White
HADOOP-1503Fix for broken build by HADOOP-1498Major.stackstack
HADOOP-1501Block reports from all datanodes arrive at the namenode within a small band of timeMajor.dhruba borthakurdhruba borthakur
HADOOP-1493possible double setting of java.library.path introduced by HADOOP-838Major.Enis SoztutarEnis Soztutar
HADOOP-1492DataNode version mismatch during handshake() causes NullPointerException.Major.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1489Input file get truncated for text files with \r\nMajorioBwolen Yang
HADOOP-1486ReplicationMonitor thread goes awayBlocker.Koji Noguchidhruba borthakur
HADOOP-1482SecondaryNameNode does not roll portsMajor.dhruba borthakurdhruba borthakur
HADOOP-1479NPE in HStore#get if StoreFile only has keys < than passed keyMinor.stackstack
HADOOP-1475local filecache disappearsBlocker.Christian KunzOwen O'Malley
HADOOP-1472Timed-out tasks are marked as ‘KILLED’ rather than as ‘FAILED’ which means the framework doesn't fail a TIP with 4 or more timed-out attemptsBlocker.Arun C MurthyArun C Murthy
HADOOP-1461Corner-case deadlock in TaskTrackerBlocker.Arun C MurthyArun C Murthy
HADOOP-1459FileSystem.getFileCacheHints returns IP addresses rather than hostnames, which breaks ‘data-locality’ in map-reduceBlocker.Arun C Murthydhruba borthakur
HADOOP-1456TestDecommission fails with assertion Number of replicas for block1 expected:<3> but was:<2>Major.dhruba borthakurdhruba borthakur
HADOOP-1453exists() not necessary before DFS.openMajor.Raghu AngadiRaghu Angadi
HADOOP-1448Setting the replication factor of a file too high causes namenode cpu overloadMajor.dhruba borthakurHairong Kuang
HADOOP-1446Metrics from the TaskTracker are updated only when map/reduce tasks start/end/failMajor.Devaraj DasDevaraj Das
HADOOP-1444Block allocation method does not check pendingCreates for duplicate block idsMajor.dhruba borthakurdhruba borthakur
HADOOP-1443TestFileCorruption fails with ArrayIndexOutOfBoundsExceptionCritical.Nigel DaleyKonstantin Shvachko
HADOOP-1442Zero-byte input files are not included in InputSplitMajor.Milind BhandarkarSenthil Subramanian
HADOOP-1428ChecksumFileSystem : some operations implicitly not supported.MajorfsRaghu Angadi
HADOOP-1414Findbugs - Bad PracticeMajor.dhruba borthakurdhruba borthakur
HADOOP-1412FindBugs: Dodgy bugs in fs, filecache, io, and util packagesMajor.Hairong KuangHairong Kuang
HADOOP-1406Metrics based on Map-Reduce Counters are not cleaned upMajor.David BowenDavid Bowen
HADOOP-1400JobClient rpc times out getting job statusBlocker.Nigel DaleyOwen O'Malley
HADOOP-1396FileNotFound exception on DFS blockBlocker.Devaraj Dasdhruba borthakur
HADOOP-1394FindBugs : Performance : in dfsMajor.Raghu AngadiRaghu Angadi
HADOOP-1393using Math.abs(Random.getInt()) does not guarantee a positive numberMajor.Owen O'MalleyOwen O'Malley
HADOOP-1392FindBugs : Fix some correctness bugs reported in DFS, FS, etc.Major.Raghu AngadiRaghu Angadi
HADOOP-1390Inconsistent Synchronization cleanup for {Configuration, TaskLog, MapTask, Server}.javaMinorconf, ipcDevaraj DasDevaraj Das
HADOOP-1387FindBugs -> PerformanceMajor.Arun C MurthyArun C Murthy
HADOOP-1372DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocatorMajor.Devaraj Dasdhruba borthakur
HADOOP-1367Inconsistent synchronization of NetworkTopology.distFrom; locked 50% of timeMajorioNigel DaleyHairong Kuang
HADOOP-1364Inconsistent synchronization of SequenceFile$Reader.noBufferedValues; locked 66% of timeMinorioNigel DaleyOwen O'Malley
HADOOP-1359Variable dereferenced then later checked for nullMinor.Nigel DaleyHairong Kuang
HADOOP-1357Call to equals() comparing different types in CopyFiles.cleanup(Configuration, JobConf, String, String)MinorutilNigel DaleyArun C Murthy
HADOOP-1355Possible null pointer dereference in TaskLogAppender.append(LoggingEvent)Minor.Nigel DaleyArun C Murthy
HADOOP-1344getJobName not accessible from JobClientMinor.Michael Bieniosek
HADOOP-1335C++ reducers under hadoop-pipes are not started when there are no key-value pairs to be reducedMajor.Christian KunzOwen O'Malley
HADOOP-1311Bug in BytesWritable.set(byte[] newData, int offset, int length)MajorioSrikanth Kakanidhruba borthakur
HADOOP-1300deletion of excess replicas does not take into account ‘rack-locality’Major.Koji NoguchiHairong Kuang
HADOOP-1285ChecksumFileSystem : Can't read when io.file.buffer.size < bytePerChecksumMajorfsRaghu Angadi
HADOOP-1269DFS Scalability: namenode throughput impacted becuase of global FSNamesystem lockMajor.dhruba borthakurdhruba borthakur
HADOOP-1261Restart of the same data-node should not generate edits log records.Minor.Konstantin ShvachkoRaghu Angadi
HADOOP-1234map tasks fail because they do not find application in file cacheMajor.Christian KunzArun C Murthy
HADOOP-1226makeQualified should return an instance of a DfsPath when passed a DfsPathMajor.Koji Noguchidhruba borthakur
HADOOP-1207hdfs -rm should NOT fail if one of the files to remove is missingMinor.arkady borkovskyTsz Wo Nicholas Sze
HADOOP-1197The misleading Configuration.set(String, Object) should be removedMinorconfOwen O'MalleyOwen O'Malley
HADOOP-1193Map/reduce job gets OutOfMemoryException when set map out to be compressedBlocker.Hairong KuangArun C Murthy
HADOOP-1139All block trasitions should be logged at log level INFOMajor.dhruba borthakurdhruba borthakur
HADOOP-1124ChecksumFileSystem does not handle ChecksumError correctlyMajorfsHairong KuangHairong Kuang
HADOOP-1084updating a hdfs file, doesn't cause the distributed file cache to update itselfBlocker.Owen O'MalleyArun C Murthy
HADOOP-911Multithreading issue with libhdfs libraryBlocker.Christian KunzChristian Kunz
HADOOP-690NPE in jobcontrolMajor.Johan OskarssonOwen O'Malley
HADOOP-71The SequenceFileRecordReader uses the default FileSystem rather than the supplied oneMajor.Owen O'MalleyOwen O'Malley

TESTS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-1629Block CRC Unit Tests: upgrade testBlocker.Nigel DaleyRaghu Angadi
HADOOP-1628Block CRC Unit Tests: protocol testsBlocker.Nigel DaleyRaghu Angadi

SUB-TASKS:

JIRASummaryPriorityComponentReporterContributor

OTHER:

JIRASummaryPriorityComponentReporterContributor
HADOOP-1449Example for contrib/data_joinMinor.Senthil SubramanianSenthil Subramanian
HADOOP-1336turn on speculative execution by defaulMajor.Owen O'MalleyOwen O'Malley