Apache Hadoop Changelog

Release 0.15.0 - 2007-10-19

INCOMPATIBLE CHANGES:

JIRASummaryPriorityComponentReporterContributor

IMPORTANT ISSUES:

JIRASummaryPriorityComponentReporterContributor

NEW FEATURES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-1963Code contribution of Kosmos Filesystem implementation of Hadoop Filesystem interfaceMajorfsSriram RaoSriram Rao
HADOOP-1914HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenodeMajor.Hairong KuangHairong Kuang
HADOOP-1894Add fancy graphs for mapred task statusesMajor.Enis SoztutarEnis Soztutar
HADOOP-1880SleepJobMajor.Enis SoztutarEnis Soztutar
HADOOP-1851Map output compression codec cannot be set independently of job output compression codecMajor.Riccardo BoscoloArun C Murthy
HADOOP-1822Allow SOCKS proxy configuration to remotely access the DFS and submit JobsMinoripcChristophe TatonChristophe Taton
HADOOP-1809Add link to irc channel #hadoopMajor.Enis SoztutarEnis Soztutar
HADOOP-1727Make ...hbase.io.MapWritable more generic so that it can be included in ...hadoop.ioMinorioJim KellermanJim Kellerman
HADOOP-1351Want to kill a particular task or attemptMajor.Owen O'MalleyEnis Soztutar
HADOOP-789DFS shell should return a list of nodes for a file saying that where the blocks for these files are located.Minor.Mahadev konarMahadev konar

IMPROVEMENTS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-2046Documentation: improve mapred javadocsBlockerdocumentationArun C MurthyArun C Murthy
HADOOP-1971Constructing a JobConf without a class leads to a very misleading error message.Minor.Ted DunningEnis Soztutar
HADOOP-1968Wildcard input syntax (glob) should support {}Majorfseric baldeschwielerHairong Kuang
HADOOP-1942Increase the concurrency of transaction logging to edits logBlocker.dhruba borthakurdhruba borthakur
HADOOP-1933Consider include/exclude files while listing datanodes.Major.Raghu AngadiRaghu Angadi
HADOOP-1926Design/implement a set of compression benchmarks for the map-reduce frameworkMajor.Arun C MurthyArun C Murthy
HADOOP-1921Save the configuration of completed/failed jobs and make them available via the web-ui.Major.Arun C MurthyAmar Kamat
HADOOP-1908Restructure data node code so that block sending/receiving is seperated from data transfer header handlingMajor.Hairong KuangHairong Kuang
HADOOP-1906JobConf should warn about the existance of obsolete mapred-default.xml.MajorconfOwen O'MalleyArun C Murthy
HADOOP-1881Update documentation for hadoop's configuration post HADOOP-785MajordocumentationArun C MurthyArun C Murthy
HADOOP-1878Change priority feature in the job details JSP page misses spaces between each priority linkTrivial.Thomas FriolThomas Friol
HADOOP-1803Generalize making contrib bin content executable in ant package targetMinorbuildstackstack
HADOOP-1779Small INodeDirectory enhancement to get all existing INodes components on a pathTrivial.Christophe TatonChristophe Taton
HADOOP-1777Typo issue in the job details JSP pageTrivial.Thomas FriolThomas Friol
HADOOP-1774Remove use of INode.parent in Block CRC upgradeMajor.Raghu AngadiRaghu Angadi
HADOOP-1767JobClient CLI cleanup and improvementMinor.Christophe TatonChristophe Taton
HADOOP-1766Merging Block and BlockInfo classes on name-node.Major.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1762Namenode does not need to store storageID and datanodeID persistentlyMajor.Raghu AngadiRaghu Angadi
HADOOP-1759File name should be represented by a byte array instead of a StringMajor.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1756Add toString() methods to some Writable typesMajorioAndrzej BialeckiAndrzej Bialecki
HADOOP-1750We should log better if something goes wrong with the process forkMajor.Owen O'MalleyOwen O'Malley
HADOOP-1744Small cleanup of DistributedFileSystem and DFSClient (next)Trivial.Christophe TatonChristophe Taton
HADOOP-1743INode refactoringMajor.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1731contrib jar file names should include hadoop version numberMajor.Doug CuttingDoug Cutting
HADOOP-1718Test coverage target in build files using cloverMajorbuildNigel DaleyNigel Daley
HADOOP-1703Small cleanup of DistributedFileSystem and DFSClientTrivial.Christophe TatonChristophe Taton
HADOOP-1693Remove LOG members from PendingReplicationBlocks and ReplicationTargetChooser.Major.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1687Name-node memory size estimates and optimization proposal.Major.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1667organize CHANGES.txt messages into sections for future releasesMajordocumentationDoug CuttingDoug Cutting
HADOOP-1654IOUtils classMajorioEnis SoztutarEnis Soztutar
HADOOP-1626DFSAdmin. Help messages are missing for -finalizeUpgrade and -metasave.Blocker.Konstantin ShvachkoLohit Vijayarenu
HADOOP-1621Make FileStatus a concrete classMajorfsChris DouglasChris Douglas
HADOOP-1610Add metrics for failed tasksMajor.Devaraj DasDevaraj Das
HADOOP-1595Add an option to setReplication method to wait for completion of replicationMajor.Christian KunzTsz Wo Nicholas Sze
HADOOP-1592Print the diagnostic error messages for FAILED task-attempts to the user console via TaskCompletionEventsMajor.Arun C MurthyAmar Kamat
HADOOP-1500typo's in dfs webuiTrivial.Nigel DaleyNigel Daley
HADOOP-1436Redesign Tool and ToolBase API and releted functionalityMajorutilEnis SoztutarEnis Soztutar
HADOOP-1425Rework the various programs in ‘examples’ to extend ToolBaseMinor.Arun C MurthyEnis Soztutar
HADOOP-1266Remove DatanodeDescriptor dependency from NetworkTopologyMajor.Konstantin ShvachkoHairong Kuang
HADOOP-1231Add generics to Mapper and Reducer interfacesMajor.Owen O'MalleyTom White
HADOOP-1158JobTracker should collect statistics of failed map output fetches, and take decisions to reexecute map tasks and/or restart the (possibly faulty) Jetty server on the TaskTrackerMajor.Devaraj DasArun C Murthy
HADOOP-785Divide the server and client configurationsMajorconfOwen O'MalleyArun C Murthy

BUG FIXES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-2103HADOOP-2046 caused some javadoc anomaliesMajordocumentationNigel DaleyNigel Daley
HADOOP-2102ToolBase doesn't keep configurationBlockerutilDennis KubesDennis Kubes
HADOOP-2080ChecksumFileSystem checksum file size incorrect.BlockerfsRichard LeeOwen O'Malley
HADOOP-2073Datanode corruption if machine dies while writing VERSION fileBlocker.Michael BieniosekKonstantin Shvachko
HADOOP-2072RawLocalFileStatus is causing Path problemsMajorfsDennis Kubes
HADOOP-2070Test org.apache.hadoop.mapred.pipes.TestPipes.unknown failedBlocker.Mukund MadhugiriOwen O'Malley
HADOOP-2051JobTracker's TaskCommitQueue is vulnerable to non-IOExceptionsBlocker.Arun C MurthyArun C Murthy
HADOOP-2048DISTCP mapper should report progress more oftenBlocker.Runping QiChris Douglas
HADOOP-2044Namenode encounters ClassCastException exceptions for INodeFileUnderConstructionBlocker.dhruba borthakurdhruba borthakur
HADOOP-2033In SequenceFile sync doesn't work unless the file is compressed (block or record)Blocker.Owen O'MalleyOwen O'Malley
HADOOP-2031Lost tasktracker not handled properly leading to tips wrongly being kept as completed, and hence not rescheduledBlocker.Devaraj DasDevaraj Das
HADOOP-2028distcp fails if log dir not specified and destination not presentBlockerutilChris DouglasChris Douglas
HADOOP-2026Namenode prints out too many log lines for “Number of transactions”Blocker.dhruba borthakurdhruba borthakur
HADOOP-2023TestLocalDirAllocator fails on WindowsBlockerfsMukund MadhugiriHairong Kuang
HADOOP-2022Task times are not saved correctly (bug in hadoop-1874)Blocker.Devaraj DasAmar Kamat
HADOOP-2018Broken pipe SocketException in DataNode$DataXceiverBlocker.Konstantin ShvachkoHairong Kuang
HADOOP-2016Race condition in removing a KILLED task from tasktrackerBlocker.Devaraj DasArun C Murthy
HADOOP-1997TestCheckpoint fails on WindowsBlocker.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1992Sort validation is taking considerably longer than beforeBlocker.Mukund MadhugiriArun C Murthy
HADOOP-1983jobs using pipes interface with tasks not using java output format have a good chance of not updating progress and timing outMajor.Christian KunzOwen O'Malley
HADOOP-1978Name-node should remove edits.new during startup rather than renaming it to edits.Blocker.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1973NPE at JobTracker startup..Blocker.Gautam KowshikAmareshwari Sriramadasu
HADOOP-1961-get, -copyToLocal fail when single filename is passedBlocker.Koji NoguchiRaghu Angadi
HADOOP-1959Use of File.separator in StatusHttpServer prevents running Junit tests inside eclipse on WindowsMinor.Jim KellermanJim Kellerman
HADOOP-1955Corrupted block replication retries for everBlocker.Koji NoguchiRaghu Angadi
HADOOP-1953the job tracker should wait beteween calls to try and delete the system directoryBlocker.Owen O'MalleyOwen O'Malley
HADOOP-1948Spurious error message during block crc upgrade.Blocker.Raghu AngadiRaghu Angadi
HADOOP-1946du should be not called on every heartbeatBlocker.Konstantin ShvachkoHairong Kuang
HADOOP-1944Maps which ran on trackers declared ‘lost’ are being marked as FAILED rather than KILLEDBlocker.Arun C MurthyDevaraj Das
HADOOP-1940TestDFSUpgradeFromImage doesn't shut down its MiniDFSClusterMajortestChris DouglasChris Douglas
HADOOP-1935NullPointerException in internalReleaseCreateBlocker.Konstantin Shvachkodhruba borthakur
HADOOP-1934the os.name string on Mac OS contains spaces, which causes the c++ compilation to failMajor.Owen O'MalleyOwen O'Malley
HADOOP-1932Test dfs.TestFileCreation.testFileCreation failed on WindowsBlockertestMukund Madhugiridhruba borthakur
HADOOP-1930Too many fetch-failures issueBlocker.Christian KunzArun C Murthy
HADOOP-1925Hadoop Pipes doesn't compile on solarisMajor.Owen O'MalleyOwen O'Malley
HADOOP-1910Extra checks in DFS.create() are not necessary.Minor.Raghu AngadiRaghu Angadi
HADOOP-1907JobClient.runJob kills the job for failed tasks with no diagnosticsMajor.Christian KunzChristian Kunz
HADOOP-1904ArrayIndexOutOfBoundException in BlocksMapBlocker.Konstantin ShvachkoKonstantin Shvachko
HADOOP-1897about.html page is there but not linked.Major.Enis SoztutarEnis Soztutar
HADOOP-1892In the Job UI, some links don't workMajor.Devaraj DasAmar Kamat
HADOOP-1890Revert a debug patch.Trivial.Raghu AngadiRaghu Angadi
HADOOP-1889Fix path in EC2 scripts for building your own AMIMajorcontrib/cloudTom WhiteTom White
HADOOP-1887ArrayIndexOutOfBoundsException with trunkMajor.Raghu Angadidhruba borthakur
HADOOP-1885Race condition in MiniDFSCluster shutdownMajortestChris DouglasChris Douglas
HADOOP-1882Remove extra '*'s from FsShell.limitDecimal()Minor.Raghu AngadiRaghu Angadi
HADOOP-1875multiple dfs.client.buffer.dir directories are not treated as alternativesBlockerfsChristian KunzHairong Kuang
HADOOP-1874lost task trackers -- jobs hangBlocker.Christian KunzDevaraj Das
HADOOP-1846DatanodeReport should distinguish live datanodes from dead datanodesMajor.Hairong KuangHairong Kuang
HADOOP-1840Task's diagnostic messages are lost sometimesCritical.Arun C MurthyArun C Murthy
HADOOP-1838Files created with an pre-0.15 gets blocksize as zero, causing performance degradationBlocker.dhruba borthakurdhruba borthakur
HADOOP-1832listTables() returns duplicate tablesMajor.Andrew HitchcockJim Kellerman
HADOOP-1825hadoop-daemon.sh script fails if HADOOP_PID_DIR doesn't existMinorscriptsMichael BieniosekMichael Bieniosek
HADOOP-1819The JobTracker should ensure that it is running on the right host.Major.Owen O'MalleyOwen O'Malley
HADOOP-1818MutliFileInputFormat returns “empty” MultiFileSplit when number of paths < number of splitsMajor.Thomas FriolThomas Friol
HADOOP-1817MultiFileSplit does not write and read the total lengthMajor.Thomas FriolThomas Friol
HADOOP-1812TestIPC and TestRPC should use dynamically allocated portsMajoripcDoug CuttingDoug Cutting
HADOOP-1810Incorrect Value type in MRBench (SmallJobs)Blocker.Devaraj DasDevaraj Das
HADOOP-1806DfsTask no longer compilesMajorbuildChris DouglasChris Douglas
HADOOP-1795Task.moveTaskOutputs is escaping special characters in output filenamesCritical.Frédéric BertinFrédéric Bertin
HADOOP-1792df command doesn't exist under windowsMajorfsBenjamin FrancisoudMahadev konar
HADOOP-1788Increase the buffer size of pipes from 1k to 128kBlocker.Owen O'MalleyAmareshwari Sriramadasu
HADOOP-1783keyToPath in Jets3tFileSystemStore needs to return absolute pathMajorfs/s3Ahad RanaTom White
HADOOP-1775MapWritable and SortedMapWritable - Writable problemsMajorioJim KellermanJim Kellerman
HADOOP-1772Hadoop does not run in Cygwin in WindowsCriticalscriptsTsz Wo Nicholas SzeTsz Wo Nicholas Sze
HADOOP-1771streaming hang when IOException in MROutputThread. (NPE)Blocker.Koji NoguchiLohit Vijayarenu
HADOOP-1758processing escapes in a jute record is quadraticBlockerrecordDick KingVivek Ratan
HADOOP-1749TestDFSUpgrade some times fails with an assertMajor.Raghu AngadiEnis Soztutar
HADOOP-1748Task Trackers fail to launch tasks when they have relative log directories configuredMajor.Owen O'MalleyOwen O'Malley
HADOOP-1739ConnectException in TaskTracker ChildMajor.Srikanth KakaniDoug Cutting
HADOOP-1708make files visible in the namespace as soon as they are createdMajor.dhruba borthakurdhruba borthakur
HADOOP-1695Secondary Namenode halt when SocketTimeoutException at startupBlocker.Koji Noguchidhruba borthakur
HADOOP-1692DfsTask cache interferes with operationMinorutilChris DouglasChris Douglas
HADOOP-1689.sh scripts do not work on SolarisMinorscriptsDavid BiesackDoug Cutting
HADOOP-1656HDFS does not record the blocksize for a fileMajor.Sameer Paranjpyedhruba borthakur
HADOOP-1651Some improvements in progress reportingMajor.Devaraj DasDevaraj Das
HADOOP-1636constant should be user-configurable: MAX_COMPLETE_USER_JOBS_IN_MEMORYMajor.Michael BieniosekMichael Bieniosek
HADOOP-1601GenericWritable should use ReflectionUtils.newInstance to avoid problems with classloadersMajorioOwen O'MalleyEnis Soztutar
HADOOP-1573Support for 0 reducers in PIPESMajor.Christian KunzOwen O'Malley
HADOOP-1569distcp should use the Path -> FileSystem interface like the rest of HadoopMajorutilOwen O'MalleyChris Douglas
HADOOP-1565DFSScalability: reduce memory usage of namenodeMajor.dhruba borthakurdhruba borthakur
HADOOP-1463dfs.datanode.du.reserved semantics being violatedBlocker.Hairong KuangHairong Kuang
HADOOP-1316“Go to parent directory” does not work on windows.Minor.Konstantin ShvachkoMahadev konar
HADOOP-1076Periodic checkpointing cannot resume if the secondary name-node fails.Major.Konstantin Shvachkodhruba borthakur
HADOOP-1018Single lost heartbeat leads to a “Lost task tracker”Major.Andrzej BialeckiArun C Murthy
HADOOP-999DFS Client should create file when the user creates the fileMajor.Owen O'MalleyTsz Wo Nicholas Sze
HADOOP-932File locking interface and implementation should be remvoed.MinorfsRaghu AngadiRaghu Angadi
HADOOP-795hdfs -cp /a/b/c /x/y acts like hdfs -cp /a/b/c/* /x/yMinor.arkady borkovskyMahadev konar
HADOOP-120Reading an ArrayWriter does not work because valueClass does not get initializedMajorioDick KingCameron Pope
HADOOP-89files are not visible until they are closedCritical.Yoram Arnondhruba borthakur

TESTS:

JIRASummaryPriorityComponentReporterContributor

SUB-TASKS:

JIRASummaryPriorityComponentReporterContributor

OTHER:

JIRASummaryPriorityComponentReporterContributor
HADOOP-1879Warnings With JDK1.6.0_02Minor.Nilay VaishNilay Vaish