Apache Hadoop Changelog

Release 0.23.3 - 2012-09-20

INCOMPATIBLE CHANGES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-8551fs -mkdir creates parent directories without the -p optionMajorfsRobert Joseph EvansJohn George
HDFS-3318Hftp hangs on transfers >2GBBlockerhdfs-clientDaryn SharpDaryn Sharp
MAPREDUCE-4311Capacity scheduler.xml does not accept decimal values for capacity and maximum-capacity settingsMajorcapacity-sched, mrv2Thomas GravesKarthik Kambatla
MAPREDUCE-4072User set java.library.path seems to overwrite default creating problems native lib loadingMajormrv2Anupam SethAnupam Seth
MAPREDUCE-3812Lower default allocation sizes, fix allocation configurations and document themMajormrv2, performanceVinod Kumar VavilapalliHarsh J

IMPORTANT ISSUES:

JIRASummaryPriorityComponentReporterContributor

NEW FEATURES:

JIRASummaryPriorityComponentReporterContributor
HDFS-2978The NameNode should expose name dir statuses via JMXMajornamenodeAaron T. MyersAaron T. Myers
MAPREDUCE-3773Add queue metrics with buckets for job run timesMajorjobtrackerOwen O'MalleyOwen O'Malley

IMPROVEMENTS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-8700Move the checksum type constants to an enumMinorutilTsz Wo Nicholas SzeTsz Wo Nicholas Sze
HADOOP-8635Cannot cancel paths registered deleteOnExitCriticalfsDaryn SharpDaryn Sharp
HADOOP-8535Cut hadoop build times in half (upgrade maven-compiler-plugin to 2.5.1)MajorbuildJonathan EaglesJonathan Eagles
HADOOP-8525Provide Improved Traceability for ConfigurationTrivial.Robert Joseph EvansRobert Joseph Evans
HADOOP-8373Port RPC.getServerAddress to 0.23MajoripcDaryn SharpDaryn Sharp
HADOOP-8335Improve Configuration's address handlingMajorutilDaryn SharpDaryn Sharp
HADOOP-8286Simplify getting a socket address from confMajorconfDaryn SharpDaryn Sharp
HADOOP-8242AbstractDelegationTokenIdentifier: add getter methods for owner and realuserMinor.Colin Patrick McCabeColin Patrick McCabe
HADOOP-8240Allow users to specify a checksum type on create()MajorfsKihwal LeeKihwal Lee
HADOOP-8239Extend MD5MD5CRC32FileChecksum to show the actual checksum type being usedMajorfsKihwal LeeKihwal Lee
HADOOP-8227Allow RPC to limit ephemeral port range.Blocker.Robert Joseph EvansRobert Joseph Evans
HADOOP-8141Add method to init krb5 cipher suitesTrivialsecurityTodd LipconTodd Lipcon
HADOOP-7510Tokens should use original hostname provided instead of ipMajorsecurityDaryn SharpDaryn Sharp
HDFS-2652Port token service changes from 205Major.Daryn SharpDaryn Sharp
HDFS-2421Improve the concurrency of SerialNumberMap in NameNodeMajornamenodeHairong KuangJing Zhao
HDFS-208name node should warn if only one dir is listed in dfs.name.dirMinornamenodeAllen WittenauerUma Maheswara Rao G
MAPREDUCE-4614Simplify debugging a job's tokensMajorclient, taskDaryn SharpDaryn Sharp
MAPREDUCE-4375Show Configuration Tracability in MR UIMajorapplicationmasterRobert Joseph EvansRobert Joseph Evans
MAPREDUCE-4301Dedupe some strings in MRAM for memory savingsMajorapplicationmasterRobert Joseph EvansRobert Joseph Evans
MAPREDUCE-4283Display tail of aggregated logs by defaultMajorjobhistoryserver, mrv2Jason LoweJason Lowe
MAPREDUCE-4210Expose listener address for WebAppMajorwebappsDaryn SharpDaryn Sharp
MAPREDUCE-4205retrofit all JVM shutdown hooks to use ShutdownHookManagerMajormrv2Alejandro AbdelnurAlejandro Abdelnur
MAPREDUCE-4190Improve web UI for task attempts userlog linkMajormrv2, webappsThomas GravesThomas Graves
MAPREDUCE-4151RM scheduler web page should filter apps to those that are relevant to schedulingMajormrv2, webappsJason LoweJason Lowe
MAPREDUCE-4079Allow MR AppMaster to limit ephemeral port range.Blockermr-am, mrv2Robert Joseph EvansRobert Joseph Evans
MAPREDUCE-4059The history server should have a separate pluggable storage/query interfaceMajormrv2Robert Joseph EvansRobert Joseph Evans
MAPREDUCE-4017Add jobname to jobsummary logTrivialjobhistoryserver, jobtrackerKoji NoguchiThomas Graves
MAPREDUCE-3935Annotate Counters.Counter and Counters.Group as @PublicMajorclientTom WhiteTom White
MAPREDUCE-3907Document entries mapred-default.xml for the jobhistory server.MinordocumentationEugene KoontzEugene Koontz
MAPREDUCE-3906Fix inconsistency in documentation regarding mapreduce.jobhistory.principalTrivialdocumentationEugene KoontzEugene Koontz
MAPREDUCE-3871Allow symlinking in LocalJobRunner DistributedCacheMajordistributed-cacheTom WhiteTom White
MAPREDUCE-3850Avoid redundant calls for tokens in TokenCacheMajorsecurityDaryn SharpDaryn Sharp
MAPREDUCE-3842stop webpages from automatic refreshingCriticalmrv2, webappsAlejandro AbdelnurThomas Graves
MAPREDUCE-3659Host-based token supportMajorsecurityDaryn SharpDaryn Sharp

BUG FIXES:

JIRASummaryPriorityComponentReporterContributor
HADOOP-8727Gracefully deprecate dfs.umaskmode in 2.x onwardsMajorconfHarsh JHarsh J
HADOOP-8726The Secrets in Credentials are not available to MR tasksMajorsecurityBenoy AntonyDaryn Sharp
HADOOP-8725MR is broken when security is offBlockersecurityDaryn SharpDaryn Sharp
HADOOP-8709globStatus changed behavior from 0.20/1.xCriticalfsJason LoweJason Lowe
HADOOP-8703distcpV2: turn CRC checking off for 0 byte sizeMajor.Dave ThompsonDave Thompson
HADOOP-8697TestWritableName fails intermittently with JDK7MajortestTrevor RobinsonTrevor Robinson
HADOOP-8695TestPathData fails intermittently with JDK7MajortestTrevor RobinsonTrevor Robinson
HADOOP-8693TestSecurityUtil fails intermittently with JDK7MajortestTrevor RobinsonTrevor Robinson
HADOOP-8692TestLocalDirAllocator fails intermittently with JDK7MajortestTrevor RobinsonTrevor Robinson
HADOOP-8637FilterFileSystem#setWriteChecksum is brokenCriticalfsDaryn SharpDaryn Sharp
HADOOP-8634Ensure FileSystem#close doesn't squawk for deleteOnExit pathsCriticalfsDaryn SharpDaryn Sharp
HADOOP-8633Interrupted FsShell copies may leave tmp filesCriticalfsDaryn SharpDaryn Sharp
HADOOP-8627FS deleteOnExit may delete the wrong pathCriticalfsDaryn SharpDaryn Sharp
HADOOP-8614IOUtils#skipFully hangs forever on EOFMinor.Colin Patrick McCabeColin Patrick McCabe
HADOOP-8613AbstractDelegationTokenIdentifier#getUser() should set token auth typeCritical.Daryn SharpDaryn Sharp
HADOOP-8611Allow fall-back to the shell-based implementation when JNI-based users-group mapping failsMajorsecurityKihwal LeeRobert Parker
HADOOP-8606FileSystem.get may return the wrong filesystemMajorfsDaryn SharpDaryn Sharp
HADOOP-8599Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of fileMajorfsAndrey KlochkovAndrey Klochkov
HADOOP-8587HarFileSystem access of harMetaCache isn't threadsafeMinorfsEli CollinsEli Collins
HADOOP-8573Configuration tries to read from an inputstream resource multiple times.MajorconfRobert Joseph EvansRobert Joseph Evans
HADOOP-8550hadoop fs -touchz automatically created parent directoriesMajorfsRobert Joseph EvansJohn George
HADOOP-8543Invalid pom.xml files on 0.23 branchMajorbuildRadim KolarRadim Kolar
HADOOP-8501Gridmix fails to compile on OpenJDK7u4MajorbenchmarksRadim KolarRadim Kolar
HADOOP-8495Update Netty to avoid leaking file descriptors during shuffleCriticalbuildJason LoweJason Lowe
HADOOP-8450Remove src/test/systemTrivialtestColin Patrick McCabeEli Collins
HADOOP-8393hadoop-config.sh missing variable exports, causes Yarn jobs to fail with ClassNotFoundException MRAppMasterMajorscriptsPatrick HuntPatrick Hunt
HADOOP-8390TestFileSystemCanonicalization fails with JDK7MajortestTrevor RobinsonTrevor Robinson
HADOOP-8370Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not foundMajornativeTrevor RobinsonTrevor Robinson
HADOOP-8342HDFS command fails with exception following merge of HADOOP-8325MajorfsRandy ClaytonAlejandro Abdelnur
HADOOP-8341Fix or filter findbugs issues in hadoop-toolsMajor.Robert Joseph EvansRobert Joseph Evans
HADOOP-8334HttpServer sometimes returns incorrect portMajor.Daryn SharpDaryn Sharp
HADOOP-8330TestSequenceFile.testCreateUsesFsArg() is brokenMinortestJohn GeorgeJohn George
HADOOP-8328Duplicate FileSystem Statistics object for ‘file’ schemeMajorfsTom WhiteTom White
HADOOP-8327distcpv2 and distcpv1 jars should not coexistMajor.Dave ThompsonDave Thompson
HADOOP-8325Add a ShutdownHookManager to be used by different components instead of the JVM shutdownhookCriticalfsAlejandro AbdelnurAlejandro Abdelnur
HADOOP-8317Update maven-assembly-plugin to 2.3 - fix build on FreeBSDMajorbuildRadim Kolar
HADOOP-8305distcp over viewfs is brokenMajorviewfsJohn GeorgeJohn George
HADOOP-8288Remove references of mapred.child.ulimit etc. since they are not being used any moreMajor.Ravi PrakashRavi Prakash
HADOOP-8225DistCp fails when invoked by OozieBlockersecurityMithun RadhakrishnanDaryn Sharp
HADOOP-8197Configuration logs WARNs on every use of a deprecated keyCriticalconfAlejandro AbdelnurAlejandro Abdelnur
HADOOP-8180Remove hsqldb since its not needed from pom.xmlMajor.Ravi PrakashRavi Prakash
HADOOP-8179risk of NPE in CopyCommands processArguments()MinorfsSteve LoughranDaryn Sharp
HADOOP-8172Configuration no longer sets all keys in a deprecated key list.CriticalconfRobert Joseph EvansAnupam Seth
HADOOP-8167Configuration deprecation logic breaks backwards compatibilityBlockerconfAlejandro AbdelnurAlejandro Abdelnur
HADOOP-8144pseudoSortByDistance in NetworkTopology doesn't work properly if no local node and first node is local rack nodeMinorioJunping DuJunping Du
HADOOP-8129ViewFileSystemTestSetup setupForViewFileSystem is erring when the user's home directory is somewhere other than /home (eg. /User) etc.Majorfs, testRavi PrakashAhmed Radwan
HADOOP-8110TestViewFsTrash occasionally failsMajorfsTsz Wo Nicholas SzeJason Lowe
HADOOP-8104Inconsistent Jackson versionsMajor.Colin Patrick McCabeAlejandro Abdelnur
HADOOP-8088User-group mapping cache incorrectly does negative caching on transient failuresMajorsecurityKihwal LeeKihwal Lee
HADOOP-8060Add a capability to discover and set checksum types per file.Majorfs, utilKihwal LeeKihwal Lee
HADOOP-8014ViewFileSystem does not correctly implement getDefaultBlockSize, getDefaultReplication, getContentSummaryMajorfsDaryn SharpJohn George
HADOOP-8005Multiple SLF4J binding message in .out file for all daemonsMajorscriptsJoe CrobakJason Lowe
HADOOP-7967Need generalized multi-token filesystem supportCriticalfs, securityDaryn SharpDaryn Sharp
HADOOP-7621alfredo config should be in a file not readable by usersCriticalsecurityAlejandro AbdelnurAaron T. Myers
HADOOP-6963Fix FileUtil.getDU. It should not include the size of the directory or follow symbolic linksCriticalfsOwen O'MalleyRavi Prakash
HDFS-3890filecontext mkdirs doesn't apply umask as expectedCritical.Thomas GravesThomas Graves
HDFS-3873Hftp assumes security is disabled if token fetch failsMajorhdfs-clientDaryn SharpDaryn Sharp
HDFS-3861Deadlock in DFSClientBlockerhdfs-clientKihwal LeeKihwal Lee
HDFS-3852TestHftpDelegationToken is broken after HADOOP-8225Majorhdfs-client, securityAaron T. MyersDaryn Sharp
HDFS-3841Port HDFS-3835 to branch-0.23Major.Robert Joseph EvansRobert Joseph Evans
HDFS-3794WebHDFS Open used with Offset returns the original (and incorrect) Content Length in the HTTP Header.MajorwebhdfsRavi PrakashRavi Prakash
HDFS-3788distcp can't copy large files using webhdfs due to missing Content-Length headerCriticalwebhdfsEli CollinsTsz Wo Nicholas Sze
HDFS-3718Datanode won't shutdown because of runaway DataBlockScanner threadCriticaldatanodeKihwal LeeKihwal Lee
HDFS-3696Create files with WebHdfsFileSystem goes OOM when file size is bigCriticalwebhdfsKihwal LeeTsz Wo Nicholas Sze
HDFS-3688Namenode loses datanode hostname if datanode re-registersMajordatanodeJason LoweJason Lowe
HDFS-3646LeaseRenewer can hold reference to inactive DFSClient instances foreverCriticalhdfs-clientKihwal LeeKihwal Lee
HDFS-3622Backport HDFS-3541 to branch-0.23Major.Robert Joseph EvansRobert Joseph Evans
HDFS-3603Decouple TestHDFSTrash from TestTrashMajortestJason LoweJason Lowe
HDFS-3597SNN can fail to start on upgradeMinornamenodeAndy IsaacsonAndy Isaacson
HDFS-3591Backport HDFS-3357 to branch-0.23Major.Robert Joseph EvansRobert Joseph Evans
HDFS-3581FSPermissionChecker#checkPermission sticky bit check missing range checkMajornamenodeEli CollinsEli Collins
HDFS-3577WebHdfsFileSystem can not read files larger than 24KBBlockerwebhdfsAlejandro AbdelnurTsz Wo Nicholas Sze
HDFS-3486offlineimageviewer can't read fsimage files that contain persistent delegation tokensMinorsecurity, toolsColin Patrick McCabeColin Patrick McCabe
HDFS-3442Incorrect count for Missing Replicas in FSCK reportMinor.suja sAndrew Wang
HDFS-3359DFSClient.close should close cached socketsCriticalhdfs-clientTodd LipconTodd Lipcon
HDFS-3334ByteRangeInputStream leaks streamsMajorhdfs-clientDaryn SharpDaryn Sharp
HDFS-3331setBalancerBandwidth do not checkSuperuserPrivilegeMajornamenodeTsz Wo Nicholas SzeTsz Wo Nicholas Sze
HDFS-3321Error message for insufficient data nodes to come out of safemode is wrong.Major.Ravi PrakashRavi Prakash
HDFS-3312Hftp selects wrong token serviceBlockerhdfs-clientDaryn SharpDaryn Sharp
HDFS-3308hftp/webhdfs can't get tokens if authority has no portCriticalwebhdfsDaryn SharpDaryn Sharp
HDFS-3177Allow DFSClient to find out and use the CRC type being used for a file.Majordatanode, hdfs-clientKihwal LeeKihwal Lee
HDFS-3176JsonUtil should not parse the MD5MD5CRC32FileChecksum bytes on its own.Majorhdfs-clientKihwal LeeKihwal Lee
HDFS-3166Hftp connections do not have a timeoutCriticalhdfs-clientDaryn SharpDaryn Sharp
HDFS-3136Multiple SLF4J binding warningMajorbuildJason LoweJason Lowe
HDFS-3067NPE in DFSInputStream.readBuffer if read is repeated on corrupted blockMajorhdfs-clientHenry RobinsonHenry Robinson
HDFS-3037TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racyMinortestAaron T. MyersAaron T. Myers
HDFS-2285BackupNode should reject requests trying to modify namespaceMajornamenodeKonstantin ShvachkoKonstantin Shvachko
MAPREDUCE-4641Exception in commitJob marks job as successful in job historyMajormrv2Jason LoweJason Lowe
MAPREDUCE-4633history server doesn't set permissions on all subdirsCriticaljobhistoryserverThomas Gravesoss.wakayama
MAPREDUCE-4612job summary file permissions not set when its createdCritical.Thomas GravesThomas Graves
MAPREDUCE-4611MR AM dies badly when Node is decomissionedCritical.Robert Joseph EvansRobert Joseph Evans
MAPREDUCE-4604In mapred-default, mapreduce.map.maxattempts & mapreduce.reduce.maxattempts defaults are set to 4 as well as mapreduce.job.maxtaskfailures.per.tracker.Criticalmrv2Ravi PrakashRavi Prakash
MAPREDUCE-4600TestTokenCache.java from MRV1 no longer compilesCritical.Robert Joseph EvansDaryn Sharp
MAPREDUCE-4570ProcfsBasedProcessTree#constructProcessInfo() prints a warning if procfsDir/<pid>/stat is not found.Minormrv2Ahmed RadwanAhmed Radwan
MAPREDUCE-4569TestHsWebServicesJobsQuery fails on jdk7Major.Thomas GravesThomas Graves
MAPREDUCE-4562Support for “FileSystemCounter” legacy counter group name for compatibility reasons is creating incorrect counter nameMajor.Jarek Jarcec CechoJarek Jarcec Cecho
MAPREDUCE-4504SortValidator writes to wrong directoryMajormrv2Robert Joseph EvansRobert Joseph Evans
MAPREDUCE-4496AM logs link is missing user nameMajorapplicationmaster, mrv2Jason LoweJason Lowe
MAPREDUCE-4493Distibuted Cache Compatability IssuesCriticalmrv2Robert Joseph EvansRobert Joseph Evans
MAPREDUCE-4492Configuring total queue capacity between 100.5 and 99.5 at perticular level is sucessfullMinormrv2Nishan ShettyMayank Bansal
MAPREDUCE-4467IndexCache failures due to missing synchronizationCriticalnodemanagerAndrey KlochkovKihwal Lee
MAPREDUCE-4457mr job invalid transition TA_TOO_MANY_FETCH_FAILURE at FAILEDCriticalmrv2Thomas GravesRobert Joseph Evans
MAPREDUCE-4456LocalDistributedCacheManager can get an ArrayIndexOutOfBounds when creating symlinksMajormrv2Robert Joseph EvansRobert Joseph Evans
MAPREDUCE-4449Incorrect MR_HISTORY_STORAGE property name in JHAdminConfigMajormrv2Ahmed RadwanAhmed Radwan
MAPREDUCE-4448Nodemanager crashes upon application cleanup if aggregation failed to startCriticalmrv2, nodemanagerJason LoweJason Lowe
MAPREDUCE-4444nodemanager fails to start when one of the local-dirs is badBlockernodemanagerNathan RobertsJason Lowe
MAPREDUCE-4437Race in MR ApplicationMaster can cause reducers to never be scheduledCriticalapplicationmaster, mrv2Jason LoweJason Lowe
MAPREDUCE-4423Potential infinite fetching of map outputCriticalmrv2Robert Joseph EvansRobert Joseph Evans
MAPREDUCE-4419./mapred queue -info <queuename> -showJobs displays all the jobs irrespective of <queuename>Majormrv2Nishan ShettyDevaraj K
MAPREDUCE-4402TestFileInputFormat fails intermittentlyMajortestJason LoweJason Lowe
MAPREDUCE-4395Possible NPE at ClientDistributedCacheManager#determineTimestampsCriticaldistributed-cache, job submission, mrv2Bhallamudi Venkata Siva KameshBhallamudi Venkata Siva Kamesh
MAPREDUCE-4392Counters.makeCompactString() changed behavior from 0.20Majormrv2Jason LoweJason Lowe
MAPREDUCE-4387RM gets fatal error and exits during TestRMMajorresourcemanagerKihwal LeeKihwal Lee
MAPREDUCE-4384Race conditions in IndexCacheMajornodemanagerKihwal LeeKihwal Lee
MAPREDUCE-4379Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contextsBlockermrv2, nodemanagerDevaraj KDevaraj K
MAPREDUCE-4376TestClusterMRNotification times outMajormrv2, testJason LoweKihwal Lee
MAPREDUCE-4372Deadlock in Resource Manager between SchedulerEventDispatcher.EventProcessor and Shutdown hook managerMajormrv2, resourcemanagerDevaraj KDevaraj K
MAPREDUCE-4361Fix detailed metrics for protobuf-based RPC on 0.23Majormrv2Jason LoweJason Lowe
MAPREDUCE-4341add types to capacity scheduler properties documentationMajorcapacity-sched, mrv2Thomas GravesKarthik Kambatla
MAPREDUCE-4320gridmix mainClass wrong in pom.xmlMajorcontrib/gridmixThomas GravesThomas Graves
MAPREDUCE-4307TeraInputFormat calls FileSystem.getDefaultBlockSize() without a Path - Failure when using ViewFileSystemMajormrv2Ahmed RadwanAhmed Radwan
MAPREDUCE-4302NM goes down if error encountered during log aggregationCriticalnodemanagerDaryn SharpDaryn Sharp
MAPREDUCE-4300OOM in AM can turn it into a zombie.MajorapplicationmasterRobert Joseph EvansRobert Joseph Evans
MAPREDUCE-4299Terasort hangs with MR2 FifoSchedulerMajormrv2Tom WhiteTom White
MAPREDUCE-4297Usersmap file in gridmix should not fail on empty linesMajorcontrib/gridmixRavi PrakashRavi Prakash
MAPREDUCE-4295RM crashes due to DNS issueCriticalmrv2, resourcemanagerThomas GravesThomas Graves
MAPREDUCE-4290JobStatus.getState() API is giving ambiguous valuesMajormrv2Nishan ShettyDevaraj K
MAPREDUCE-4270data_join test classes are in the wrong packgeMajormrv2Brock NolandThomas Graves
MAPREDUCE-4269documentation: Gridmix has javadoc warnings in StressJobFactoryMajormrv2Jonathan EaglesJonathan Eagles
MAPREDUCE-4267mavenize pipesCriticalmrv2Thomas GravesThomas Graves
MAPREDUCE-4264Got ClassCastException when using mapreduce.history.server.delegationtoken.required=trueBlockermrv2Thomas GravesThomas Graves
MAPREDUCE-4252MR2 job never completes with 1 pending taskMajormrv2Tom WhiteTom White
MAPREDUCE-4250hadoop-config.sh missing variable exports, causes Yarn jobs to fail with ClassNotFoundException MRAppMasterMajornodemanagerPatrick HuntPatrick Hunt
MAPREDUCE-4238mavenize data_joinCriticalmrv2Thomas GravesThomas Graves
MAPREDUCE-4237TestNodeStatusUpdater can fail if localhost has a domain associated with itMajor.Robert Joseph EvansRobert Joseph Evans
MAPREDUCE-4233NPE can happen in RMNMNodeInfo.Critical.Robert Joseph EvansRobert Joseph Evans
MAPREDUCE-4228mapreduce.job.reduce.slowstart.completedmaps is not working properly to delay the scheduling of the reduce tasksMajorapplicationmaster, mrv2Jason LoweJason Lowe
MAPREDUCE-4226ConcurrentModificationException in FileSystemCounterGroupMajormrv2Tom WhiteTom White
MAPREDUCE-4220RM apps page starttime/endtime sorts are incorrectMinormrv2Jonathan EaglesJonathan Eagles
MAPREDUCE-4215RM app page shows 500 error on appid parse errorMajormrv2Jonathan EaglesJonathan Eagles
MAPREDUCE-4211Error conditions (missing appid, appid not found) are masked in the RM app pageMinormrv2Jonathan EaglesJonathan Eagles
MAPREDUCE-4209junit dependency in hadoop-mapreduce-client is missing scope testMajorbuildRadim Kolar
MAPREDUCE-4206Sorting by Last Health-Update on the RM nodes page sorts does not work correctlyMinormrv2Jonathan EaglesJonathan Eagles
MAPREDUCE-4197Include the hsqldb jar in the hadoop-mapreduce tar fileMajor.Ravi PrakashRavi Prakash
MAPREDUCE-4194ConcurrentModificationError in DirectoryCollectionMajormrv2Jonathan EaglesJonathan Eagles
MAPREDUCE-4189TestContainerManagerSecurity is failingCriticalmrv2Devaraj KDevaraj K
MAPREDUCE-4169Container Logs appear in unsorted orderMinormrv2Jonathan EaglesJonathan Eagles
MAPREDUCE-4165Committing is misspelled as commiting in task logsTrivialmrv2Jonathan EaglesJonathan Eagles
MAPREDUCE-4160some mrv1 ant tests fail with timeout - due to 4156MajortestThomas GravesThomas Graves
MAPREDUCE-4159Job is running in Uber mode after setting “mapreduce.job.ubertask.maxreduces” to zeroMajormrv2Nishan ShettyDevaraj K
MAPREDUCE-4156ant build fails compiling JobInProgressMajorbuildThomas GravesThomas Graves
MAPREDUCE-4152map task left hanging after AM dies trying to connect to RMMajormrv2Thomas GravesThomas Graves
MAPREDUCE-4144ResourceManager NPE while handling NODE_UPDATECriticalmrv2Jason LoweJason Lowe
MAPREDUCE-4140mapreduce classes incorrectly importing “clover.org.apache.*” classesMajorclient, mrv2Patrick HuntPatrick Hunt
MAPREDUCE-4139Potential ResourceManager deadlock when SchedulerEventDispatcher is stoppedMajormrv2Jason LoweJason Lowe
MAPREDUCE-4133MR over viewfs is brokenMajor.John GeorgeJohn George
MAPREDUCE-4129Lots of unneeded counters log messagesMajormrv2Ahmed RadwanAhmed Radwan
MAPREDUCE-4128AM Recovery expects all attempts of a completed task to also be completed.Majormrv2Bikas SahaBikas Saha
MAPREDUCE-4117mapred job -status throws NullPointerExceptionCriticalclient, mrv2Devaraj KDevaraj K
MAPREDUCE-4102job counters not available in Jobhistory webui for killed jobsMajorwebappsThomas GravesBhallamudi Venkata Siva Kamesh
MAPREDUCE-4099ApplicationMaster may fail to remove staging directoryCriticalmrv2Jason LoweJason Lowe
MAPREDUCE-4097tools testcases fail because missing mrapp-generated-classpath file in classpathMajorbuildAlejandro AbdelnurRoman Shaposhnik
MAPREDUCE-4092commitJob Exception does not fail job (regression in 0.23 vs 0.20)Blockermrv2Jonathan EaglesJonathan Eagles
MAPREDUCE-4091tools testcases failing because of MAPREDUCE-4082Criticalbuild, testAlejandro AbdelnurAlejandro Abdelnur
MAPREDUCE-4089Hung Tasks never time out.Blockermrv2Robert Joseph EvansRobert Joseph Evans
MAPREDUCE-4082hadoop-mapreduce-client-app's mrapp-generated-classpath file should not be in the module JARCriticalbuildAlejandro AbdelnurAlejandro Abdelnur
MAPREDUCE-4074Client continuously retries to RM When RM goes down before launching Application MasterMajor.Devaraj Kxieguiming
MAPREDUCE-4073CS assigns multiple off-switch containers when using multi-level-queuesCriticalmrv2, schedulerSiddharth SethSiddharth Seth
MAPREDUCE-4068Jars in lib subdirectory of the submittable JAR are not added to the classpathBlockermrv2Ahmed RadwanRobert Kanter
MAPREDUCE-4062AM Launcher thread can hang foreverMajormrv2Thomas GravesThomas Graves
MAPREDUCE-4060Multiple SLF4J binding warningMajorbuildJason LoweJason Lowe
MAPREDUCE-4053Counters group names deprecation is wrong, iterating over group names deprecated names don't show upMajormrv2Alejandro AbdelnurRobert Joseph Evans
MAPREDUCE-4050Invalid node linkMajormrv2Bhallamudi Venkata Siva KameshBhallamudi Venkata Siva Kamesh
MAPREDUCE-4048NullPointerException exception while accessing the Application Master UIMajormrv2Devaraj KDevaraj K
MAPREDUCE-4040History links should use hostname rather than IP address.Minorjobhistoryserver, mrv2Bhallamudi Venkata Siva KameshBhallamudi Venkata Siva Kamesh
MAPREDUCE-4031Node Manager hangs on shut downCriticalmrv2, nodemanagerDevaraj KDevaraj K
MAPREDUCE-4024RM webservices can't query on finalStatusMajormrv2Thomas GravesThomas Graves
MAPREDUCE-4020Web services returns incorrect JSON for deep queue treeMajormrv2, webappsJason LoweAnupam Seth
MAPREDUCE-4012Hadoop Job setup error leaves no useful info to users (when LinuxTaskController is used)Minor.Koji NoguchiThomas Graves
MAPREDUCE-4010TestWritableJobConf fails on trunkCriticalmrv2Jason LoweAlejandro Abdelnur
MAPREDUCE-3999Tracking link gives an error if the AppMaster hasn't started yetMajormrv2, webappsRavi PrakashRavi Prakash
MAPREDUCE-3992Reduce fetcher doesn't verify HTTP status code of responseMajormrv1Todd LipconTodd Lipcon
MAPREDUCE-3988mapreduce.job.local.dir doesn't point to a single directory on a node.Majormrv2Vinod Kumar VavilapalliEric Payne
MAPREDUCE-3947yarn.app.mapreduce.am.resource.mb not documentedMinor.Todd LipconDevaraj K
MAPREDUCE-3932MR tasks failing and crashing the AM when available-resources/headRoom becomes zeroCriticalmr-am, mrv2Vinod Kumar VavilapalliRobert Joseph Evans
MAPREDUCE-3927Shuffle hang when set map.failures.percentCriticalmrv2MengWangBhallamudi Venkata Siva Kamesh
MAPREDUCE-3893allow capacity scheduler configs maximum-applications and maximum-am-resource-percent configurable on a per queue basisCriticalmrv2Thomas GravesThomas Graves
MAPREDUCE-3889job client tries to use /tasklog interface, but that doesn't exist anymoreCriticalmrv2Thomas GravesDevaraj K
MAPREDUCE-3870Invalid App MetricsMajormrv2Bhallamudi Venkata Siva KameshBhallamudi Venkata Siva Kamesh
MAPREDUCE-3782teragen terasort jobs fail when using webhdfs://Criticalmrv2Arpit GuptaJason Lowe
MAPREDUCE-3728ShuffleHandler can't access results when configured in a secure modeCriticalmrv2, nodemanagerRoman ShaposhnikDing Yuan
MAPREDUCE-3682Tracker URL says AM tasks run on localhostMajormrv2David CapwellRavi Prakash
MAPREDUCE-3672Killed maps shouldn't be counted towards JobCounter.NUM_FAILED_MAPSMajormr-am, mrv2Vinod Kumar VavilapalliAnupam Seth
MAPREDUCE-3650testGetTokensForHftpFS() failsBlockermrv2Thomas GravesRavi Prakash
MAPREDUCE-3621TestDBJob and TestDataDrivenDBInputFormat ant tests failMajormrv2Thomas GravesRavi Prakash
MAPREDUCE-3543Mavenize Gridmix.Criticalmrv2Mahadev konarThomas Graves
MAPREDUCE-3506Calling getPriority on JobInfo after parsing a history log with JobHistoryParser throws a NullPointerExceptionMinorclient, mrv2Ratandeep RattiJason Lowe
MAPREDUCE-3350Per-app RM page should have the list of application-attempts like on the app JHS pageCriticalmrv2, webappsVinod Kumar VavilapalliJonathan Eagles
MAPREDUCE-3348mapred job -status fails to give info even if the job is present in HistoryMajormrv2Devaraj KDevaraj K
MAPREDUCE-3082archive command take wrong path for input file with current directoryMajorharchiveRajit SahaJohn George
MAPREDUCE-2374“Text File Busy” errors launching MR tasksMajor.Todd LipconAndy Isaacson
MAPREDUCE-2289Permissions race can make getStagingDir fail on local filesystemMajorjob submissionTodd LipconAhmed Radwan
YARN-87NM ResourceLocalizationService does not set permissions of local cache directoriesCriticalnodemanagerJason LoweJason Lowe
YARN-68NodeManager will refuse to shutdown indefinitely due to container log aggregationMajornodemanagerpatrick whiteDaryn Sharp
YARN-66aggregated logs permissions not set properlyCriticalnodemanagerThomas GravesThomas Graves
YARN-63RMNodeImpl is missing valid transitions from the UNHEALTHY stateMajorresourcemanagerJason LoweJason Lowe
YARN-58NM leaks filesystemsCriticalnodemanagerDaryn SharpJason Lowe
YARN-31TestDelegationTokenRenewer fails on jdk7Major.Thomas GravesThomas Graves
YARN-27Failed refreshQueues due to misconfiguration prevents further refreshing of queuesMajor.Ramya SunilArun C Murthy
YARN-25remove old aggregated logsMajor.Thomas GravesRobert Joseph Evans
YARN-14Symlinks to peer distributed cache files no longer workMajornodemanagerJason LoweJason Lowe

TESTS:

JIRASummaryPriorityComponentReporterContributor
HADOOP-8283Allow tests to control token service valueMajortestDaryn SharpDaryn Sharp
HDFS-3258Test for HADOOP-8144 (pseudoSortByDistance in NetworkTopology for first rack local node)MajortestEli CollinsJunping Du
HDFS-2505Add a test to verify getFileChecksum works with ViewFSMinortestRavi PrakashRavi Prakash
MAPREDUCE-4212TestJobClientGetJob sometimes failsMajortestDaryn SharpDaryn Sharp
MAPREDUCE-3983TestTTResourceReporting can fail, and should just be deletedMajormrv1Robert Joseph EvansRavi Prakash

SUB-TASKS:

JIRASummaryPriorityComponentReporterContributor
HDFS-3168Clean up FSNamesystem and BlockManagerMajornamenodeTsz Wo Nicholas SzeTsz Wo Nicholas Sze
MAPREDUCE-4163consistently set the bind addressMajormrv2Daryn SharpDaryn Sharp
MAPREDUCE-4162Correctly set token serviceMajorclient, mrv2Daryn SharpDaryn Sharp
MAPREDUCE-4161create sockets consistentlyMajorclient, mrv2Daryn SharpDaryn Sharp
MAPREDUCE-3972Locking and exception issues in JobHistory Server.Majormrv2Robert Joseph EvansRobert Joseph Evans
MAPREDUCE-3942Randomize master key generation for ApplicationTokenSecretManager and roll it every so oftenMajormrv2, securityVinod Kumar VavilapalliVinod Kumar Vavilapalli
MAPREDUCE-3940ContainerTokens should have an expiry intervalMajormrv2, securityVinod Kumar VavilapalliVinod Kumar Vavilapalli
MAPREDUCE-3613web service calls header contains 2 content typesCriticalmrv2Thomas GravesThomas Graves
YARN-60NMs rejects all container tokens after secret key rollsBlockernodemanagerDaryn SharpVinod Kumar Vavilapalli
YARN-39RM-NM secret-keys should be randomly generated and rolled every so oftenCritical.Vinod Kumar VavilapalliVinod Kumar Vavilapalli

OTHER:

JIRASummaryPriorityComponentReporterContributor
MAPREDUCE-4134Remove references of mapred.child.ulimit etc. since they are not being used any moreMajormrv2Ravi PrakashRavi Prakash
MAPREDUCE-4051Remove the empty hadoop-mapreduce-project/assembly/all.xml fileMajor.Ravi PrakashRavi Prakash
YARN-1Move YARN out of hadoop-mapreduceMajor.Arun C MurthyArun C Murthy