Branch-hadoop-1-win (branched from branch-1 at r1293509 on 2012-02-24 17:46:42) | |
HADOOP-8223 Initial patch for branch-1-win. (David Lao via Sanjay) | |
HADOOP-8234 Enable user group mappings on Windows. (Bikas Saha via Sanjay) | |
MAPREDUCE-4201 Getting PID not working on Windows. Termination of | |
Task/TaskJVM's not working. (Bikas Saha via Sanjay) | |
MAPREDUCE-4204 Refactor ProcfsBasedProcessTree to make the resource | |
collection object pluggable. (Bikas Saha via Sanjay) | |
HADOOP-8412 TestModTime, TestDelegationToken and TestAuthenticationToken fail | |
intermittently on Windows. (Ivan Mitic via Sanjay Radia) | |
HADOOP-8235 Support file permissions and ownership on Windows for | |
RawLocalFileSystem. (Chuan Liu via sanjay) | |
HADOOP-8411 TestStorageDirecotyFailure, TestTaskLogsTruncater, | |
TestWebHdfsUrl and TestSecurityUtil fail on Windows. | |
(Ivan Mitic via Sanjay Radia) | |
HADOOP-8440 HarFileSystem.decodeHarURI fails for URIs whose host contains | |
numbers. (Ivan Mitic via Sanjay Radia) | |
HADOOP-8374 Improve support for hard link manipulation on Windows. | |
(Bikas Saha via Sanjay) | |
HDFS-3424 TestDatanodeBlockScanner and TestReplication fail intermittently | |
on Windows (Bikas Saha via Sanjay) | |
HADOOP-8424. Fix classpath so that web-ui works on Windows. (Bikas Saha | |
via acmurthy) | |
MAPREDUCE-4321. Fix both DCE & LCE to use File.getAbsolutePath of the | |
taskjvm.(sh,cmd) to ensure it works on Windows. (Ivan Mitic via acmurthy) | |
MAPREDUCE-4260 Use JobObject to spawn tasks on Windows (Bikas Saha via Sanjay) | |
HADOOP-8440 HarFileSystem.decodeHarURI 2nd patch. | |
(Ivan Mitic via Sanjay Radia) | |
HADOOP-8409 Fix TestCommandLineJobSubmission and TestGenericOptionsParser | |
to work for windows (Ivan Mitic via Sanjay Radia) | |
HADOOP-8454 Fix the ‘chmod =[perm]’ bug in winutils. (Chuan Liu via sanjay) | |
MAPREDUCE-4203. Added an implementation of the process tree for Windows. | |
(Bikas Saha via acmurthy) | |
HADOOP-8486 Resource leak ... SequenceFile. (Kanna Karanam via Sanjay) | |
HADOOP-8534 Some tests leave a config file open causing failure on windows. | |
(Ivan Mitic via Sanjay Radia) | |
HADOOP-8487 Many HDFS tests use a test path intended for local file system | |
tests. (Ivan Mitic via Sanjay Radia) | |
MAPREDUCE-4332 Fix command-length abort issues on Windows. (Ivan Mitic via | |
acmurthy) | |
MAPREDUCE-4369. Fix streaming test failures caused by | |
WindowsResourceCalculatorPlugin. (Bikas Saha via acmurthy) | |
MAPREDUCE-4368. Fix TaskRunner to deal with java.library.path with a | |
quoted path on Windows. (John Gordon via acmurthy) | |
HADOOP-8414 Address problems related to localhost resolving to 127.0.0.1 | |
on Windows. (Ivan Mitic via Sanjay Radia) | |
HADOOP-8544 Move an assertion location in 'winutils chmod'. | |
(Chuan Liu via sanjay) | |
MAPREDUCE-4598. Add support for NodeHealthCheck script for Windows. | |
(Bikas Saha via acmurthy) | |
MAPREDUCE-4510. Fix needless check/logging of getconf in Windows. | |
(Bikas Saha via acmurthy) | |
HADOOP-8453. Added unit tests for winutils. (Chuan Liu via acmurthy) | |
MAPREDUCE-4374. Completed support for configurable environment for child | |
map/reduce tasks. (Chuan Liu via acmurthy) | |
HADOOP-8734. Modify LocalJobRunner to support private distributed cache. | |
(Ivan Mitic via vinodkv) | |
HDFS-3564. Improve BlockPlacementPolicy interface and change | |
BlockPlacementPolicyDefault to make it more extendible. (Sumadhur Reddy | |
Bolli via szetszwo) | |
HDFS-3566. Add AzureBlockPlacementPolicy to handle fault and upgrade | |
domains in Azure. (Sumadhur Reddy Bolli via szetszwo) | |
HADOOP-8911. CRLF characters in source and text files. | |
HDFS-6527. Backport HADOOP-7389: Use of TestingGroups by tests causes | |
subsequent tests to fail. (Ivan Mitic via szetszwo) | |
HADOOP-8618. Fix build failures due to release 1.0.3 merge. | |
(Bikas Saha via suresh) | |
HDFS-3763. TestNameNodeMXBean fails on Windows. (Brandon Li via suresh) | |
HADOOP-8739. Fix command line parsing on Windows. (Bikas Saha via | |
acmurthy) | |
HDFS-3766. Fix TestStorageRestore on Windows. (Brandon Li via suresh) | |
HADOOP-8657. Fix TestCLI to not hardcode file length since it's | |
encoding/platform dependent. (Bikas Saha via acmurthy) | |
MAPREDUCE-4597. Fix intermittent failures in TestKillSubProcesses. | |
(Bikas Saha via acmurthy) | |
HADOOP-8732. Fix intermittent failures in tests by serializing process | |
creation to get around a bug in MS Windows. (Ivan Mitic via acmurthy) | |
HDFS-3833. TestDFSShell fails on windows due to concurrent file | |
read/write. (Brandon Li via suresh) | |
HADOOP-8664. hadoop streaming job need the full path to commands even | |
when they are in the path. (Bikas Saha via suresh) | |
MAPREDUCE-4564. Fix issues with timing out shell processes. (Bikas Saha | |
via acmurthy) | |
HADOOP-8457. Address file ownership issue for users in Admin group on | |
windows. (Ivan Mitic via suresh) | |
MAPREDUCE-4400. Avoid task finish sleep to improve small job/workflow | |
latency. (llu) | |
HADOOP-8733. Fix the tests TestStreamingTaskLog, TestJvmManager | |
and TestLinuxTaskControllerLaunchArgs to not fail on Windows. (Ivan Mitic via | |
vinodkv) | |
HADOOP-8694. Add support for windows native symbolic links. | |
(Chuan Liu via suresh) | |
HADOOP-8836. UGI should throw exception in case winutils.exe cannot be | |
loaded. (Bikas Saha via suresh) | |
MAPREDUCE-4657. WindowsResourceCalculatorPlugin has NPE. | |
(Bikas Saha via suresh) | |
HADOOP-8879. TestUserGroupInformation fails on Windows when run as | |
Administrator. (Bikas Saha via suresh) | |
HADOOP-8880. Missing jersey jars as dependency in the pom causes | |
hive tests to fail. (gkesavan via suresh) | |
HADOOP-8903. support HADOOP_USER_CLASSPATH_FIRST in windows hadoop cmd. | |
(Thejas Nair via suresh) | |
HADOOP-8908. Refactor winutil.exe related code. (Chuan Liu via suresh) | |
HADOOP-8911. CRLF characters in source and text files. | |
(Raja Aluri via suresh) | |
HADOOP-8912. Add .gitattributes file to prevent CRLF and LF mismatches | |
for source and text files. (Raja Aluri via suresh) | |
HADOOP-8868. FileUtil#chmod should normalize the path before calling | |
into shell APIs. (Ivan Mitic via suresh) | |
HADOOP-8936. Local users should be able to query a domain user's | |
groups on Windows. (Chuan Liu via suresh) | |
HADOOP-8869. Links at the bottom of the jobdetails page do not render | |
correctly in IE9. (Ivan Mitic via suresh) | |
HADOOP-8935. Make 'winutils ls' show the SID if the owner does not | |
exist on the system. (Chuan Liu via suresh) | |
HDFS-4065. TestDFSShell.testGet sporadically fails attempting to corrupt | |
block files due to race condition. (Chris Nauroth via suresh) | |
HADOOP-8899. Classpath exceeds maximum Windows command limit. | |
(Ahmed El Baz via suresh) | |
HADOOP-8763. Implement set-group-owner for Windows correctly. (Chuan Liu via | |
vinodkv) | |
HADOOP-8902. Enable Gridmix v1 & v2 benchmarks on Windows platform. | |
(Mike Liddell via suresh) | |
HADOOP-8564. Port and extend Hadoop native libraries for Windows to address | |
datanode concurrent reading and writing issue. (Chuan Liu via suresh) | |
HADOOP-4093. Fix a bug that AzureBlockPlacementPolicy#chooseTarget only | |
returns one DN when replication factor is greater than 3. (Jing Zhao via | |
szetszwo) | |
HADOOP-8907. Provide means to look for zlib1.dll next to hadoop.dll | |
on Windows. (Ivan Mitic via suresh) | |
HADOOP-8872. FileSystem#length returns zero for symlinks on | |
windows+java6. (Ivan Mitic via suresh) | |
HADOOP-8949. Remove FileUtil.CygPathCommand dead code. (Chris Nauroth via | |
suresh) | |
HADOOP-8420. Hadoop Common creating package-info.java must not depend on sh. | |
(Bikas Saha via suresh) | |
HADOOP-8972. Move winutils tests from bat to Java. (Ivan Mitic via suresh) | |
HADOOP-8983. Minor fixes to windows batch scripts to accept alternate | |
config directory location. (Ramya Bharathi Nimmagadda via suresh) | |
HADOOP-9007. TestJobStatusPersistency.testJobStoreDisablingWithInvalidPath | |
failure. (Chuan Liu via suresh) | |
HADOOP-9006. Winutils should keep Administrators privileges intact. | |
(Chuan Liu via suresh) | |
HADOOP-9026. Hadoop.cmd fails to initialize if user's %path% variable has | |
parenthesis. (Ivan Mitic via suresh) | |
HADOOP-9027. Build fails on Windows without sh/sed/echo in the path. | |
(Ivan Mitic via suresh) | |
HADOOP-8847. Change untar to use Java API on Windows instead of spawning | |
tar process. (Bikas Saha via suresh) | |
HADOOP-8456. Support spaces in user names and group names in results | |
returned via winutils. (Ivan Mitic via suresh) | |
HADOOP-9050. Remove java5 dependencies from build. (Raja Aluri via suresh) | |
HADOOP-9062. hadoop-env.cmd overwrites the value of *_OPTS set before | |
install. (Ganeshan Iyer via suresh) | |
HADOOP-9074. Hadoop install scripts for Windows. (Steve Maine, | |
Ramya Nimmagadda, Chuan Liu, Brad Sarsfield and Ivan Mitic via suresh) | |
HADOOP-9110. HADOOP-9110. winutils ls off-by-one error indexing MONTHS array | |
can cause access violation. (Chris Nauroth via suresh) | |
HADOOP-9102. winutils task isAlive does not return a non-zero exit code if | |
the requested task is not alive. (Chris Nauroth via suresh) | |
HADOOP-9061. Java6+Windows does not work well with symlinks. | |
(Ivan Mitic via suresh) | |
HADOOP-8645. HADOOP_HOME and -Dhadoop.home (from hadoop wrapper script) are | |
not uniformly handled. (John Gordon via suresh) | |
HADOOP-9185. TestFileCreation.testFsClose should clean up on exit. | |
(Arpit Agarwal via suresh) | |
HADOOP-9250. Windows installer bugfixes. (Ivan Mitic via suresh) | |
HADOOP-9463. branch-1-win fails to build with OpenJDK7. | |
(Ivan Mitic via suresh) | |
HADOOP-8731. Fixed TrackerDistributedCacheManager to address failure in the | |
test TestTrackerDistributedCacheManager on Windows. (Ivan Mitic via vinodkv) | |
HADOOP-9480. Windows installer should pick the config from | |
src\packages\win\template\conf. (Ivan Mitic via suresh) | |
HADOOP-9452. Windows install scripts bugfixes. (Ivan Mitic via suresh) | |
HADOOP-9472. Cleanup hadoop-config.cmd. (Ivan Mitic via suresh) | |
HADOOP-9043. disallow in winutils creating symlinks with forwards slashes. | |
(Chris Nauroth via suresh) | |
HADOOP-9579. Contrib ant test target not setting the java.library.path. | |
(ivanmi) | |
HADOOP-9552. Windows log4j template should suppress info messages from | |
mortbay.log. (ivanmi) | |
HDFS-4677. Editlog should support synchronous writes. (ivanmi) | |
MAPREDUCE-5224. JobTracker should allow the system directory to be in | |
non-default FS. (Xi Fang via ivanmi) | |
HADOOP-9681. FileUtil.unTarUsingJava() should close the InputStream upon | |
finishing. (Chuan Liu via cnauroth) | |
HADOOP-9677. TestSetupAndCleanupFailure#testWithDFS fails on Windows. | |
(Xi Fang via cnauroth) | |
MAPREDUCE-5330. JVM manager should not forcefully kill the process on | |
Signal.TERM on Windows. (Xi Fang via cnauroth) | |
HADOOP-9687. TestJobTrackerQuiescence and TestFileLengthOnClusterRestart | |
failed caused by incorrect DFS path construction on Windows. (Xi Fang via | |
cnauroth) | |
HADOOP-9685. hadoop-config.cmd: builds a classpath that is too long on | |
windows. (Raja Aluri via cnauroth) | |
MAPREDUCE-5371. TestProxyUserFromEnv#testProxyUserFromEnvironment failed | |
caused by domains of windows users. (Xi Fang via cnauroth) | |
MAPREDUCE-5278. Distributed cache is broken when JT staging dir is not on the | |
default FS. (Xi Fang via cnauroth) | |
HADOOP-9718. Branch-1-win TestGroupFallback#testGroupWithFallback() failed | |
caused by java.lang.UnsatisfiedLinkError. (Xi Fang via cnauroth) | |
HADOOP-9719. Branch-1-win TestFsShellReturnCode#testChgrp() failed caused by | |
incorrect exit codes. (Xi Fang via cnauroth) | |
HADOOP-9722. Branch-1-win TestNativeIO failed caused by Window incompatible | |
test case. (Xi Fang via cnauroth) | |
HDFS-4975. Branch-1-win TestReplicationPolicy failed caused by stale data | |
node handling. (Xi Fang via cnauroth) | |
MAPREDUCE-5391. TestNonLocalJobJarSubmission fails on Windows due to missing | |
classpath entries. (cnauroth) | |
HDFS-5001. TestAzureBlockPlacementPolicy and | |
TestReplicationPolicyWithNodeGroup failed caused by 1) old APIs and | |
2) incorrect value of depthOfAllLeaves. (Xi Fang via cnauroth) | |
HADOOP-9609. Remove sh dependency of bin-package target. (Chuan Liu via | |
cnauroth) | |
HADOOP-9790. Job token path is not unquoted properly. (Xi Fang via cnauroth) | |
HADOOP-9551. Backport common utils introduced with HADOOP-9413 | |
to branch-1-win. (ivanmi) | |
HADOOP-9802. Support Snappy codec on Windows. (cnauroth) | |
HADOOP-9791. Add a test case covering long paths for new FileUtil access | |
check methods (ivanmi) | |
HADOOP-10094. NPE in GenericOptionsParser#preProcessForWindows(). | |
(Enis Soztutar via cnauroth) | |
Merged from branch-1 | |
HDFS-385. Backport: Add support for an experimental API that allows a | |
module external to HDFS to specify how HDFS blocks should be placed. | |
(Sumadhur Reddy Bolli via szetszwo) | |
HADOOP-8820. Backport HADOOP-8469 and HADOOP-8470: Make NetworkTopology | |
class pluggable and add NetworkTopologyWithNodeGroup, a 4-layer | |
implementation of NetworkTopology. (Junping Du and Jing Zhao via szetszwo) | |
HDFS-3941. Backport HDFS-3498 and HDFS-3601: Support replica removal in | |
BlockPlacementPolicy and make BlockPlacementPolicyDefault extensible for | |
reusing code in subclasses, and add BlockPlacementPolicyWithNodeGroup to | |
support block placement with 4-layer network topology. (Junping Du and | |
Jing Zhao via szetszwo) | |
HADOOP-7868. Hadoop native fails to compile when default linker | |
option is -Wl,--as-needed. (Trevor Robinson via eli) | |
HADOOP-6496. HttpServer sends wrong content-type for CSS files. | |
(Todd Lipcon, Backport by Ivan Mitic via suresh). | |
HADOOP-7827. jsp pages missing DOCTYPE tags. (Dave Vronay, Backport by | |
Ivan Mitic via suresh) | |
HADOOP-8900. BuiltInGzipDecompressor throws IOException - stored gzip size | |
doesn't match decompressed size. (Andy Isaacson via suresh) | |
HADOOP-9036. Fix racy test case TestSinkQueue (Backport HADOOP-7292). | |
(Luke Lu backport by suresh) | |
HADOOP-9095. Backport HADOOP-8372: NetUtils.normalizeHostName() incorrectly | |
handles hostname starting with a numeric character. (Jing Zhao via | |
szetszwo) | |
HADOOP-9099. TestNetUtils fails if "UnknownHost" is resolved as a valid | |
hostname. (Ivan Mitic via szetszwo) | |
HADOOP-9111. Change some JUnit 3 tests to JUnit 4 so that @Ignore tests can | |
be run with ant 1.8.x. (Jing Zhao via szetszwo) | |
HADOOP-9090. Support on-demand publish of metrics. (Mostafa Elhemali via | |
suresh) | |
HADOOP-8617. Backport HADOOP-6148, HADOOP-6166 and HADOOP-7333 for a pure | |
Java CRC32 calculator implementation. (Brandon Li via szetszwo) | |
HADOOP-7096. Allow setting of end-of-record delimiter for TextInputFormat | |
(Ahmed Radwan, backported by suresh) | |
HDFS-3942. Backport HDFS-3495 and HDFS-4234: Update Balancer to support new | |
NetworkTopology with NodeGroup and use generic code for choosing datanode | |
in Balancer. (Junping Du via szetszwo) | |
HDFS-4320. Add a separate configuration for namenode rpc address instead | |
of using fs.default.name. (Mostafa Elhemali via suresh) | |
HDFS-4341. Set default data dir permission in MiniDFSClusterWithNodeGroup. | |
(Ivan Mitic via szetszwo) | |
HDFS-4337. Backport HDFS-4240: For nodegroup-aware block placement, when a | |
node is excluded, the nodes in the same nodegroup should also be excluded. | |
(Meng Gong via szetszwo) | |
HADOOP-9174. TestSecurityUtil fails with Open JDK 7. (Arpit Agarwal via | |
suresh) | |
HADOOP-9175. TestWritableName fails with Open JDK 7. (Arpit Agarwal via | |
suresh) | |
MAPREDUCE-4909. TestKeyValueTextInputFormat fails with Open JDK 7 on | |
Windows. (Arpit Agarwal via suresh) | |
HDFS-4355. TestNameNodeMetrics.testCorruptBlock fails with open JDK7. | |
(Brandon Li via suresh) | |
HDFS-4358. TestCheckpoint failure with JDK7. (Arpit Agarwal via suresh) | |
HADOOP-9179. TestFileSystem fails with open JDK7. (Brandon Li via suresh) | |
MAPREDUCE-4914. TestMiniMRDFSSort fails with openJDK7. (Brandon Li via | |
suresh) | |
MAPREDUCE-4915. TestShuffleExceptionCount fails with open JDK7. | |
(Brandon Li via suresh) | |
HADOOP-9191. TestAccessControlList and TestJobHistoryConfig fail with | |
JDK7. (Arpit Agarwal via suresh) | |
HADOOP-9177. Address issues that reported by static code analysis on | |
winutils. (Ivan Mitic via suresh) | |
MAPREDUCE-4969. TestKeyValueTextInputFormat test fails with Open JDK 7. | |
(Arpit Agarwal via suresh) | |
HADOOP-8973. DiskChecker cannot reliably detect an inaccessible disk on | |
Windows with NTFS ACLs. (Chris Nauroth via suresh) | |
MAPREDUCE-782. Use PureJavaCrc32 in mapreduce spills. (Todd Lipcon, backport by Brandon Li via sseth) | |
HADOOP-9071. Configure ivy log levels for resolve/retrieve. | |
(Giridharan Kesavan via suresh) | |
HDFS-4413. Secondary namenode won't start if HDFS isn't the default | |
file system. (Mostafa Elhemali via suresh) | |
HADOOP-9467. Metrics2 record filter should check name as well as tags. | |
(Ganeshan Iyler via llu) | |
HADOOP-9450. HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH | |
is PREpended instead of APpended. (Chris Nauroth and harsh via harsh) | |
MAPREDUCE-5161. Merge MAPREDUCE-1806 from branch-1 to branch-1-win. | |
CombineFileInputFormat fix for paths not on default FS (Chris Nauroth via | |
bikas) | |
MAPREDUCE-5066. Added a timeout for the job.end.notification.url. | |
(Ivan Mitic via acmurthy) | |
MAPREDUCE-4737. Ensure that mapreduce APIs are semantically consistent | |
with mapred API w.r.t Mapper.cleanup and Reducer.cleanup; in the sense | |
that cleanup is now called even if there is an error. The old mapred API | |
already ensures that Mapper.close and Reducer.close are invoked during | |
error handling. Note that it is an incompatible change, however end-users | |
can override Mapper.run and Reducer.run to get the old (inconsistent) | |
behaviour. (acmurthy) | |
HDFS-4782. Backport edit log corruption toleration. (Chris Nauroth via | |
szetszwo) | |
HADOOP-8745. Incorrect version numbers in hadoop-core POM. | |
(Matthias Friedrich via eli) | |
HADOOP-8981. TestMetricsSystemImpl fails on Windows. (Xuan Gong, backported | |
by Chris Nauroth via suresh) | |
HADOOP-9624. TestFSMainOperationsLocalFileSystem failed when the Hadoop test | |
root path has "X" in its name. (Xi Fang via cnauroth) | |
HDFS-4261. Fix bugs in Balaner causing infinite loop and | |
TestBalancerWithNodeGroup timeing out. (Junping Du via szetszwo) | |
HDFS-3794. WebHDFS Open used with Offset returns the original (and incorrect) | |
Content Length in the HTTP Header. (Tsz Wo (Nicholas), SZE via cnauroth) | |
MAPREDUCE-5405. Job recovery can fail if task log directory symlink from | |
prior run still exists. (cnauroth) | |
HADOOP-9507. LocalFileSystem rename() is broken in some cases when | |
destination exists. (cnauroth) | |
HADOOP-9665. Fixed BlockDecompressorStream#decompress to return -1 rather | |
than throw EOF at end of file. (Zhijie Shen via acmurthy) | |
HADOOP-9768. chown and chgrp reject users and groups with spaces on platforms | |
where spaces are otherwise acceptable. (cnauroth) | |
HADOOP-9801. Configuration#writeXml uses platform defaulting encoding, which | |
may mishandle multi-byte characters. (cnauroth) | |
HDFS-4963. Improve multihoming support in namenode. (Arpit Agarwal via | |
cnauroth) | |
HDFS-5211. Race condition between DistributedFileSystem#close and | |
FileSystem#close can cause return of a closed DistributedFileSystem instance | |
from the FileSystem cache. (cnauroth) | |
MAPREDUCE-5351. Fixed a memory leak in JobTracker due to stable FS objects in | |
FSCache. (Sandy Ryza via acmurthy) | |
MAPREDUCE-5508. JobTracker memory leak caused by unreleased FileSystem | |
objects in JobInProgress#cleanupJob. (Xi Fang via cnauroth) | |
HADOOP-10142. Avoid groups lookup for unprivileged users such as "dr.who" | |
(vinay, backported by Xi Fang via cnauroth) | |
HDFS-5516. WebHDFS does not require user name when anonymous http requests | |
are disallowed. (Miodrag Radulovic via cnauroth) | |
MAPREDUCE-5808. Port output replication factor configurable for terasort to | |
Hadoop 1.x. (Chuan Liu via cnauroth) |