blob: fa4ac16588f4868b45349918f0bbdf2701033ca9 [file] [log] [blame]
Hadoop Change Log
Release 2.6.5 - 2016-10-08
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
HADOOP-12805. Annotate CanUnbuffer with @InterfaceAudience.Public
(Ted Yu via vinayakumarb)
HADOOP-12800. Copy docker directory from 2.8 to 2.7/2.6 repos to enable
pre-commit Jenkins runs (zhz)
HADOOP-12789. log classpath of ApplicationClassLoader at INFO level
(Sangjin Lee via mingma)
HADOOP-13290. Appropriate use of generics in FairCallQueue. (Jonathan Hung
via zhz)
HADOOP-13298. Fix the leftover L&N files in
hadoop-build-tools/src/main/resources/META-INF/. (ozawa)
HADOOP-11301. [optionally] update jmx cache to drop old metrics
(Maysam Yabandeh via stack)
OPTIMIZATIONS
HADOOP-12810. FileSystem#listLocatedStatus causes unnecessary RPC calls
(Ryan Blue via vinayakumarb)
BUG FIXES
HADOOP-12773. HBase classes fail to load with client/job classloader
enabled (sjlee)
HADOOP-12589. Fix intermittent test failure of TestCopyPreserveFlag
(iwasakims)
HADOOP-12559. KMS connection failures should trigger TGT renewal. (Zhe
Zhang via xiaoyuyao)
HADOOP-12682. Fix TestKMS#testKMSRestart* failure. (Wei-Chiu Chuang via
xiaoyuyao)
HADOOP-12958. PhantomReference for filesystem statistics can trigger OOM
(Sangjin Lee via jlowe)
HADOOP-7817. RawLocalFileSystem.append() should give FSDataOutputStream
with accurate .getPos() (kanaka kumar avvaru via vinayakumarb)
HADOOP-13042. Restore lost leveldbjni LICENSE and NOTICE changes. (wang)
HADOOP-13043. Add LICENSE.txt entries for bundled javascript dependencies.
(wang)
HADOOP-12893. Verify LICENSE.txt and NOTICE.txt.
(Xiao Chen and Andrew Wang via aajisaka)
HADOOP-13189. FairCallQueue makes callQueue larger than the configured
capacity. (Vinitha Gankidi via shvachko)
HADOOP-13255. KMSClientProvider should check and renew tgt when doing
delegation token operations. (Xiao Chen via xiaoyuyao)
HADOOP-13350. Additional fix to LICENSE and NOTICE. (Xiao Chen via wang)
HADOOP-13297. Add missing dependency in setting
maven-remote-resource-plugin to fix builds. (Sean Busbey via aajisaka)
HADOOP-13494. ReconfigurableBase can log sensitive information. (Sean
Mackrory via wang)
HADOOP-13579. Fix source-level compatibility after HADOOP-11252.
(Tsuyoshi Ozawa via aajisaka)
HADOOP-13052. ChecksumFileSystem mishandles crc file permissions.
(Daryn Sharp via kihwal)
HADOOP-13192. org.apache.hadoop.util.LineReader cannot handle multibyte
delimiters correctly. (binde via aajisaka)
HADOOP-12348. MetricsSystemImpl creates MetricsSourceAdapter with wrong
time unit parameter. (zxu via rkanter)
HADOOP-12482. Race condition in JMX cache update. (Tony Wu via lei)
HADOOP-11361. Fix a race condition in MetricsSourceAdapter.updateJmxCache.
(Vinayakumar B, Yongjun Zhang, and Brahma Reddy Battula via ozawa)
HADOOP-13434. Add quoting to Shell class. (Owen O'Malley)
Release 2.6.4 - 2016-02-11
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
OPTIMIZATIONS
BUG FIXES
HADOOP-11252. RPC client does not time out by default.
(Wilfred Spiegelenburg and Masatake Iwasaki via aajisaka)
HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile.
(kanaka kumar avvaru via wheat9)
HADOOP-12107. long running apps may have a huge number of StatisticsData
instances under FileSystem (Sangjin Lee via Ming Ma)
HADOOP-12706. TestLocalFsFCStatistics#testStatisticsThreadLocalDataCleanUp
times out occasionally (Sangjin Lee and Colin Patrick McCabe via jlowe)
HADOOP-12736. TestTimedOutTestsListener#testThreadDumpAndDeadlocks
sometimes times out. (Xiao Chen via aajisaka)
Release 2.6.3 - 2015-12-17
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
HADOOP-12413. AccessControlList should avoid calling getGroupNames in
isUserInList with empty groups. (Zhihai Xu via cnauroth)
OPTIMIZATIONS
BUG FIXES
HADOOP-12230. hadoop-project declares duplicate, conflicting curator
dependencies. (Rakesh R via aajisaka)
HADOOP-9242. Duplicate surefire plugin config in hadoop-common.
(Andrey Klochkov via suresh)
HADOOP-10668. TestZKFailoverControllerStress#testExpireBackAndForth
occasionally fails. (Ming Ma via cnauroth)
HADOOP-11267. TestSecurityUtil fails when run with JDK8 because of empty
principal names. (Stephen Chu via wheat9)
HADOOP-10134 [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc
comments. (apurtell via stevel)
HADOOP-12526. there are duplicate dependency definitions in pom's (sjlee)
HADOOP-12577. Bumped up commons-collections version to 3.2.2 to address a
security flaw. (Wei-Chiu Chuang via vinodkv)
Release 2.6.2 - 2015-10-28
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
OPTIMIZATIONS
BUG FIXES
Release 2.6.1 - 2015-09-23
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
HADOOP-7139. Allow appending to existing SequenceFiles
(kanaka kumar avvaru via vinayakumarb)
HADOOP-12280. Skip unit tests based on maven profile rather than
NativeCodeLoader.isNativeCodeLoaded (Masatake Iwasaki via Colin P. McCabe)
OPTIMIZATIONS
HADOOP-11238. Update the NameNode's Group Cache in the background when
possible (Chris Li via Colin P. McCabe)
HADOOP-11506. Configuration variable expansion regex expensive for long
values. (Gera Shegalov via gera)
BUG FIXES
HADOOP-11466: FastByteComparisons: do not use UNSAFE_COMPARER on the SPARC
architecture because it is slower there (Suman Somasundar via Colin P.
McCabe)
HADOOP-10786. Fix UGI#reloginFromKeytab on Java 8. (Stephen Chu via wheat9)
HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification
pipe is full (zhaoyunjiong via cmccabe)
HADOOP-11343. Overflow is not properly handled in caclulating final iv for
AES CTR. (Jerry Chen via wang)
HADOOP-11368. Fix SSLFactory truststore reloader thread leak in
KMSClientProvider. (Arun Suresh via wang)
HADOOP-11350. The size of header buffer of HttpServer is too small when
HTTPS is enabled. (Benoy Antony via wheat9)
HADOOP-11482. Use correct UGI when KMSClientProvider is called by a proxy
user. Contributed by Arun Suresh.
HADOOP-11295. RPC Server Reader thread can't shutdown if RPCCallQueue is
full. (Ming Ma via kihwal)
HADOOP-11604. Prevent ConcurrentModificationException while closing domain
sockets during shutdown of DomainSocketWatcher thread. (cnauroth)
HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream
should be non static. (Sean Busbey via yliu)
HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt
synchronization. (Sean Busbey via yliu)
HADOOP-11812. Implement listLocatedStatus for ViewFileSystem to speed up
split calculation (gera)
HADOOP-11730. Regression: s3n read failure recovery broken.
(Takenori Sato via stevel)
HADOOP-11802: DomainSocketWatcher thread terminates sometimes after there
is an I/O error during requestShortCircuitShm (cmccabe)
HADOOP-11491. HarFs incorrectly declared as requiring an authority.
(Brahma Reddy Battula via gera)
HADOOP-8151. Error handling in snappy decompressor throws invalid
exceptions. (Matt Foley via harsh)
HADOOP-11932. MetricsSinkAdapter may hang when being stopped.
(Brahma Reddy Battula via jianhe)
HADOOP-11934. Use of JavaKeyStoreProvider in LdapGroupsMapping causes
infinite loop. (Larry McCay via cnauroth)
Release 2.6.0 - 2014-11-18
INCOMPATIBLE CHANGES
NEW FEATURES
HADOOP-10433. Key Management Server based on KeyProvider API. (tucu)
HADOOP-10893. isolated classloader on the client side (Sangjin Lee via
jlowe)
HADOOP-10400. Incorporate new S3A FileSystem implementation. (Jordan
Mendelson and Dave Wang via atm)
IMPROVEMENTS
HADOOP-10808. Remove unused native code for munlock. (cnauroth)
HADOOP-10815. Implement Windows equivalent of mlock. (cnauroth)
HADOOP-7664. Remove warmings when overriding final parameter configuration
if the override value is same as the final parameter value.
(Ravi Prakash via suresh)
HADOOP-10673. Update rpc metrics when the call throws an exception. (Ming Ma
via jing9)
HADOOP-10845. Add common tests for ACLs in combination with viewfs.
(Stephen Chu via cnauroth)
HADOOP-10839. Add unregisterSource() to MetricsSystem API.
(Shanyu Zhao via cnauroth)
HADOOP-10607. Create an API to separate credentials/password storage
from applications (Larry McCay via omalley)
HADOOP-10732. Fix locking in credential update. (Ted Yu via omalley)
HADOOP-10733. Fix potential null dereference in CredShell. (Ted Yu via
omalley)
HADOOP-10610. Upgrade S3n s3.fs.buffer.dir to support multi directories.
(Ted Malaska via atm)
HADOOP-10817. ProxyUsers configuration should support configurable
prefixes. (tucu)
HADOOP-10755. Support negative caching of user-group mapping.
(Lei Xu via wang)
HADOOP-10855. Allow Text to be read with a known Length. (todd)
HADOOP-10887. Add XAttrs to ViewFs and make XAttrs + ViewFileSystem
internal dir behavior consistent. (Stephen Chu via wang)
HADOOP-10882. Move DirectBufferPool into common util. (todd)
HADOOP-8069. Enable TCP_NODELAY by default for IPC. (Todd Lipcon via
Arpit Agarwal)
HADOOP-10902. Deletion of directories with snapshots will not output
reason for trash move failure. (Stephen Chu via wang)
HADOOP-10900. CredentialShell args should use single-dash style. (wang)
HADOOP-10903. Enhance hadoop classpath command to expand wildcards or write
classpath into jar manifest. (cnauroth)
HADOOP-10791. AuthenticationFilter should support externalizing the
secret for signing and provide rotation support. (rkanter via tucu)
HADOOP-10771. Refactor HTTP delegation support out of httpfs to common.
(tucu)
HADOOP-10835. Implement HTTP proxyuser support in HTTP authentication
client/server libraries. (tucu)
HADOOP-10820. Throw an exception in GenericOptionsParser when passed
an empty Path. (Alex Holmes and Zhihai Xu via wang)
HADOOP-10281. Create a scheduler, which assigns schedulables a priority
level. (Chris Li via Arpit Agarwal)
HADOOP-8944. Shell command fs -count should include human readable option
(Jonathan Allen via aw)
HADOOP-10231. Add some components in Native Libraries document (Akira
AJISAKA via aw)
HADOOP-10650. Add ability to specify a reverse ACL (black list) of users
and groups. (Benoy Antony via Arpit Agarwal)
HADOOP-10335. An ip whilelist based implementation to resolve Sasl
properties per connection. (Benoy Antony via Arpit Agarwal)
HADOOP-10975. org.apache.hadoop.util.DataChecksum should support calculating
checksums in native code (James Thomas via Colin Patrick McCabe)
HADOOP-10201. Add listing to KeyProvider API. (Larry McCay via omalley)
HADOOP-10177. Create CLI tools for managing keys. (Larry McCay via omalley)
HADOOP-10432. Refactor SSLFactory to expose static method to determine
HostnameVerifier. (tucu)
HADOOP-10429. KeyStores should have methods to generate the materials
themselves, KeyShell should use them. (tucu)
HADOOP-10427. KeyProvider implementations should be thread safe. (tucu)
HADOOP-10428. JavaKeyStoreProvider should accept keystore password via
configuration falling back to ENV VAR. (tucu)
HADOOP-10430. KeyProvider Metadata should have an optional description,
there should be a method to retrieve the metadata from all keys. (tucu)
HADOOP-10431. Change visibility of KeyStore.Options getter methods to
public. (tucu)
HADOOP-10534. KeyProvider getKeysMetadata should take a list of names
rather than returning all keys. (omalley)
HADOOP-10719. Add generateEncryptedKey and decryptEncryptedKey
methods to KeyProvider. (asuresh via tucu)
HADOOP-10817. ProxyUsers configuration should support configurable
prefixes. (tucu)
HADOOP-10881. Clarify usage of encryption and encrypted encryption
key in KeyProviderCryptoExtension. (wang)
HADOOP-10770. KMS add delegation token support. (tucu)
HADOOP-10698. KMS, add proxyuser support. (tucu)
HADOOP-8896. Javadoc points to Wrong Reader and Writer classes
in SequenceFile (Ray Chiang via aw)
HADOOP-10998. Fix bash tab completion code to work (Jim Hester via aw)
HADOOP-10880. Move HTTP delegation tokens out of URL querystring to
a header. (tucu)
HADOOP-11005. Fix HTTP content type for ReconfigurationServlet.
(Lei Xu via wang)
HADOOP-10814. Update Tomcat version used by HttpFS and KMS to latest
6.x version. (rkanter via tucu)
HADOOP-10994. KeyProviderCryptoExtension should use CryptoCodec for
generation/decryption of keys. (tucu)
HADOOP-11021. Configurable replication factor in the hadoop archive
command. (Zhe Zhang via wang)
HADOOP-11030. Define a variable jackson.version instead of using constant
at multiple places. (Juan Yu via kasha)
HADOOP-10990. Add missed NFSv3 request and response classes (brandonli)
HADOOP-10863. KMS should have a blacklist for decrypting EEKs.
(asuresh via tucu)
HADOOP-11054. Add a KeyProvider instantiation based on a URI. (tucu)
HADOOP-11015. Http server/client utils to propagate and recreate
Exceptions from server to client. (tucu)
HADOOP-11060. Create a CryptoCodec test that verifies interoperability
between the JCE and OpenSSL implementations. (hitliuyi via tucu)
HADOOP-11070. Create MiniKMS for testing. (tucu)
HADOOP-11057. checknative command to probe for winutils.exe on windows.
(Xiaoyu Yao via cnauroth)
HADOOP-10758. KMS: add ACLs on per key basis. (tucu)
HADOOP-9540. Expose the InMemoryS3 and S3N FilesystemStores implementations
for Unit testing. (Steve Lougran)
HADOOP-10373 create tools/hadoop-amazon for aws/EMR support (stevel)
HADOOP-11074. Move s3-related FS connector code to hadoop-aws (David S.
Wang via Colin Patrick McCabe)
HADOOP-11091. Eliminate old configuration parameter names from s3a (David
S. Wang via Colin Patrick McCabe)
HADOOP-10868. AuthenticationFilter should support externalizing the
secret for signing and provide rotation support. (rkanter via tucu)
HADOOP-10922. User documentation for CredentialShell. (Larry McCay via wang)
HADOOP-11016. KMS should support signing cookies with zookeeper secret
manager. (tucu)
HADOOP-11106. Document considerations of HAR and Encryption. (clamb via wang)
HADOOP-10970. Cleanup KMS configuration keys. (wang)
HADOOP-11017. KMS delegation token secret manager should be able to use
zookeeper as store. (asuresh via tucu)
HADOOP-11009. Add Timestamp Preservation to DistCp (Gary Steelman via aw)
HADOOP-11101. How about inputstream close statement from catch block to
finally block in FileContext#copy() ( skrho via vinayakumarb )
HADOOP-8808. Update FsShell documentation to mention deprecation of some
of the commands, and mention alternatives (Akira AJISAKA via aw)
HADOOP-10954. Adding site documents of hadoop-tools (Masatake Iwasaki
via aw)
HADOOP-10731. Remove @date JavaDoc comment in ProgramDriver class (Henry
Saputra via aw)
HADOOP-11153. Make number of KMS threads configurable. (wang)
HADOOP-11007. Reinstate building of ant tasks support. (jlowe via kihwal)
HADOOP-11178. Fix findbugs exclude file. (Arun Suresh via wang)
HADOOP-11174. Delegation token for KMS should only be got once if it
already exists. (Yi Liu via wang)
HADOOP-11184. Update Hadoop's lz4 to version r123. (cmccabe)
HADOOP-11181. Generalized o.a.h.s.t.d.DelegationTokenManager to handle all
sub-classes of AbstractDelegationTokenIdentifier. (zjshen)
HADOOP-11207. Enhanced common DelegationTokenAuthenticationHandler to support
proxy-users on Delegation-token management operations. (Zhijie Shen via
vinodkv)
HADOOP-11216. Improve Openssl library finding. (cmccabe via yliu)
HADOOP-11254. Changed visibility of AccessControlList to be public for
consumption by ecosystem. (Zhijie Shen via vinodkv)
HADOOP-11286. Copied LimitInputStream from guava-0.14 to hadoop to avoid
issues with newer versions of guava in applications. (Christopher Tubbs
via acmurthy)
OPTIMIZATIONS
HADOOP-10838. Byte array native checksumming. (James Thomas via todd)
HADOOP-10696. Add optional attributes to KeyProvider Options and Metadata.
(tucu)
HADOOP-10695. KMSClientProvider should respect a configurable timeout.
(yoderme via tucu)
HADOOP-10757. KeyProvider KeyVersion should provide the key name.
(asuresh via tucu)
HADOOP-10769. Create KeyProvider extension to handle delegation tokens.
(Arun Suresh via atm)
HADOOP-10812. Delegate KeyProviderExtension#toString to underlying
KeyProvider. (wang)
HADOOP-10736. Add key attributes to the key shell. (Mike Yoder via wang)
HADOOP-10824. Refactor KMSACLs to avoid locking. (Benoy Antony via umamahesh)
HADOOP-10841. EncryptedKeyVersion should have a key name property.
(asuresh via tucu)
HADOOP-10842. CryptoExtension generateEncryptedKey method should
receive the key name. (asuresh via tucu)
HADOOP-10750. KMSKeyProviderCache should be in hadoop-common.
(asuresh via tucu)
HADOOP-10720. KMS: Implement generateEncryptedKey and decryptEncryptedKey
in the REST API. (asuresh via tucu)
HADOOP-10891. Add EncryptedKeyVersion factory method to
KeyProviderCryptoExtension. (wang)
HADOOP-10756. KMS audit log should consolidate successful similar requests.
(asuresh via tucu)
HADOOP-10793. KeyShell args should use single-dash style. (wang)
HADOOP-10936. Change default KeyProvider bitlength to 128. (wang)
HADOOP-10224. JavaKeyStoreProvider has to protect against corrupting
underlying store. (asuresh via tucu)
HADOOP-10282. Create a FairCallQueue: a multi-level call queue which
schedules incoming calls and multiplexes outgoing calls. (Chris Li via
Arpit Agarwal)
HADOOP-10833. Remove unused cache in UserProvider. (Benoy Antony)
HADOOP-11112. TestKMSWithZK does not use KEY_PROVIDER_URI. (tucu via wang)
HADOOP-11111 MiniKDC to use locale EN_US for case conversions. (stevel)
HADOOP-10681. Remove unnecessary synchronization from Snappy & Zlib
codecs. (Gopal Vijayaraghavan via acmurthy)
HADOOP-11194. Ignore .keep files. (kasha)
HADOOP-11195. Move Id-Name mapping in NFS to the hadoop-common area for
better maintenance (Yongjun Zhang via brandonli)
HADOOP-11247. Fix a couple javac warnings in NFS. (Brandon Li via wheat9)
BUG FIXES
HADOOP-11182. GraphiteSink emits wrong timestamps (Sascha Coenen via raviprak)
HADOOP-10781. Unportable getgrouplist() usage breaks FreeBSD (Dmitry
Sivachenko via Colin Patrick McCabe)
HADOOP-10507. FsShell setfacl can throw ArrayIndexOutOfBoundsException when
no perm is specified. (Stephen Chu and Sathish Gurram via cnauroth)
HADOOP-10780. hadoop_user_info_alloc fails on FreeBSD due to incorrect
sysconf use (Dmitry Sivachenko via Colin Patrick McCabe)
HADOOP-10810. Clean up native code compilation warnings. (cnauroth)
HADOOP-9921. daemon scripts should remove pid file on stop call after stop
or process is found not running ( vinayakumarb )
HADOOP-10591. Compression codecs must used pooled direct buffers or
deallocate direct buffers when stream is closed (cmccabe)
HADOOP-10857. Native Libraries Guide doen't mention a dependency on
openssl-development package (ozawa via cmccabe)
HADOOP-10866. RawLocalFileSystem fails to read symlink targets via the stat
command when the format of the stat command uses non-curly quotes (yzhang
via cmccabe)
HADOOP-10830. Missing lock in JavaKeyStoreProvider.createCredentialEntry.
(Benoy Antony via umamahesh)
HADOOP-10928. Incorrect usage on `hadoop credential list`.
(Josh Elser via wang)
HADOOP-10927. Fix CredentialShell help behavior and error codes.
(Josh Elser via wang)
HADOOP-10933. FileBasedKeyStoresFactory Should use Configuration.getPassword
for SSL Passwords. (lmccay via tucu)
HADOOP-10905. LdapGroupsMapping Should use configuration.getPassword for SSL
and LDAP Passwords. (lmccay via brandonli)
HADOOP-10931 compile error on tools/hadoop-openstack (xukun via stevel)
HADOOP-10929. Typo in Configuration.getPasswordFromCredentialProviders
(lmccay via brandonli)
HADOOP-10402. Configuration.getValByRegex does not substitute for
variables. (Robert Kanter via kasha)
HADOOP-10851. NetgroupCache does not remove group memberships. (Benoy
Antony via Arpit Agarwal)
HADOOP-10962. Flags for posix_fadvise are not valid in some architectures
(David Villegas via Colin Patrick McCabe)
HADOOP-10966. Hadoop Common native compilation broken in windows.
(David Villegas via Arpit Agarwal)
HADOOP-10843. TestGridmixRecord unit tests failure on PowerPC (Jinghui Wang
via Colin Patrick McCabe)
HADOOP-10121. Fix javadoc spelling for HadoopArchives#writeTopLevelDirs
(Akira AJISAKA via aw)
HADOOP-10964. Small fix for NetworkTopologyWithNodeGroup#sortByDistance.
(Yi Liu via wang)
HADOOP-10059. RPC authentication and authorization metrics overflow to
negative values on busy clusters (Tsuyoshi OZAWA and Akira AJISAKA
via jlowe)
HADOOP-10973. Native Libraries Guide contains format error. (Peter Klavins
via Arpit Agarwal)
HADOOP-10972. Native Libraries Guide contains mis-spelt build line (Peter
Klavins via aw)
HADOOP-10873. Fix dead link in Configuration javadoc (Akira AJISAKA
via aw)
HADOOP-10968. hadoop native build fails to detect java_libarch on
ppc64le (Dinar Valeev via Colin Patrick McCabe)
HADOOP-10141. Create KeyProvider API to separate encryption key storage
from the applications. (omalley)
HADOOP-10237. JavaKeyStoreProvider needs to set keystore permissions
correctly. (Larry McCay via omalley)
HADOOP-10244. TestKeyShell improperly tests the results of delete (Larry
McCay via omalley)
HADOOP-10583. bin/hadoop key throws NPE with no args and assorted other fixups. (clamb via tucu)
HADOOP-10586. KeyShell doesn't allow setting Options via CLI. (clamb via tucu)
HADOOP-10645. TestKMS fails because race condition writing acl files. (tucu)
HADOOP-10611. KMS, keyVersion name should not be assumed to be
keyName@versionNumber. (tucu)
HADOOP-10816. KeyShell returns -1 on error to the shell, should be 1.
(Mike Yoder via wang)
HADOOP-10826. Iteration on KeyProviderFactory.serviceLoader is
thread-unsafe. (benoyantony viat tucu)
HADOOP-10920. site plugin couldn't parse hadoop-kms index.apt.vm.
(Akira Ajisaka via wang)
HADOOP-10937. Need to set version name correctly before decrypting EEK.
(Arun Suresh via wang)
HADOOP-10925. Compilation fails in native link0 function on Windows.
(cnauroth)
HADOOP-10918. JMXJsonServlet fails when used within Tomcat. (tucu)
HADOOP-10939. Fix TestKeyProviderFactory testcases to use default 128 bit
length keys. (Arun Suresh via wang)
HADOOP-10862. Miscellaneous trivial corrections to KMS classes.
(asuresh via tucu)
HADOOP-10967. Improve DefaultCryptoExtension#generateEncryptedKey
performance. (hitliuyi via tucu)
HADOOP-10488. TestKeyProviderFactory fails randomly. (tucu)
HADOOP-10989. Work around buggy getgrouplist() implementations on Linux that
return 0 on failure. (cnauroth)
HADOOP-8815. RandomDatum needs to override hashCode().
(Brandon Li via suresh)
HADOOP-11056. OsSecureRandom.setConf() might leak file descriptors. (yzhang
via cmccabe)
HADOOP-11040. Return value of read(ByteBuffer buf) in CryptoInputStream is
incorrect in some cases. (Yi Liu via wang)
HADOOP-11140. hadoop-aws only need test-scoped dependency on
hadoop-common's tests jar. (Juan Yu via wang)
HADOOP-1110. JavaKeystoreProvider should not report a key as created if it
was not flushed to the backing file.
HADOOP-11130. NFS updateMaps OS check is reversed (brandonli)
HADOOP-11113. Namenode not able to reconnect to KMS after KMS restart.
(Arun Suresh via wang)
HADOOP-11160. Fix typo in nfs3 server duplicate entry reporting.
(Charles Lamb via wheat9)
HADOOP-11151. Automatically refresh auth token and retry on auth failure.
(Arun Suresh via wang)
HADOOP-11168. Remove duplicated entry "dfs.webhdfs.enabled" in the useri
doc. (Yi Liu via wheat9)
HADOOP-11169. Fix DelegationTokenAuthenticatedURL to pass the connection
Configurator to the authenticator. (Arun Suresh via wang)
HADOOP-10404. Some accesses to DomainSocketWatcher#closed are not protected
by lock (cmccabe)
HADOOP-11161. Expose close method in KeyProvider to give clients of
Provider implementations a hook to release resources. (Arun Suresh via atm)
HADOOP-11133. Should trim the content of keystore password file for JavaKeyStoreProvider
(Yi Liu via umamahesh)
HADOOP-11193. Fix uninitialized variables in NativeIO.c
(Xiaoyu Yao via wheat9)
HADOOP-11176. KMSClientProvider authentication fails when both currentUgi
and loginUgi are a proxied user. (Arun Suresh via atm)
HADOOP-11198. Fix typo in javadoc for FileSystem#listStatus().
(Li Lu via wheat9)
HADOOP-11253. Hadoop streaming test TestStreamXmlMultipleRecords fails on
Windows. (Varun Vasudev via wheat9)
BREAKDOWN OF HDFS-6134 AND HADOOP-10150 SUBTASKS AND RELATED JIRAS
HADOOP-10734. Implement high-performance secure random number sources.
(Yi Liu via Colin Patrick McCabe)
HADOOP-10603. Crypto input and output streams implementing Hadoop stream
interfaces. (Yi Liu and Charles Lamb)
HADOOP-10628. Javadoc and few code style improvement for Crypto
input and output streams. (Yi Liu via clamb)
HADOOP-10632. Minor improvements to Crypto input and output streams.
(Yi Liu)
HADOOP-10635. Add a method to CryptoCodec to generate SRNs for IV. (Yi Liu)
HADOOP-10653. Add a new constructor for CryptoInputStream that
receives current position of wrapped stream. (Yi Liu)
HADOOP-10662. NullPointerException in CryptoInputStream while wrapped
stream is not ByteBufferReadable. Add tests using normal stream. (Yi Liu)
HADOOP-10713. Refactor CryptoCodec#generateSecureRandom to take a byte[].
(wang via yliu)
HADOOP-10693. Implementation of AES-CTR CryptoCodec using JNI to OpenSSL.
(Yi Liu via cmccabe)
HADOOP-10803. Update OpensslCipher#getInstance to accept CipherSuite#name
format. (Yi Liu)
HADOOP-10735. Fall back AesCtrCryptoCodec implementation from OpenSSL to
JCE if non native support. (Yi Liu)
HADOOP-10870. Failed to load OpenSSL cipher error logs on systems with old
openssl versions (cmccabe)
HADOOP-10853. Refactor get instance of CryptoCodec and support create via
algorithm/mode/padding. (Yi Liu)
HADOOP-10919. Copy command should preserve raw.* namespace
extended attributes. (clamb)
HDFS-6873. Constants in CommandWithDestination should be static. (clamb)
HADOOP-10871. incorrect prototype in OpensslSecureRandom.c (cmccabe)
HADOOP-10886. CryptoCodec#getCodecclasses throws NPE when configurations not
loaded. (umamahesh)
--
HADOOP-10911. hadoop.auth cookie after HADOOP-10710 still not proper
according to RFC2109. (gchanan via tucu)
HADOOP-11036. Add build directory to .gitignore (Tsuyoshi OZAWA via aw)
HADOOP-11012. hadoop fs -text of zero-length file causes EOFException
(Eric Payne via jlowe)
HADOOP-11063. KMS cannot deploy on Windows, because class names are too long.
(cnauroth)
HADOOP-11067. warning message 'ssl.client.truststore.location has not
been set' gets printed for hftp command. (Xiaoyu Yao via Arpit Agarwal)
HADOOP-11069. KMSClientProvider should use getAuthenticationMethod() to
determine if in proxyuser mode or not. (tucu)
HADOOP-11073. Credential Provider related Unit Tests Failure on Windows.
(Xiaoyu Yao via cnauroth)
HADOOP-11071. KMSClientProvider should drain the local generated EEK cache
on key rollover. (tucu)
HADOOP-10925. Compilation fails in native link0 function on Windows.
(cnauroth)
HADOOP-11077. NPE if hosts not specified in ProxyUsers. (gchanan via tucu)
HADOOP-9989. Bug introduced in HADOOP-9374, which parses the -tokenCacheFile
as binary file but set it to the configuration as JSON file. (zxu via tucu)
HADOOP-11085. Excessive logging by org.apache.hadoop.util.Progress when
value is NaN (Mit Desai via jlowe)
HADOOP-11083. After refactoring of HTTP proxyuser to common, doAs param is
case sensitive. (tucu)
HADOOP-11088. Unittest TestKeyShell, TestCredShell and TestKMS assume UNIX
path separator for JECKS key store path. (Xiaoyu Yao via cnauroth)
HADOOP-11096. KMS: KeyAuthorizationKeyProvider should verify the keyversion
belongs to the keyname on decrypt. (tucu)
HADOOP-11097. kms docs say proxyusers, not proxyuser for config params.
(clamb via tucu)
HADOOP-11062. CryptoCodec testcases requiring OpenSSL should be run
only if -Pnative is used. (asuresh via tucu)
HADOOP-11099. KMS return HTTP UNAUTHORIZED 401 on ACL failure. (tucu)
HADOOP-11105. MetricsSystemImpl could leak memory in registered callbacks.
(Chuan Liu via cnauroth)
HADOOP-10982. KMS: Support for multiple Kerberos principals. (tucu)
HADOOP-11109. Site build is broken. (Jian He via atm)
HADOOP-10946. Fix a bunch of typos in log messages (Ray Chiang via aw)
HADOOP-10131. NetWorkTopology#countNumOfAvailableNodes() is returning
wrong value if excluded nodes passed are not part of the cluster tree
(vinayakumarb)
HADOOP-11064. UnsatisifedLinkError with hadoop 2.4 JARs on hadoop-2.6 due to
NativeCRC32 method changes. (cnauroth)
HADOOP-11048. user/custom LogManager fails to load if the client
classloader is enabled (Sangjin Lee via jlowe)
HADOOP-10552. Fix usage and example at FileSystemShell.apt.vm
(Kenji Kikushima via aw)
HADOOP-11143 NetUtils.wrapException loses inner stack trace on BindException
(stevel)
HADOOP-11049. javax package system class default is too broad (Sangjin Lee
via jlowe)
HADOOP-11154. Update BUILDING.txt to state that CMake 3.0 or newer is
required on Mac. (cnauroth)
HADOOP-11145. TestFairCallQueue fails. (Akira AJISAKA via cnauroth)
HADOOP-11117 UGI HadoopLoginModule doesn't catch & wrap all
kerberos-related exceptions (stevel)
HADOOP-11163. MetricsSystemImpl may miss a registered source.
(Chuan Liu via cnauroth)
HADOOP-11179. Java untar should handle the case that the file entry comes
without its parent directory entry. (Craig Welch via zjshen)
HADOOP-11175. Fix several issues of hadoop security configuration in user
doc. (Yi Liu via cnauroth)
HADOOP-11122. Fix findbugs in ZK DelegationTokenSecretManagers.
(Arun Suresh via kasha)
HADOOP-11228. Winutils task: unsecure path should not call
AddNodeManagerAndUserACEsToObject. (Remus Rusanu via jianhe)
HADOOP-11170. ZKDelegationTokenSecretManager fails to renewToken created by
a peer. (Arun Suresh and Gregory Chanan via kasha)
HADOOP-11217. Disable SSLv3 in KMS. (Robert Kanter via kasha)
HADOOP-11068. Match hadoop.auth cookie format to jetty output.
(Gregory Chanan via cnauroth)
HADOOP-9740. Fix FsShell '-text' command to be able to read Avro
files stored in HDFS and other filesystems. (Allan Yan via jlowe)
HADOOP-11250. fix endmacro of set_find_shared_library_without_version in
CMakeLists (Yi Liu via Colin P. McCabe)
HADOOP-11221. IdentityHashStore assumes System.identityHashCode() is
non-negative. (Jinghui Wang via szetszwo)
HADOOP-11241. Fixed intermittent TestNMSimulator failure due to timing issue.
(Varun Vasudev via zjshen)
HADOOP-11265. Credential and Key Shell Commands not available on Windows.
(Larry McCay via cnauroth)
HADOOP-11280. TestWinUtils#testChmod fails after removal of
NO_PROPAGATE_INHERIT_ACE. (cnauroth)
HADOOP-11282. Skip NFS TestShellBasedIdMapping tests that are irrelevant on
Windows. (cnauroth)
HADOOP-9576. Changed NetUtils#wrapException to throw EOFException instead
of wrapping it as IOException. (Steve Loughran via jianhe)
Release 2.5.2 - 2014-11-19
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
OPTIMIZATIONS
BUG FIXES
HADOOP-11243. SSLFactory shouldn't allow SSLv3. (Wei Yan via kasha)
HADOOP-11260. Patch up Jetty to disable SSLv3. (Mike Yoder via kasha)
HADOOP-11307. create-release script should run git clean first (kasha)
Release 2.5.1 - 2014-09-05
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
HADOOP-10956. Fix create-release script to include docs and necessary txt
files. (kasha)
OPTIMIZATIONS
BUG FIXES
HADOOP-11001. Fix test-patch to work with the git repo. (kasha)
HADOOP-11065. Rat check should exclude "**/build/**". (kasha)
Release 2.5.0 - 2014-08-11
INCOMPATIBLE CHANGES
NEW FEATURES
HADOOP-10498. Add support for proxy server. (daryn)
HADOOP-9704. Write metrics sink plugin for Hadoop/Graphite (Chu Tong, Alex Newman and Babak Behzad via raviprak)
HADOOP-8943. Support multiple group mapping providers. (Kai Zheng via brandonli)
HADOOP-9361 Strictly define the expected behavior of filesystem APIs and
write tests to verify compliance (stevel)
IMPROVEMENTS
HADOOP-10451. Remove unused field and imports from SaslRpcServer.
(Benoy Antony via jing9)
HADOOP-10345. Sanitize the the inputs (groups and hosts) for the proxyuser
configuration. (Benoy Antony via jing9)
HADOOP-10454. Provide FileContext version of har file system. (Kihwal Lee
via jeagles)
HADOOP-10104. Update jackson to 1.9.13 (Akira Ajisaka via stevel)
HADOOP-10503. Move junit up to v 4.11. (cnauroth)
HADOOP-10535. Make the retry numbers in ActiveStandbyElector configurable.
(jing9)
HADOOP-10322. Add ability to read principal names from a keytab.
(Benoy Antony and Daryn Sharp via kihwal)
HADOOP-10549. MAX_SUBST and varPat should be final in Configuration.java.
(Gera Shegalov via cnauroth)
HADOOP-10471. Reduce the visibility of constants in ProxyUsers.
(Benoy Antony via wheat9)
HADOOP-10556. Add toLowerCase support to auth_to_local rules
for service name. (tucu)
HADOOP-10467. Enable proxyuser specification to support list of users in
addition to list of groups (Benoy Antony via Arpit Agarwal)
HADOOP-10158. SPNEGO should work with multiple interfaces/SPNs.
(daryn via kihwal)
HADOOP-10572. Example NFS mount command must pass noacl as it isn't
supported by the server yet. (Harsh J via brandonli)
HADOOP-10588. Workaround for jetty6 acceptor startup issue. (kihwal)
HADOOP-10609. .gitignore should ignore .orig and .rej files. (kasha)
HADOOP-10614. CBZip2InputStream is not threadsafe (Xiangrui Meng via
Sandy Ryza)
HADOOP-10618. Remove SingleNodeSetup.apt.vm. (Akira Ajisaka via
Arpit Agarwal)
HADOOP-10566. Refactor proxyservers out of ProxyUsers. (Benoy Antony via
Arpit Agarwal)
HDFS-9968. ProxyUsers does not work with NetGroups. (Benoy Antony via
Devaraj Das)
HADOOP-10448. Support pluggable mechanism to specify proxy user settings.
(Benoy Antony via Arpit Agarwal)
HADOOP-9555. HA functionality that uses ZooKeeper may experience inadvertent
TCP RST and miss session expiration event due to bug in client connection
management. (cnauroth)
HADOOP-10376. Refactor refresh*Protocols into a single generic
refreshConfigProtocol. (Chris Li via Arpit Agarwal)
HADOOP-6350. Documenting Hadoop metrics. (Akira Ajisaka via Arpit Agarwal)
HADOOP-10691. Improve the readability of 'hadoop fs -help'.
(Lei Xu via wang)
HADOOP-10688. Expose thread-level FileSystem StatisticsData (Sandy Ryza)
HADOOP-10657. Have RetryInvocationHandler log failover attempt at INFO
level. (Ming Ma via jing9)
HADOOP-10666. Remove Copyright /d/d/d/d Apache Software Foundation from
the source files license header. (Henry Saputra via wang)
HADOOP-10557. FsShell -cp -pa option for preserving extended ACLs.
(Akira Ajisaka via cnauroth)
HADOOP-10279. Create multiplexer, a requirement for the fair queue.
(Chris Li via Arpit Agarwal)
HADOOP-10659. Refactor AccessControlList to reuse utility functions
and to improve performance. (Benoy Antony via Arpit Agarwal)
HADOOP-10665. Make Hadoop Authentication Handler loads case in-sensitive
(Benoy Antony via vinayakumarb)
HADOOP-10652. Refactor Proxyusers to use AccessControlList. (Benoy
Antony via Arpit Agarwal)
HADOOP-10747. Support configurable retries on SASL connection failures in
RPC client. (cnauroth)
HADOOP-10754. Reenable several HA ZooKeeper-related tests on Windows.
(cnauroth)
HADOOP-10565. Support IP ranges (CIDR) in proxyuser.hosts. (Benoy Antony
via Arpit Agarwal)
HADOOP-10649. Allow overriding the default ACL for service authorization
(Benoy Antony via Arpit Agarwal)
HADOOP-10767. Clean up unused code in Ls shell command. (cnauroth)
HADOOP-9651 Filesystems to throw FileAlreadyExistsException in
createFile(path, overwrite=false) when the file exists (stevel)
HADOOP-9495 Define behaviour of Seekable.seek(), write tests,
fix all hadoop implementations for compliance
HADOOP-10312 Shell.ExitCodeException to have more useful toString (stevel)
HADOOP-10782. Fix typo in DataChecksum class. (Jingguo Yao via suresh)
HADOOP-10896. Update compatibility doc to capture visibility of
un-annotated classes/ methods. (kasha)
OPTIMIZATIONS
HADOOP-10674. Improve PureJavaCrc32 performance and use java.util.zip.CRC32
for Java 7 and above. (szetszwo)
BUG FIXES
HADOOP-10378. Typo in help printed by hdfs dfs -help.
(Mit Desai via suresh)
HADOOP-10418. SaslRpcClient should not assume that remote principals are in
the default_realm. (atm)
HADOOP-10426. Declare CreateOpts.getOpt(..) with generic type argument,
removes unused FileContext.getFileStatus(..) and fixes various javac
warnings. (szetszwo)
HADOOP-10439. Fix compilation error in branch-2 after HADOOP-10426. (wheat9)
HADOOP-10414. Incorrect property name for RefreshUserMappingProtocol in
hadoop-policy.xml. (Joey Echeverria via atm)
HADOOP-10459. distcp V2 doesn't preserve root dir's attributes when -p is
specified. (Yongjun Zhang via atm)
HADOOP-10462. DF#getFilesystem is not parsing the command output.
(Akira AJISAKA via umamahesh)
HADOOP-10468. TestMetricsSystemImpl.testMultiThreadedPublish fails
intermediately. (wheat9)
HADOOP-10475. ConcurrentModificationException in
AbstractDelegationTokenSelector.selectToken(). (jing9)
HADOOP-10350. BUILDING.txt should mention openssl dependency required
for hadoop-pipes (Vinayakumar B)
HADOOP-10495. TestFileUtil fails on Windows due to bad permission
assertions. (cnauroth)
HADOOP-10496. Metrics system FileSink can leak file descriptor. (cnauroth)
HADOOP-10500. TestDoAsEffectiveUser fails on JDK7 due to failure to reset
proxy user configuration. (cnauroth)
HADOOP-10499. Remove unused parameter from ProxyUsers.authorize().
(Benoy Antony via cnauroth)
HADOOP-9919. Update hadoop-metrics2.properties examples to Yarn.
(Akira AJISAKA via suresh)
HADOOP-10526. Chance for Stream leakage in CompressorStream. (Rushabh
Shah via kihwal)
HADOOP-10251. Both NameNodes could be in STANDBY State if SNN network is unstable
(Vinayakumar B via umamahesh)
HADOOP-10531. hadoop-config.sh - bug in --hosts argument.
(Sebastien Barrier via wang)
HADOOP-10539. Provide backward compatibility for ProxyUsers.authorize()
call. (Benoy Antony via cnauroth)
HADOOP-10540. Datanode upgrade in Windows fails with hardlink error.
(Chris Nauroth and Arpit Agarwal)
HADOOP-10508. RefreshCallQueue fails when authorization is enabled.
(Chris Li via wheat9)
HADOOP-10547. Give SaslPropertiesResolver.getDefaultProperties() public
scope. (Benoy Antony via Arpit Agarwal)
HADOOP-10543. RemoteException's unwrapRemoteException method failed for
PathIOException. (Yongjun Zhang via atm)
HADOOP-10568. Add s3 server-side encryption. (David S. Wang via atm)
HADOOP-10541. InputStream in MiniKdc#initKDCServer for minikdc.ldiff is not
closed. (Swarnim Kulkarni via cnauroth)
HADOOP-10517. InputStream is not closed in two methods of JarFinder.
(Ted Yu via cnauroth)
HADOOP-10581. TestUserGroupInformation#testGetServerSideGroups fails
because groups stored in Set and ArrayList are compared.
(Mit Desai via kihwal)
HADOOP-10585. Retry polices ignore interrupted exceptions (Daryn Sharp via
jeagles)
HADOOP-10401. ShellBasedUnixGroupsMapping#getGroups does not always return
primary group first (Akira AJISAKA via Colin Patrick McCabe)
HADOOP-10489. UserGroupInformation#getTokens and UserGroupInformation
#addToken can lead to ConcurrentModificationException (Robert Kanter via atm)
HADOOP-10625. Trim configuration names when putting/getting them
to properties. (Wangda Tan via xgong)
HADOOP-10602. Documentation has broken "Go Back" hyperlinks.
(Akira AJISAKA via cnauroth)
HADOOP-10639. FileBasedKeyStoresFactory initialization is not using default
for SSL_REQUIRE_CLIENT_CERT_KEY. (tucu)
HADOOP-10638. Updating hadoop-daemon.sh to work as expected when nfs is
started as a privileged user. (Manikandan Narayanaswamy via atm)
HADOOP-10342. Add a new method to UGI to use a Kerberos login subject to
build a new UGI. (Larry McCay via omalley)
HADOOP-10630. Possible race condition in RetryInvocationHandler. (jing9)
HADOOP-10658. SSLFactory expects truststores being configured. (tucu via atm)
HADOOP-10647. String Format Exception in SwiftNativeFileSystemStore.java.
(Gene Kim via stevel)
HADOOP-9099. NetUtils.normalizeHostName fails on domains where
UnknownHost resolves to an IP address. (Ivan Mitic via suresh)
HADOOP-10664. TestNetUtils.testNormalizeHostName fails. (atm)
HADOOP-10656. The password keystore file is not picked by LDAP group mapping
(brandonli)
HADOOP-10622. Shell.runCommand can deadlock (Gera Shegalov via jlowe)
HADOOP-10686. Writables are not always configured.
(Abraham Elmahrek via kasha)
HADOOP-10678. SecurityUtil has unnecessary synchronization on collection
used for only tests. (Benoy Antony via cnauroth)
HADOOP-10683. Users authenticated with KERBEROS are recorded as being
authenticated with SIMPLE. (Benoy Antony via cnauroth)
HADOOP-10702. KerberosAuthenticationHandler does not log the principal names
correctly. (Benoy Antony via cnauroth)
HADOOP-10699. Fix build native library on mac osx (Binglin Chang via
jlowe)
HADOOP-10660. GraphiteSink should implement Closeable (Chen He and Ted Yu via raviprak)
HADOOP-10716. Cannot use more than 1 har filesystem.
(Rushabh Shah via cnauroth)
HADOOP-9559. When metrics system is restarted MBean names get incorrectly
flagged as dupes. (Mostafa Elhemali and Mike Liddell via cnauroth)
HADOOP-10746. TestSocketIOWithTimeout#testSocketIOWithTimeout fails on
Power PC. (Jinghui Wang via Arpit Agarwal)
HADOOP-9705. FsShell cp -p does not preserve directory attibutes.
(Akira AJISAKA via cnauroth)
HADOOP-10739. Renaming a file into a directory containing the same
filename results in a confusing I/O error (chang li via jlowe)
HADOOP-10533 S3 input stream NPEs in MapReduce join (stevel)
HADOOP-10419 BufferedFSInputStream NPEs on getPos() on a closed stream
(stevel)
HADOOP-10801 dead link in site.xml (Akira AJISAKA via stevel)
HADOOP-10590. ServiceAuthorizationManager is not threadsafe. (Benoy Antony via vinayakumarb)
HADOOP-10711. Cleanup some extra dependencies from hadoop-auth. (rkanter via tucu)
HADOOP-10479. Fix new findbugs warnings in hadoop-minikdc.
(Swarnim Kulkarni via wheat9)
HADOOP-10715. Remove public GraphiteSink#setWriter (Babak Behzad via raviprak)
HADOOP-10710. hadoop.auth cookie is not properly constructed according to
RFC2109. (Juan Yu via tucu)
HADOOP-10864. Tool documentenation is broken. (Akira Ajisaka
via Arpit Agarwal)
HADOOP-10872. TestPathData fails intermittently with "Mkdirs failed
to create d1". (Yongjun Zhang via Arpit Agarwal)
HADOOP-10890. TestDFVariations.testMount fails intermittently. (Yongjun
Zhang via Arpit Agarwal)
HADOOP-10894. Fix dead link in ToolRunner documentation. (Akira Ajisaka
via Arpit Agarwal)
HADOOP-10910. Increase findbugs maxHeap size. (wang)
BREAKDOWN OF HADOOP-10514 SUBTASKS AND RELATED JIRAS
HADOOP-10520. Extended attributes definition and FileSystem APIs for
extended attributes. (Yi Liu via wang)
HADOOP-10546. Javadoc and other small fixes for extended attributes in
hadoop-common. (Charles Lamb via wang)
HADOOP-10521. FsShell commands for extended attributes. (Yi Liu via wang)
HADOOP-10548. Improve FsShell xattr error handling and other fixes. (Charles Lamb via umamahesh)
HADOOP-10567. Shift XAttr value encoding code out for reuse. (Yi Liu via umamahesh)
HADOOP-10621. Remove CRLF for xattr value base64 encoding for better display.(Yi Liu via umamahesh)
HADOOP-10575. Small fixes for XAttrCommands and test. (Yi Liu via umamahesh)
HADOOP-10561. Copy command with preserve option should handle Xattrs.
(Yi Liu via cnauroth)
Release 2.4.1 - 2014-06-23
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
HADOOP-10466. Lower the log level in UserGroupInformation. (Nicolas
Liochon via szetszwo)
OPTIMIZATIONS
BUG FIXES
HADOOP-10455. When there is an exception, ipc.Server should first check
whether it is an terse exception. (szetszwo)
HADOOP-10456. Bug in Configuration.java exposed by Spark
(ConcurrentModificationException). (Nishkam Ravi via cnauroth)
HADOOP-10473. TestCallQueueManager should interrupt before counting calls.
(szetszwo)
HADOOP-10490. TestMapFile and TestBloomMapFile leak file descriptors.
(cnauroth)
HADOOP-10522. JniBasedUnixGroupMapping mishandles errors. (kihwal)
HADOOP-10527. Fix incorrect return code and allow more retries on EINTR.
(kihwal)
HADOOP-10612. NFS failed to refresh the user group id mapping table (brandonli)
HADOOP-10562. Namenode exits on exception without printing stack trace
in AbstractDelegationTokenSecretManager. (Suresh Srinivas via Arpit
Agarwal)
Release 2.4.0 - 2014-04-07
INCOMPATIBLE CHANGES
HADOOP-8691. FsShell can print "Found xxx items" unnecessarily often.
(Daryn Sharp via wheat9)
NEW FEATURES
HADOOP-10184. Hadoop Common changes required to support HDFS ACLs. (See
breakdown of tasks below for features and contributors)
IMPROVEMENTS
HADOOP-10139. Update and improve the Single Cluster Setup document.
(Akira Ajisaka via Arpit Agarwal)
HADOOP-10295. Allow distcp to automatically identify the checksum type of
source files and use it for the target. (jing9 and Laurent Goujon)
HADOOP-10333. Fix grammatical error in overview.html document.
(René Nyffenegger via suresh)
HADOOP-10343. Change info to debug log in LossyRetryInvocationHandler. (arpit)
HADOOP-10348. Deprecate hadoop.ssl.configuration in branch-2, and remove
it in trunk. (Haohui Mai via jing9)
HADOOP-9454. Support multipart uploads for s3native. (Jordan Mendelson and
Akira AJISAKA via atm)
HADOOP-10374. InterfaceAudience annotations should have
RetentionPolicy.RUNTIME (Enis Soztutar via Arpit Agarwal)
HADOOP-10278. Refactor to make CallQueue pluggable. (Chris Li via
Arpit Agarwal)
HADOOP-10285. Admin interface to swap callqueue at runtime. (Chris Li via
Arpit Agarwal)
HADOOP-10314. The ls command help still shows outdated 0.16 format.
(Rushabh S Shah via kihwal)
HADOOP-10379. Protect authentication cookies with the HttpOnly and Secure
flags. (wheat9)
HADOOP-10211. Enable RPC protocol to negotiate SASL-QOP values between
clients and servers. (Benoy Antony via Arpit Agarwal)
HADOOP-10386. Log proxy hostname in various exceptions being thrown in a HA
setup. (wheat9)
HADOOP-3679. Fixup assert ordering in unit tests to yield meaningful error
messages. (Jay Vyas via cdouglas)
HADOOP-10221. Add a plugin to specify SaslProperties for RPC protocol
based on connection properties. (Benoy Antony and Daryn Sharp via
Arpit Agarwal)
HADOOP-10015. UserGroupInformation prints out excessive warnings.
(Nicolas Liochon via szetszwo)
HADOOP-10423. Clarify compatibility policy document for combination of
new client and old server. (Chris Nauroth via kasha)
HADOOP-10280. Make Schedulables return a configurable identity of user
or group. (Chris Li via Arpit Agarwal)
OPTIMIZATIONS
BUG FIXES
HADOOP-10320. Javadoc in InterfaceStability.java lacks final </ul>.
(René Nyffenegger via cnauroth)
HADOOP-10085. CompositeService should allow adding services while being
inited. (Steve Loughran via kasha)
HADOOP-10327. Trunk windows build broken after HDFS-5746.
(Vinay via cnauroth)
HADOOP-10330. TestFrameDecoder fails if it cannot bind port 12345.
(Arpit Agarwal)
HADOOP-10326. M/R jobs can not access S3 if Kerberos is enabled. (bc Wong
via atm)
HADOOP-10338. Cannot get the FileStatus of the root inode from the new
Globber (cmccabe)
HADOOP-10249. LdapGroupsMapping should trim ldap password read from file.
(Dilli Armugam via suresh)
HADOOP-10346. Deadlock while logging tokens (jlowe)
HADOOP-10328. loadGenerator exit code is not reliable.
(Haohui Mai via cnauroth)
HADOOP-10355. Fix TestLoadGenerator#testLoadGenerator. (Haohui Mai via jing9)
HADOOP-10070. RPC client doesn't use per-connection conf to determine
server's expected Kerberos principal name. (atm)
HADOOP-10368. InputStream is not closed in VersionInfo ctor.
(Tsuyoshi OZAWA via szetszwo)
HADOOP-10353. FsUrlStreamHandlerFactory is not thread safe.
(Tudor Scurtu via cnauroth)
HADOOP-10393. Fix the javac warnings in hadoop-auth. (szetszwo)
HADOOP-10395. TestCallQueueManager is flaky. (Arpit Agarwal)
HADOOP-10394. TestAuthenticationFilter is flaky. (Arpit Agarwal)
HADOOP-10337 ConcurrentModificationException from
MetricsDynamicMBeanBase.createMBeanInfo() (Liang Xie via stack)
HADOOP-10407. Fix the javac warnings in org.apache.hadoop.ipc package.
(szetszwo)
HADOOP-10191. Missing executable permission on viewfs internal dirs.
(Gera Shegalov via cnauroth)
HADOOP-10422. Remove redundant logging of RPC retry attempts. (cnauroth)
HADOOP-10425. LocalFileSystem.getContentSummary should not count crc files.
(szetszwo)
HADOOP-10440. HarFsInputStream.read(byte[]) updates position incorrectly.
(guodongdong via szetszwo)
HADOOP-10437. Fix the javac warnings in the conf and the util package.
(szetszwo)
HADOOP-10441. Namenode metric "rpc.RetryCache/NameNodeRetryCache.CacheHit"
can't be correctly processed by Ganglia. (jing9)
HADOOP-10449. Fix the javac warnings in the security package. (szetszwo)
HADOOP-10450. Build zlib native code bindings in hadoop.dll for Windows.
(cnauroth)
HADOOP-10301. AuthenticationFilter should return Forbidden for failed
authentication. (Daryn Sharp via jing9)
HADOOP-9525. Add tests that validate winutils chmod behavior on folders
(ivanmi)
BREAKDOWN OF HADOOP-10184 SUBTASKS AND RELATED JIRAS
HADOOP-10185. FileSystem API for ACLs. (cnauroth)
HADOOP-10186. Remove AclReadFlag and AclWriteFlag in FileSystem API.
(Haohui Mai via cnauroth)
HADOOP-10187. FsShell CLI: add getfacl and setfacl with minimal support for
getting and setting ACLs. (Vinay via cnauroth)
HADOOP-10192. FileSystem#getAclStatus has incorrect JavaDocs. (cnauroth)
HADOOP-10220. Add ACL indicator bit to FsPermission. (cnauroth)
HADOOP-10241. Clean up output of FsShell getfacl. (Chris Nauroth via wheat9)
HADOOP-10213. Fix bugs parsing ACL spec in FsShell setfacl.
(Vinay via cnauroth)
HADOOP-10277. setfacl -x fails to parse ACL spec if trying to remove the
mask entry. (Vinay via cnauroth)
HADOOP-10270. getfacl does not display effective permissions of masked
entries. (cnauroth)
HADOOP-10344. Fix TestAclCommands after merging HADOOP-10338 patch.
(cnauroth)
HADOOP-10352. Recursive setfacl erroneously attempts to apply default ACL to
files. (cnauroth)
HADOOP-10354. TestWebHDFS fails after merge of HDFS-4685 to trunk. (cnauroth)
HADOOP-10361. Correct alignment in CLI output for ACLs. (cnauroth)
HADOOP-10399. FileContext API for ACLs. (Vinayakumar B via cnauroth)
HADOOP-10442. Group look-up can cause segmentation fault when certain
JNI-based mapping module is used. (Kihwal Lee via jeagles)
Release 2.3.1 - UNRELEASED
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
OPTIMIZATIONS
BUG FIXES
Release 2.3.0 - 2014-02-18
INCOMPATIBLE CHANGES
NEW FEATURES
HADOOP-8545. Filesystem Implementation for OpenStack Swift
(Dmitry Mezhensky, David Dobbins, Stevel via stevel)
IMPROVEMENTS
HADOOP-10046. Print a log message when SSL is enabled.
(David S. Wang via wang)
HADOOP-10079. log a warning message if group resolution takes too long.
(cmccabe)
HADOOP-9623. Update jets3t dependency to 0.9.0 (Amandeep Khurana via Colin
Patrick McCabe)
HADOOP-10132. RPC#stopProxy() should log the class of proxy when IllegalArgumentException
is encountered (Ted yu via umamahesh)
HADOOP-10248. Property name should be included in the exception where property value
is null (Akira AJISAKA via umamahesh)
HADOOP-10086. User document for authentication in secure cluster.
(Masatake Iwasaki via Arpit Agarwal)
HADOOP-10274 Lower the logging level from ERROR to WARN for UGI.doAs method
(Takeshi Miao via stack)
HADOOP-9784. Add a builder for HttpServer. (Junping Du via llu)
HADOOP 9871. Fix intermittent findbugs warnings in DefaultMetricsSystem.
(Junping Du via llu)
HADOOP-9319. Update bundled LZ4 source to r99. (Binglin Chang via llu)
HADOOP-9432 Add support for markdown .md files in site documentation (stevel)
HADOOP-9241. DU refresh interval is not configurable (harsh)
HADOOP-9417. Support for symlink resolution in LocalFileSystem /
RawLocalFileSystem. (Andrew Wang via Colin Patrick McCabe)
HADOOP-9703. org.apache.hadoop.ipc.Client leaks threads on stop.
(Tsuyoshi OZAWA vi Colin Patrick McCabe)
HADOOP-9618. Add thread which detects GC pauses.
(Todd Lipcon via Colin Patrick McCabe)
HADOOP-9848. Create a MiniKDC for use with security testing.
(ywskycn via tucu)
HADOOP-9860. Remove class HackedKeytab and HackedKeytabEncoder from
hadoop-minikdc once jira DIRSERVER-1882 solved. (ywskycn via tucu)
HADOOP-9866. convert hadoop-auth testcases requiring kerberos to
use minikdc. (ywskycn via tucu)
HADOOP-9487 Deprecation warnings in Configuration should go to their
own log or otherwise be suppressible (Chu Tong via stevel)
HADOOP-9889. Refresh the Krb5 configuration when creating a new kdc in
Hadoop-MiniKDC (Wei Yan via Sandy Ryza)
HADOOP-9915. o.a.h.fs.Stat support on Mac OS X (Binglin Chang via Colin
Patrick McCabe)
HADOOP-9998. Provide methods to clear only part of the DNSToSwitchMapping.
(Junping Du via Colin Patrick McCabe)
HADOOP-9063. enhance unit-test coverage of class
org.apache.hadoop.fs.FileUtil (Ivan A. Veselovsky via jlowe)
HADOOP-9254. Cover packages org.apache.hadoop.util.bloom,
org.apache.hadoop.util.hash (Vadim Bondarev via jlowe)
HADOOP-9225. Cover package org.apache.hadoop.compress.Snappy (Vadim
Bondarev, Andrey Klochkov and Nathan Roberts via jlowe)
HADOOP-9199. Cover package org.apache.hadoop.io with unit tests (Andrey
Klochkov via jeagles)
HADOOP-9470. eliminate duplicate FQN tests in different Hadoop modules
(Ivan A. Veselovsky via daryn)
HADOOP-9494. Excluded auto-generated and examples code from clover reports
(Andrey Klochkov via jeagles)
HADOOP-9897. Add method to get path start position without drive specifier in
o.a.h.fs.Path. (Binglin Chang via cnauroth)
HADOOP-9078. enhance unit-test coverage of class
org.apache.hadoop.fs.FileContext (Ivan A. Veselovsky via jeagles)
HDFS-5276. FileSystem.Statistics should use thread-local counters to avoid
multi-threaded performance issues on read/write. (Colin Patrick McCabe)
HADOOP-9291. enhance unit-test coverage of package o.a.h.metrics2 (Ivan A.
Veselovsky via jeagles)
HADOOP-10064. Upgrade to maven antrun plugin version 1.7 (Arpit Agarwal via
jeagles)
HADOOP-9594. Update apache commons math dependency (Timothy St. Clair via
stevel)
HADOOP-10095. In CodecPool, synchronize pool and codecList separately in
order to reduce lock contention. (Nicolas Liochon via szetszwo)
HADOOP-10067. Missing POM dependency on jsr305 (Robert Rati via stevel)
HADOOP-10103. update commons-lang to 2.6 (Akira AJISAKA via stevel)
HADOOP-10111. Allow DU to be initialized with an initial value (Kihwal Lee
via jeagles)
HADOOP-10126. LightWeightGSet log message is confusing. (Vinay via suresh)
HADOOP-10127. Add ipc.client.connect.retry.interval to control the frequency
of connection retries (Karthik Kambatla via Sandy Ryza)
HADOOP-10102. Update commons IO from 2.1 to 2.4 (Akira Ajisaka via stevel)
HADOOP-10168. fix javadoc of ReflectionUtils#copy. (Thejas Nair via suresh)
HADOOP-10164. Allow UGI to login with a known Subject (bobby)
HADOOP-10169. Remove the unnecessary synchronized in JvmMetrics class.
(Liang Xie via jing9)
HADOOP-10198. DomainSocket: add support for socketpair.
(Colin Patrick McCabe via wang)
HADOOP-10208. Remove duplicate initialization in StringUtils.getStringCollection.
(Benoy Antony via jing9)
HADOOP-9420. Add percentile or max metric for rpcQueueTime, processing time.
(Liang Xie via wang)
HADOOP-9652. Allow RawLocalFs#getFileLinkStatus to fill in the link owner
and mode if requested. (Andrew Wang via Colin Patrick McCabe)
HADOOP-10305. Add "rpc.metrics.quantile.enable" and
"rpc.metrics.percentiles.intervals" to core-default.xml.
(Akira Ajisaka via wang)
HADOOP-10317. Rename branch-2.3 release version from 2.4.0-SNAPSHOT
to 2.3.0-SNAPSHOT. (wang)
HADOOP-10313. Script and jenkins job to produce Hadoop release artifacts.
(tucu)
OPTIMIZATIONS
HADOOP-10142. Avoid groups lookup for unprivileged users such as "dr.who"
(vinay via cmccabe)
HADOOP-9748. Reduce blocking on UGI.ensureInitialized (daryn)
HADOOP-10047. Add a direct-buffer based apis for compression. (Gopal V
via acmurthy)
HADOOP-10172. Cache SASL server factories (daryn)
HADOOP-10173. Remove UGI from DIGEST-MD5 SASL server creation (daryn via
kihwal)
HADOOP-10228. FsPermission#fromShort() should cache FsAction.values().
(Haohui Mai via cnauroth)
HADOOP-10143 replace WritableFactories's hashmap with ConcurrentHashMap
(Liang Xie via stack)
BUG FIXES
HADOOP-10028. Malformed ssl-server.xml.example. (Haohui Mai via jing9)
HADOOP-10030. FsShell -put/copyFromLocal should support Windows local path.
(Chuan Liu via cnauroth)
HADOOP-10031. FsShell -get/copyToLocal/moveFromLocal should support Windows
local path. (Chuan Liu via cnauroth)
HADOOP-10039. Add Hive to the list of projects using
AbstractDelegationTokenSecretManager. (Haohui Mai via jing9)
HADOOP-10040. hadoop.cmd in UNIX format and would not run by default on
Windows. (cnauroth)
HADOOP-10055. FileSystemShell.apt.vm doc has typo "numRepicas".
(Akira Ajisaka via cnauroth)
HADOOP-10072. TestNfsExports#testMultiMatchers fails due to non-deterministic
timing around cache expiry check. (cnauroth)
HADOOP-9898. Set SO_KEEPALIVE on all our sockets. (todd via wang)
HADOOP-9478. Fix race conditions during the initialization of Configuration
related to deprecatedKeyMap (cmccabe)
HADOOP-9660. [WINDOWS] Powershell / cmd parses -Dkey=value from command line
as [-Dkey, value] which breaks GenericsOptionParser.
(Enis Soztutar via cnauroth)
HADOOP-10078. KerberosAuthenticator always does SPNEGO. (rkanter via tucu)
HADOOP-10110. hadoop-auth has a build break due to missing dependency.
(Chuan Liu via arp)
HADOOP-9114. After defined the dfs.checksum.type as the NULL, write file and hflush will
through java.lang.ArrayIndexOutOfBoundsException (Sathish via umamahesh)
HADOOP-10130. RawLocalFS::LocalFSFileInputStream.pread does not track
FS::Statistics (Binglin Chang via Colin Patrick McCabe)
HDFS-5560. Trash configuration log statements prints incorrect units.
(Josh Elser via Andrew Wang)
HADOOP-10081. Client.setupIOStreams can leak socket resources on exception
or error (Tsuyoshi OZAWA via jlowe)
HADOOP-10087. UserGroupInformation.getGroupNames() fails to return primary
group first when JniBasedUnixGroupsMappingWithFallback is used (cmccabe)
HADOOP-10175. Har files system authority should preserve userinfo.
(Chuan Liu via cnauroth)
HADOOP-10090. Jobtracker metrics not updated properly after execution
of a mapreduce job. (ivanmi)
HADOOP-10193. hadoop-auth's PseudoAuthenticationHandler can consume getInputStream.
(gchanan via tucu)
HADOOP-10178. Configuration deprecation always emit "deprecated" warnings
when a new key is used. (Shanyu Zhao via cnauroth)
HADOOP-10234. "hadoop.cmd jar" does not propagate exit code. (cnauroth)
HADOOP-10240. Windows build instructions incorrectly state requirement of
protoc 2.4.1 instead of 2.5.0. (cnauroth)
HADOOP-10110. hadoop-auth has a build break due to missing dependency
(Chuan Liu via cos)
HADOOP-10167. Mark hadoop-common source as UTF-8 in Maven pom files / refactoring
(Mikhail Antonov via cos)
HADOOP-9982. Fix dead links in hadoop site docs. (Akira Ajisaka via Arpit
Agarwal)
HADOOP-10212. Incorrect compile command in Native Library document.
(Akira Ajisaka via Arpit Agarwal)
HADOOP-9830. Fix typo at http://hadoop.apache.org/docs/current/
(Kousuke Saruta via Arpit Agarwal)
HADOOP-10255. Rename HttpServer to HttpServer2 to retain older
HttpServer in branch-2 for compatibility. (Haohui Mai via suresh)
HADOOP-10292. Restore HttpServer from branch-2.2 in branch-2.
(Haohui Mai via suresh)
HADOOP-10291. TestSecurityUtil#testSocketAddrWithIP fails due to test
order dependency. (Mit Desai via Arpit Agarwal)
HADOOP-9964. Fix deadlocks in TestHttpServer by synchronize
ReflectionUtils.printThreadInfo. (Junping Du via llu)
HADOOP-9582. Non-existent file to "hadoop fs -conf" doesn't throw error
(Ashwin Shankar via jlowe)
HADOOP-9817. FileSystem#globStatus and FileContext#globStatus need to work
with symlinks. (Colin Patrick McCabe via Andrew Wang)
HADOOP-9875. TestDoAsEffectiveUser can fail on JDK 7. (Aaron T. Myers via
Colin Patrick McCabe)
HADOOP-9865. FileContext#globStatus has a regression with respect to
relative path. (Chuan Lin via Colin Patrick McCabe)
HADOOP-9909. org.apache.hadoop.fs.Stat should permit other LANG.
(Shinichi Yamashita via Andrew Wang)
HADOOP-9908. Fix NPE when versioninfo properties file is missing (todd)
HADOOP-9929. Insufficient permissions for a path reported as File Not
Found. (Contributed by Colin Patrick McCabe)
HADOOP-9791. Add a test case covering long paths for new FileUtil access
check methods (ivanmi)
HADOOP-9981. globStatus should minimize its listStatus and getFileStatus
calls. (Contributed by Colin Patrick McCabe)
HADOOP-10006. Compilation failure in trunk for
o.a.h.fs.swift.util.JSONUtil (Junping Du via stevel)
HADOOP-9016. HarFsInputStream.skip(long) must never return negative value.
(Ivan A. Veselovsky via jeagles)
HADOOP-10088. copy-nativedistlibs.sh needs to quote snappy lib dir.
(Raja Aluri via cnauroth)
HADOOP-10093. hadoop-env.cmd sets HADOOP_CLIENT_OPTS with a max heap size
that is too small. (Shanyu Zhao via cnauroth)
HADOOP-10094. NPE in GenericOptionsParser#preProcessForWindows().
(Enis Soztutar via cnauroth)
HADOOP-10100. MiniKDC shouldn't use apacheds-all artifact. (rkanter via tucu)
HADOOP-10107. Server.getNumOpenConnections may throw NPE. (Kihwal Lee via
jing9)
HADOOP-10135 writes to swift fs over partition size leave temp files and
empty output file (David Dobbins via stevel)
HADOOP-10129. Distcp may succeed when it fails (daryn)
HADOOP-10058. TestMetricsSystemImpl#testInitFirstVerifyStopInvokedImmediately
fails on trunk (Chen He via jeagles)
HADOOP-10162. Fix symlink-related test failures in
TestFileContextResolveAfs and TestStat in branch-2 (Mit Desai via Colin
Patrick McCabe)
HADOOP-8753. LocalDirAllocator throws "ArithmeticException: / by zero" when
there is no available space on configured local dir. (Benoy Antony via hitesh)
HADOOP-10106. Incorrect thread name in RPC log messages. (Ming Ma via jing9)
HADOOP-9611 mvn-rpmbuild against google-guice > 3.0 yields missing cglib
dependency (Timothy St. Clair via stevel)
HADOOP-10171. TestRPC fails intermittently on jkd7 (Mit Desai via jeagles)
HADOOP-10147 HDFS-5678 Upgrade to commons-logging 1.1.3 to avoid potential
deadlock in MiniDFSCluster (stevel)
HADOOP-10207. TestUserGroupInformation#testLogin is flaky (jxiang via
cmccabe)
HADOOP-10214. Fix multithreaded correctness warnings in ActiveStandbyElector
(Liang Xie via kasha)
HADOOP-10223. MiniKdc#main() should close the FileReader it creates.
(Ted Yu via tucu)
HADOOP-10236. Fix typo in o.a.h.ipc.Client#checkResponse. (Akira Ajisaka
via suresh)
HADOOP-10146. Workaround JDK7 Process fd close bug (daryn)
HADOOP-10125. no need to process RPC request if the client connection
has been dropped (Ming Ma via brandonli)
HADOOP-10235. Hadoop tarball has 2 versions of stax-api JARs. (tucu)
HADOOP-10252. HttpServer can't start if hostname is not specified. (Jimmy
Xiang via atm)
HADOOP-10203. Connection leak in
Jets3tNativeFileSystemStore#retrieveMetadata. (Andrei Savu via atm)
HADOOP-10250. VersionUtil returns wrong value when comparing two versions.
(Yongjun Zhang via atm)
HADOOP-10288. Explicit reference to Log4JLogger breaks non-log4j users
(todd)
HADOOP-10310. SaslRpcServer should be initialized even when no secret
manager present. (atm)
HADOOP-10311. Cleanup vendor names from the code base. (tucu)
HADOOP-10273. Fix 'mvn site'. (Arpit Agarwal)
Release 2.2.0 - 2013-10-13
INCOMPATIBLE CHANGES
HADOOP-10020. Disable symlinks temporarily (cmccabe)
NEW FEATURES
IMPROVEMENTS
HADOOP-9948. Add a config value to CLITestHelper to skip tests on Windows.
(Chuan Liu via cnauroth)
HADOOP-9976. Different versions of avro and avro-maven-plugin (Karthik
Kambatla via Sandy Ryza)
HADOOP-9758. Provide configuration option for FileSystem/FileContext
symlink resolution. (Andrew Wang via Colin Patrick McCabe)
HADOOP-8315. Support SASL-authenticated ZooKeeper in ActiveStandbyElector
(todd)
OPTIMIZATIONS
BUG FIXES
HADOOP-9776. HarFileSystem.listStatus() returns invalid authority if port
number is empty. (Shanyu Zhao via ivanmi)
HADOOP-9761. ViewFileSystem#rename fails when using DistributedFileSystem.
(Andrew Wang via Colin Patrick McCabe)
HADOOP-10003. HarFileSystem.listLocatedStatus() fails.
(Jason Dere and suresh via suresh)
HADOOP-10017. Fix NPE in DFSClient#getDelegationToken when doing Distcp
from a secured cluster to an insecured cluster. (Haohui Mai via jing9)
Release 2.1.1-beta - 2013-09-23
INCOMPATIBLE CHANGES
HADOOP-9944. Fix RpcRequestHeaderProto.callId to be sint32 rather than
uint32 since ipc.Client.CONNECTION_CONTEXT_CALL_ID is signed (i.e. -3)
(acmurthy)
NEW FEATURES
IMPROVEMENTS
HADOOP-9910. proxy server start and stop documentation wrong
(Andre Kelpe via harsh)
HADOOP-9446. Support Kerberos SPNEGO for IBM JDK. (Yu Gao via llu)
HADOOP-9787. ShutdownHelper util to shutdown threads and threadpools.
(Karthik Kambatla via Sandy Ryza)
HADOOP-9803. Add a generic type parameter to RetryInvocationHandler.
(szetszwo)
HADOOP-9821. ClientId should have getMsb/getLsb methods.
(Tsuyoshi OZAWA via jing9)
HADOOP-9435. Support building the JNI code against the IBM JVM.
(Tian Hong Wang via Colin Patrick McCabe)
HADOOP-9833 move slf4j to version 1.7.5 (Kousuke Saruta via stevel)
HADOOP-9672. Upgrade Avro dependency to 1.7.4. (sandy via kihwal)
HADOOP-9789. Support server advertised kerberos principals (daryn)
HADOOP-8814. Replace string equals "" by String#isEmpty().
(Brandon Li via suresh)
HADOOP-9802. Support Snappy codec on Windows. (cnauroth)
HADOOP-9879. Move the version info of zookeeper dependencies to
hadoop-project/pom (Karthik Kambatla via Sandy Ryza)
HADOOP-9886. Turn warning message in RetryInvocationHandler to debug (arpit)
HADOOP-9906. Move HAZKUtil to o.a.h.util.ZKUtil and make inner-classes
public (Karthik Kambatla via Sandy Ryza)
HADOOP-9918. Add addIfService to CompositeService (Karthik Kambatla via
Sandy Ryza)
HADOOP-9945. HAServiceState should have a state for stopped services.
(Karthik Kambatla via atm)
HADOOP-9962. in order to avoid dependency divergence within Hadoop itself
lets enable DependencyConvergence. (rvs via tucu)
HADOOP-9669. Reduce the number of byte array creations and copies in
XDR data manipulation. (Haohui Mai via brandonli)
OPTIMIZATIONS
BUG FIXES
HADOOP-9916. Fix race in ipc.Client retry. (Binglin Chang via llu)
HADOOP-9768. chown and chgrp reject users and groups with spaces on platforms
where spaces are otherwise acceptable. (cnauroth)
HADOOP-9801. Configuration#writeXml uses platform defaulting encoding, which
may mishandle multi-byte characters. (cnauroth)
HADOOP-9806 PortmapInterface should check if the procedure is out-of-range
(brandonli)
HADOOP-9527. Add symlink support to LocalFileSystem on Windows.
(Arpit Agarwal via cnauroth)
HADOOP-9315. Port HADOOP-9249 hadoop-maven-plugins Clover fix to branch-2 to
fix build failures. (Dennis Y via cnauroth)
HADOOP-9831. Make checknative shell command accessible on Windows. (cnauroth)
HADOOP-9675. use svn:eol-style native for html to prevent line ending
issues (Colin Patrick McCabe)
HADOOP-9757. Har metadata cache can grow without limit (Cristina Abad via daryn)
HADOOP-9858. Remove unused private RawLocalFileSystem#execCommand method from
branch-2. (cnauroth)
HADOOP-9857. Tests block and sometimes timeout on Windows due to invalid
entropy source. (cnauroth)
HADOOP-9381. Document dfs cp -f option. (Keegan Witt, suresh via suresh)
HADOOP-9868. Server must not advertise kerberos realm. (daryn via kihwal)
HADOOP-9880. SASL changes from HADOOP-9421 breaks Secure HA NN. (daryn via
jing9)
HADOOP-9887. globStatus does not correctly handle paths starting with a drive
spec on Windows. (Chuan Liu via cnauroth)
HADOOP-9899. Remove the debug message, added by HADOOP-8855, from
KerberosAuthenticator. (szetszwo)
HADOOP-9894. Race condition in Shell leads to logged error stream handling
exceptions (Arpit Agarwal)
HADOOP-9774. RawLocalFileSystem.listStatus() return absolute paths when
input path is relative on Windows. (Shanyu Zhao via ivanmi)
HADOOP-9924. FileUtil.createJarWithClassPath() does not generate relative
classpath correctly. (Shanyu Zhao via ivanmi)
HADOOP-9932. Improper synchronization in RetryCache. (kihwal)
HADOOP-9958. Add old constructor back to DelegationTokenInformation to
unbreak downstream builds. (Andrew Wang)
HADOOP-9960. Upgrade Jersey version to 1.9. (Karthik Kambatla via atm)
HADOOP-9557. hadoop-client excludes commons-httpclient. (Lohit Vijayarenu via
cnauroth)
HADOOP-9350. Hadoop not building against Java7 on OSX
(Robert Kanter via stevel)
HADOOP-9961. versions of a few transitive dependencies diverged between hadoop
subprojects. (rvs via tucu)
HADOOP-9977. Hadoop services won't start with different keypass and
keystorepass when https is enabled. (cnauroth)
HADOOP-10005. No need to check INFO severity level is enabled or not.
(Jackie Chang via suresh)
Release 2.1.0-beta - 2013-08-22
INCOMPATIBLE CHANGES
HADOOP-8886. Remove KFS support. (eli)
HADOOP-9163. [RPC v9] The rpc msg in ProtobufRpcEngine.proto should be moved out to
avoid an extra copy (Sanjay Radia)
HADOOP-9151. [RPC v9] Include RPC error info in RpcResponseHeader instead of sending
it separately (sanjay Radia)
HADOOP-9380. [RPC v9] Add totalLength to rpc response (sanjay Radia)
HADOOP-9425. [RPC v9] Add error codes to rpc-response (sanjay Radia)
HADOOP-9194. [RPC v9] RPC support for QoS. (Junping Du via llu)
HADOOP-9630. [RPC v9] Remove IpcSerializationType. (Junping Du via llu)
HADOOP-9421. [RPC v9] Convert SASL to use ProtoBuf and provide
negotiation capabilities (daryn)
HADOOP-9688. Add globally unique Client ID to RPC requests. (suresh)
HADOOP-9683. [RPC v9] Wrap IpcConnectionContext in RPC headers (daryn)
HADOOP-9698. [RPC v9] Client must honor server's SASL negotiate response (daryn)
HADOOP-9832. [RPC v9] Add RPC header to client ping (daryn)
HADOOP-9820. [RPC v9] Wire protocol is insufficient to support multiplexing. (daryn via jitendra)
NEW FEATURES
HADOOP-9283. Add support for running the Hadoop client on AIX. (atm)
HADOOP-8415. Add getDouble() and setDouble() in
org.apache.hadoop.conf.Configuration (Jan van der Lugt via harsh)
HADOOP-9338. FsShell Copy Commands Should Optionally Preserve File
Attributes. (Nick White via atm)
HADOOP-8562. Enhancements to support Hadoop on Windows Server and Windows
Azure environments. (See breakdown of tasks below for subtasks and
contributors)
HADOOP-8469. Make NetworkTopology class pluggable. (Junping Du via
szetszwo)
HADOOP-8470. Add NetworkTopologyWithNodeGroup, a 4-layer implementation
of NetworkTopology. (Junping Du via szetszwo)
HADOOP-9763. Extends LightWeightGSet to support eviction of expired
elements. (Tsz Wo (Nicholas) SZE via jing9)
HADOOP-9762. RetryCache utility for implementing RPC retries.
(Suresh Srinivas via jing9)
HADOOP-9792. Retry the methods that are tagged @AtMostOnce along
with @Idempotent. (suresh)
HADOOP-9509. Implement ONCRPC and XDR. (brandonli)
HADOOP-9515. Add general interface for NFS and Mount. (brandonli)
IMPROVEMENTS
HADOOP-9164. Print paths of loaded native libraries in
NativeLibraryChecker. (Binglin Chang via llu)
HADOOP-9253. Capture ulimit info in the logs at service start time.
(Arpit Gupta via suresh)
HADOOP-8924. Add maven plugin alternative to shell script to save
package-info.java. (Chris Nauroth via suresh)
HADOOP-9117. replace protoc ant plugin exec with a maven plugin. (tucu)
HADOOP-9279. Document the need to build hadoop-maven-plugins for
eclipse and separate project builds. (Tsuyoshi Ozawa via suresh)
HADOOP-9334. Upgrade netty version. (Nicolas Liochon via suresh)
HADOOP-9343. Allow additional exceptions through the RPC layer. (sseth)
HADOOP-9318. When exiting on a signal, print the signal name first. (Colin
Patrick McCabe via atm)
HADOOP-9358. "Auth failed" log should include exception string (todd)
HADOOP-9401. CodecPool: Add counters for number of (de)compressors
leased out. (kkambatl via tucu)
HADOOP-9450. HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH
is PREpended instead of APpended. (Chris Nauroth and harsh via harsh)
HADOOP-9496. Bad merge of HADOOP-9450 on branch-2 breaks all bin/hadoop
calls that need HADOOP_CLASSPATH. (harsh)
HADOOP-9503. Remove sleep between IPC client connect timeouts.
(Varun Sharma via szetszwo)
HADOOP-9322. LdapGroupsMapping doesn't seem to set a timeout for
its directory search. (harsh)
HADOOP-9523. Provide a generic IBM java vendor flag in PlatformName.java
to support non-Sun JREs. (Tian Hong Wang via suresh)
HADOOP-9511. Adding support for additional input streams (FSDataInputStream
and RandomAccessFile) in SecureIOUtils so as to help YARN-578. (Omkar Vinit
Joshi via vinodkv)
HADOOP-9560. metrics2#JvmMetrics should have max memory size of JVM.
(Tsuyoshi Ozawa via suresh)
HADOOP-9140 Cleanup rpc PB protos (sanjay Radia)
HADOOP-9218 Document the Rpc-wrappers used internally (sanjay Radia)
HADOOP-9574. Added new methods in AbstractDelegationTokenSecretManager for
helping YARN ResourceManager to reuse code for RM restart. (Jian He via
vinodkv)
HADOOP-7391 Document Interface Classification from HADOOP-5073 (sanjay Radia)
HADOOP-9287. Parallel-testing hadoop-common (Andrey Klochkov via jlowe)
HADOOP-9604. Javadoc of FSDataOutputStream is slightly inaccurate. (Jingguo
Yao via atm)
HADOOP-9625. HADOOP_OPTS not picked up by hadoop command.
(Paul Han via arpit)
HADOOP-9649. Promoted YARN service life-cycle libraries into Hadoop Common
for usage across all Hadoop projects. (Zhijie Shen via vinodkv)
HADOOP-9517. Documented various aspects of compatibility for Apache
Hadoop. (Karthik Kambatla via acmurthy)
HADOOP-8608. Add Configuration API for parsing time durations. (cdouglas)
HADOOP-9619 Mark stability of .proto files (sanjay Radia)
HADOOP-9676. Make maximum RPC buffer size configurable (Colin Patrick
McCabe)
HADOOP-9691. RPC clients can generate call ID using AtomicInteger instead of
synchronizing on the Client instance. (cnauroth)
HADOOP-9661. Allow metrics sources to be extended. (sandyr via tucu)
HADOOP-9370. Write FSWrapper class to wrap FileSystem and FileContext for
better test coverage. (Andrew Wang via Colin Patrick McCabe)
HADOOP-9355. Abstract symlink tests to use either FileContext or
FileSystem. (Andrew Wang via Colin Patrick McCabe)
HADOOP-9673. NetworkTopology: when a node can't be added, print out its
location for diagnostic purposes. (Colin Patrick McCabe)
HADOOP-9414. Refactor out FSLinkResolver and relevant helper methods.
(Andrew Wang via Colin Patrick McCabe)
HADOOP-9416. Add new symlink resolution methods in FileSystem and
FileSystemLinkResolver. (Andrew Wang via Colin Patrick McCabe)
HADOOP-9720. Rename Client#uuid to Client#clientId. (Arpit Agarwal via
suresh)
HADOOP-9734. Common protobuf definitions for GetUserMappingsProtocol,
RefreshAuthorizationPolicyProtocol and RefreshUserMappingsProtocol (jlowe)
HADOOP-9716. Rpc retries should use the same call ID as the original call.
(szetszwo)
HADOOP-9717. Add retry attempt count to the RPC requests. (jing9)
HADOOP-9751. Add clientId and retryCount to RpcResponseHeaderProto.
(szetszwo)
HADOOP-9754. Remove unnecessary "throws IOException/InterruptedException",
and fix generic and other javac warnings. (szetszwo)
HADOOP-9760. Move GSet and related classes to common from HDFS.
(suresh)
HADOOP-9756. Remove the deprecated getServer(..) methods from RPC.
(Junping Du via szetszwo)
HADOOP-9770. Make RetryCache#state non volatile. (suresh)
HADOOP-9786. RetryInvocationHandler#isRpcInvocation should support
ProtocolTranslator. (suresh and jing9)
HADOOP-9847. TestGlobPath symlink tests fail to cleanup properly.
(cmccabe via wang)
OPTIMIZATIONS
HADOOP-9150. Avoid unnecessary DNS resolution attempts for logical URIs
(todd)
HADOOP-9845. Update protobuf to 2.5 from 2.4.x. (tucu)
HADOOP-9872. Improve protoc version handling and detection. (tucu)
BUG FIXES
HADOOP-9451. Fault single-layer config if node group topology is enabled.
(Junping Du via llu)
HADOOP-9294. GetGroupsTestBase fails on Windows. (Chris Nauroth via suresh)
HADOOP-9305. Add support for running the Hadoop client on 64-bit AIX. (atm)
HADOOP-9245. mvn clean without running mvn install before fails.
(Karthik Kambatla via suresh)
HADOOP-9246 Execution phase for hadoop-maven-plugin should be
process-resources (Karthik Kambatla and Chris Nauroth via jlowe)
HADOOP-9297. remove old record IO generation and tests. (tucu)
HADOOP-9154. SortedMapWritable#putAll() doesn't add key/value classes to
the map. (Karthik Kambatla via tomwhite)
HADOOP-9304. remove addition of avro genreated-sources dirs to build. (tucu)
HADOOP-9267. hadoop -help, -h, --help should show usage instructions.
(Andrew Wang via atm)
HADOOP-8569. CMakeLists.txt: define _GNU_SOURCE and _LARGEFILE_SOURCE.
(Colin Patrick McCabe via atm)
HADOOP-9323. Fix typos in API documentation. (suresh)
HADOOP-7487. DF should throw a more reasonable exception when mount cannot
be determined. (Andrew Wang via atm)
HADOOP-8917. add LOCALE.US to toLowerCase in SecurityUtil.replacePattern.
(Arpit Gupta via suresh)
HADOOP-9342. Remove jline from distribution. (thw via tucu)
HADOOP-9230. TestUniformSizeInputFormat fails intermittently.
(kkambatl via tucu)
HADOOP-9349. Confusing output when running hadoop version from one hadoop
installation when HADOOP_HOME points to another. (sandyr via tucu)
HADOOP-9337. org.apache.hadoop.fs.DF.getMount() does not work on Mac OS.
(Ivan A. Veselovsky via atm)
HADOOP-9369. DNS#reverseDns() can return hostname with . appended at the
end. (Karthik Kambatla via atm)
HADOOP-9379. capture the ulimit info after printing the log to the
console. (Arpit Gupta via suresh)
HADOOP-9399. protoc maven plugin doesn't work on mvn 3.0.2 (todd)
HADOOP-9407. commons-daemon 1.0.3 dependency has bad group id causing
build issues. (Sangjin Lee via suresh)
HADOOP-9405. TestGridmixSummary#testExecutionSummarizer is broken. (Andrew
Wang via atm)
HADOOP-9430. TestSSLFactory fails on IBM JVM. (Amir Sanjar via suresh)
HADOOP-9125. LdapGroupsMapping threw CommunicationException after some
idle time. (Kai Zheng via atm)
HADOOP-9429. TestConfiguration fails with IBM JAVA. (Amir Sanjar via
suresh)
HADOOP-9222. Cover package with org.apache.hadoop.io.lz4 unit tests (Vadim
Bondarev via jlowe)
HADOOP-9233. Cover package org.apache.hadoop.io.compress.zlib with unit
tests (Vadim Bondarev via jlowe)
HADOOP-9211. Set default max heap size in HADOOP_CLIENT_OPTS to 512m
in order to avoid OOME. (Plamen Jeliazkov via shv)
HADOOP-9473. Typo in FileUtil copy() method. (Glen Mazza via suresh)
HADOOP-9504. MetricsDynamicMBeanBase has concurrency issues in
createMBeanInfo (Liang Xie via jlowe)
HADOOP-9455. HADOOP_CLIENT_OPTS appended twice causes JVM failures.
(Chris Nauroth via suresh)
HADOOP-9550. Remove aspectj dependency. (kkambatl via tucu)
HADOOP-9549. WebHdfsFileSystem hangs on close(). (daryn via kihwal)
HADOOP-9485. No default value in the code for
hadoop.rpc.socket.factory.class.default. (Colin Patrick McCabe via atm)
HADOOP-9459. ActiveStandbyElector can join election even before
Service HEALTHY, and results in null data at ActiveBreadCrumb.
(Vinay and todd via todd)
HADOOP-9307. BufferedFSInputStream.read returns wrong results
after certain seeks. (todd)
HADOOP-9220. Unnecessary transition to standby in ActiveStandbyElector.
(tom and todd via todd)
HADOOP-9563. Fix incompatibility introduced by HADOOP-9523.
(Tian Hong Wang via suresh)
HADOOP-9566. Performing direct read using libhdfs sometimes raises SIGPIPE
(which in turn throws SIGABRT) causing client crashes. (Colin Patrick
McCabe via atm)
HADOOP-9481. Broken conditional logic with HADOOP_SNAPPY_LIBRARY. (Vadim
Bondarev via atm)
HADOOP-9593. stack trace printed at ERROR for all yarn clients without
hadoop.home set (stevel)
HADOOP-8957. AbstractFileSystem#IsValidName should be overridden for
embedded file systems like ViewFs (Chris Nauroth via Sanjay Radia)
HADOOP-9607. Fixes in Javadoc build (Timothy St. Clair via cos)
HADOOP-9605. Update junit dependency. (Timothy St. Clair via cos)
HADOOP-9581. hadoop --config non-existent directory should result in error
(Ashwin Shankar via jlowe)
HADOOP-9638. Parallel test changes caused invalid test path for several HDFS
tests on Windows (Andrey Klochkov via cnauroth)
HADOOP-9632. TestShellCommandFencer will fail if there is a 'host' machine in
the network. (Chuan Liu via cnauroth)
HADOOP-9624. TestFSMainOperationsLocalFileSystem failed when the Hadoop test
root path has "X" in its name. (Xi Fang via cnauroth)
HADOOP-9439. JniBasedUnixGroupsMapping: fix some crash bugs (Colin Patrick
McCabe)
HADOOP-9656. Gridmix unit tests fail on Windows and Linux. (Chuan Liu via
cnauroth)
HADOOP-9707. Fix register lists for crc32c inline assembly. (todd via
kihwal)
HADOOP-9738. TestDistCh fails. (jing9 via kihwal)
HADOOP-9759. Add support for NativeCodeLoader#getLibraryName on Windows.
(Chuan Liu via cnauroth)
HADOOP-9773. TestLightWeightCache should not set size limit to zero when
testing it. (szetszwo)
HADOOP-9507. LocalFileSystem rename() is broken in some cases when
destination exists. (cnauroth)
HADOOP-9816. RPC Sasl QOP is broken (daryn)
HADOOP-9850. RPC kerberos errors don't trigger relogin. (daryn via kihwal)
BREAKDOWN OF HADOOP-8562 SUBTASKS AND RELATED JIRAS
HADOOP-8924. Hadoop Common creating package-info.java must not depend on
sh. (Chris Nauroth via suresh)
HADOOP-8945. Merge winutils from branch-1-win to branch-trunk-win.
(Bikas Saha, Chuan Liu, Giridharan Kesavan, Ivan Mitic, and Steve Maine
ported by Chris Nauroth via suresh)
HADOOP-8946. winutils: compile codebase during Maven build on
branch-trunk-win. (Chris Nauroth via suresh)
HADOOP-8947. Merge FileUtil and Shell changes from branch-1-win to
branch-trunk-win to enable initial test pass. (Raja Aluri, Davio Lao,
Sumadhur Reddy Bolli, Ahmed El Baz, Kanna Karanam, Chuan Liu,
Ivan Mitic, Chris Nauroth, and Bikas Saha via suresh)
HADOOP-8954. "stat" executable not found on Windows. (Bikas Saha, Ivan Mitic
ported by Chris Narouth via suresh)
HADOOP-8959. TestUserGroupInformation fails on Windows due to "id" executable
not found. (Bikas Saha, Ivan Mitic, ported by Chris Narouth via suresh)
HADOOP-8955. "chmod" executable not found on Windows.
(Chris Nauroth via suresh)
HADOOP-8960. TestMetricsServlet fails on Windows. (Ivan Mitic via suresh)
HADOOP-8961. GenericOptionsParser URI parsing failure on Windows.
(Ivan Mitic via suresh)
HADOOP-8949. Remove FileUtil.CygPathCommand dead code. (Chris Nauroth via
suresh)
HADOOP-8956. FileSystem.primitiveMkdir failures on Windows cause multiple
test suites to fail. (Chris Nauroth via suresh)
HADOOP-8978. TestTrash fails on Windows. (Chris Nauroth via suresh)
HADOOP-8979. TestHttpServer fails on Windows. (Chris Nauroth via suresh)
HADOOP-8953. Shell PathData parsing failures on Windows. (Arpit Agarwal via
suresh)
HADOOP-8975. TestFileContextResolveAfs fails on Windows. (Chris Nauroth via
suresh)
HADOOP-8977. Multiple FsShell test failures on Windows. (Chris Nauroth via
suresh)
HADOOP-9005. Merge hadoop cmd line scripts from branch-1-win. (David Lao,
Bikas Saha, Lauren Yang, Chuan Liu, Thejas M Nair and Ivan Mitic via suresh)
HADOOP-9008. Building hadoop tarball fails on Windows. (Chris Nauroth via
suresh)
HADOOP-9011. saveVersion.py does not include branch in version annotation.
(Chris Nauroth via suresh)
HADOOP-9110. winutils ls off-by-one error indexing MONTHS array can cause
access violation. (Chris Nauroth via suresh)
HADOOP-9056. Build native library on Windows. (Chuan Liu, Arpit Agarwal via
suresh)
HADOOP-9144. Fix findbugs warnings. (Chris Nauroth via suresh)
HADOOP-9081. Add TestWinUtils. (Chuan Liu, Ivan Mitic, Chris Nauroth,
and Bikas Saha via suresh)
HADOOP-9146. Fix sticky bit regression on branch-trunk-win.
(Chris Nauroth via suresh)
HADOOP-9266. Fix javac, findbugs, and release audit warnings on
branch-trunk-win. (Chris Nauroth via suresh)
HADOOP-9270. Remove a stale java comment from FileUtil. (Chris Nauroth via
szetszwo)
HADOOP-9271. Revert Python build scripts from branch-trunk-win.
(Chris Nauroth via suresh)
HADOOP-9313. Remove spurious mkdir from hadoop-config.cmd.
(Ivan Mitic via suresh)
HADOOP-9309. Test failures on Windows due to UnsatisfiedLinkError
in NativeCodeLoader#buildSupportsSnappy. (Arpit Agarwal via suresh)
HADOOP-9347. Add instructions to BUILDING.txt describing how to
build on Windows. (Chris Nauroth via suresh)
HADOOP-9348. Address TODO in winutils to add more command line usage
and examples. (Chris Nauroth via suresh)
HADOOP-9354. Windows native project files missing license headers.
(Chris Nauroth via suresh)
HADOOP-9356. Remove remaining references to cygwin/cygpath from scripts.
(Chris Nauroth via suresh)
HADOOP-9232. JniBasedUnixGroupsMappingWithFallback fails on Windows
with UnsatisfiedLinkError. (Ivan Mitic via suresh)
HADOOP-9368. Add timeouts to new tests in branch-trunk-win.
(Arpit Agarwal via suresh)
HADOOP-9373. Merge CHANGES.branch-trunk-win.txt to CHANGES.txt.
(suresh)
HADOOP-9372. Fix bad timeout annotations on tests.
(Arpit Agarwal via suresh)
HADOOP-9376. TestProxyUserFromEnv fails on a Windows domain joined machine.
(Ivan Mitic via suresh)
HADOOP-9365. TestHAZKUtil fails on Windows. (Ivan Mitic via suresh)
HADOOP-9364. PathData#expandAsGlob does not return correct results for
absolute paths on Windows. (Ivan Mitic via suresh)
HADOOP-8973. DiskChecker cannot reliably detect an inaccessible disk on
Windows with NTFS ACLs. (Chris Nauroth via suresh)
HADOOP-9388. TestFsShellCopy fails on Windows. (Ivan Mitic via suresh)
HADOOP-9387. Fix DF so that it won't execute a shell command on Windows
to compute the file system/mount point. (Ivan Mitic via szetszwo)
HADOOP-9353. Activate native-win maven profile by default on Windows.
(Arpit Agarwal via szetszwo)
HADOOP-9437. TestNativeIO#testRenameTo fails on Windows due to assumption
that POSIX errno is embedded in NativeIOException. (Chris Nauroth via
suresh)
HADOOP-9443. Port winutils static code analysis change to trunk.
(Chuan Liu via suresh)
HADOOP-9290. Some tests cannot load native library on windows.
(Chris Nauroth via suresh)
HADOOP-9500. TestUserGroupInformation#testGetServerSideGroups fails on
Windows due to failure to find winutils.exe. (Chris Nauroth via suresh)
HADOOP-9490. LocalFileSystem#reportChecksumFailure not closing the
checksum file handle before rename. (Ivan Mitic via suresh)
HADOOP-9524. Fix ShellCommandFencer to work on Windows.
(Arpit Agarwal via suresh)
HADOOP-9413. Add common utils for File#setReadable/Writable/Executable &
File#canRead/Write/Execute that work cross-platform. (Ivan Mitic via suresh)
HADOOP-9532. HADOOP_CLIENT_OPTS is appended twice by Windows cmd scripts.
(Chris Nauroth via suresh)
HADOOP-9043. Disallow in winutils creating symlinks with forwards slashes.
(Chris Nauroth and Arpit Agarwal via suresh)
HADOOP-9483. winutils support for readlink command.
(Arpit Agarwal via suresh)
HADOOP-9488. FileUtil#createJarWithClassPath only substitutes environment
variables from current process environment/does not support overriding
when launching new process (Chris Nauroth via bikas)
HADOOP-9556. disable HA tests on Windows that fail due to ZooKeeper client
connection management bug. (Chris Nauroth via suresh)
HADOOP-9553. TestAuthenticationToken fails on Windows.
(Arpit Agarwal via suresh)
HADOOP-9397. Incremental dist tar build fails. (Chris Nauroth via jlowe)
HADOOP-9131. Turn off TestLocalFileSystem#testListStatusWithColons on
Windows. (Chris Nauroth via suresh)
HADOOP-9526. TestShellCommandFencer and TestShell fail on Windows.
(Arpit Agarwal via suresh)
HADOOP-8982. TestSocketIOWithTimeout fails on Windows.
(Chris Nauroth via suresh)
HADOOP-8958. ViewFs:Non absolute mount name failures when running
multiple tests on Windows. (Chris Nauroth via suresh)
HADOOP-9599. hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly.
(Mostafa Elhemali via ivanmi)
HADOOP-9637. Adding Native Fstat for Windows as needed by YARN. (Chuan Liu
via cnauroth)
HADOOP-9264. Port change to use Java untar API on Windows from
branch-1-win to trunk. (Chris Nauroth via suresh)
HADOOP-9678. TestRPC#testStopsAllThreads intermittently fails on Windows.
(Ivan Mitic via cnauroth)
HADOOP-9681. FileUtil.unTarUsingJava() should close the InputStream upon
finishing. (Chuan Liu via cnauroth)
HADOOP-9665. Fixed BlockDecompressorStream#decompress to return -1 rather
than throw EOF at end of file. (Zhijie Shen via acmurthy)
HADOOP-8440. HarFileSystem.decodeHarURI fails for URIs whose host contains
numbers. (Ivan Mitic via cnauroth)
HADOOP-9643. org.apache.hadoop.security.SecurityUtil calls
toUpperCase(Locale.getDefault()) as well as toLowerCase(Locale.getDefault())
on hadoop.security.authentication value. (markrmiller@gmail.com via tucu)
HADOOP-9701. mvn site ambiguous links in hadoop-common. (kkambatl via tucu)
Release 2.0.6-alpha - 08/22/2013
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
OPTIMIZATIONS
BUG FIXES
Release 2.0.5-alpha - 06/06/2013
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
OPTIMIZATIONS
BUG FIXES
HADOOP-9407. commons-daemon 1.0.3 dependency has bad group id causing
build issues. (Sangjin Lee via suresh)
Release 2.0.4-alpha - 2013-04-25
INCOMPATIBLE CHANGES
NEW FEATURES
IMPROVEMENTS
OPTIMIZATIONS
BUG FIXES
HADOOP-9467. Metrics2 record filter should check name as well as tags.
(Chris Nauroth and Ganeshan Iyler via llu)
HADOOP-9406. hadoop-client leaks dependency on JDK tools jar. (tucu)
HADOOP-9301. hadoop client servlet/jsp/jetty/tomcat JARs creating
conflicts in Oozie & HttpFS. (tucu)
HADOOP-9299. kerberos name resolution is kicking in even when kerberos
is not configured (daryn)
HADOOP-9408. misleading description for net.topology.table.file.name
property in core-default.xml. (rajeshbabu via suresh)
HADOOP-9444. Modify hadoop-policy.xml to replace unexpanded variables to a
default value of '*'. (Roman Shaposhnik via vinodkv)
HADOOP-9471. hadoop-client wrongfully excludes jetty-util JAR,
breaking webhdfs. (tucu)
Release 2.0.3-alpha - 2013-02-06
INCOMPATIBLE CHANGES
HADOOP-8999. SASL negotiation is flawed (daryn)
NEW FEATURES
HADOOP-8561. Introduce HADOOP_PROXY_USER for secure impersonation in child
hadoop client processes. (Yu Gao via llu)
HADOOP-8597. Permit FsShell's text command to read Avro files.
(Ivan Vladimirov Ivanov via cutting)
HADOOP-9020. Add a SASL PLAIN server (daryn via bobby)
HADOOP-9090. Support on-demand publish of metrics. (Mostafa Elhemali via
suresh)
HADOOP-9054. Add AuthenticationHandler that uses Kerberos but allows for
an alternate form of authentication for browsers. (rkanter via tucu)
IMPROVEMENTS
HADOOP-8789. Tests setLevel(Level.OFF) should be Level.ERROR.
(Andy Isaacson via eli)
HADOOP-8755. Print thread dump when tests fail due to timeout. (Andrey
Klochkov via atm)
HADOOP-8806. libhadoop.so: dlopen should be better at locating
libsnappy.so, etc. (Colin Patrick McCabe via eli)
HADOOP-8812. ExitUtil#terminate should print Exception#toString. (eli)
HADOOP-8736. Add Builder for building RPC server. (Brandon Li via Suresh)
HDFS-3957. Change MutableQuantiles to use a shared thread for rolling
over metrics. (Andrew Wang via todd)
HADOOP-8851. Use -XX:+HeapDumpOnOutOfMemoryError JVM option in the forked
tests. (Ivan A. Veselovsky via atm)
HADOOP-8783. Improve RPC.Server's digest auth (daryn)
HADOOP-8889. Upgrade to Surefire 2.12.3 (todd)
HADOOP-8804. Improve Web UIs when the wildcard address is used.
(Senthil Kumar via eli)
HADOOP-8894. GenericTestUtils.waitFor should dump thread stacks on timeout
(todd)
HADOOP-8909. Hadoop Common Maven protoc calls must not depend on external
sh script. (Chris Nauroth via suresh)
HADOOP-8911. CRLF characters in source and text files.
(Raja Aluri via suresh)
HADOOP-8912. Add .gitattributes file to prevent CRLF and LF mismatches
for source and text files. (Raja Aluri via suresh)
HADOOP-8784. Improve IPC.Client's token use (daryn)
HADOOP-8929. Add toString, other improvements for SampleQuantiles (todd)
HADOOP-8922. Provide alternate JSONP output for JMXJsonServlet to allow
javascript in browser (Damien Hardy via bobby)
HADOOP-8931. Add Java version to startup message. (eli)
HADOOP-8925. Remove the packaging. (eli)
HADOOP-8985. Add namespace declarations in .proto files for languages
other than java. (Binglin Chan via suresh)
HADOOP-9009. Add SecurityUtil methods to get/set authentication method
(daryn via bobby)
HADOOP-9010. Map UGI authenticationMethod to RPC authMethod (daryn via
bobby)
HADOOP-9013. UGI should not hardcode loginUser's authenticationType (daryn
via bobby)
HADOOP-9014. Standardize creation of SaslRpcClients (daryn via bobby)
HADOOP-9015. Standardize creation of SaslRpcServers (daryn via bobby)
HADOOP-8860. Split MapReduce and YARN sections in documentation navigation.
(tomwhite via tucu)
HADOOP-9021. Enforce configured SASL method on the server (daryn via
bobby)
HADOO-8998. set Cache-Control no-cache header on all dynamic content. (tucu)
HADOOP-9035. Generalize setup of LoginContext (daryn via bobby)
HADOOP-9093. Move all the Exception in PathExceptions to o.a.h.fs package.
(suresh)
HADOOP-9042. Add a test for umask in FileSystemContractBaseTest.
(Colin McCabe via eli)
HADOOP-9127. Update documentation for ZooKeeper Failover Controller.
(Daisuke Kobayashi via atm)
HADOOP-9004. Allow security unit tests to use external KDC. (Stephen Chu
via suresh)
HADOOP-9147. Add missing fields to FIleStatus.toString.
(Jonathan Allen via suresh)
HADOOP-8427. Convert Forrest docs to APT, incremental. (adi2 via tucu)
HADOOP-9162. Add utility to check native library availability.
(Binglin Chang via suresh)
HADOOP-9173. Add security token protobuf definition to common and
use it in hdfs. (suresh)
HADOOP-9119. Add test to FileSystemContractBaseTest to verify integrity
of overwritten files. (Steve Loughran via suresh)
HADOOP-9192. Move token related request/response messages to common.
(suresh)
HADOOP-8712. Change default hadoop.security.group.mapping to
JniBasedUnixGroupsNetgroupMappingWithFallback (Robert Parker via todd)
HADOOP-9106. Allow configuration of IPC connect timeout.
(Rober Parker via suresh)
HADOOP-9216. CompressionCodecFactory#getCodecClasses should trim the
result of parsing by Configuration. (Tsuyoshi Ozawa via todd)
HADOOP-9231. Parametrize staging URL for the uniformity of
distributionManagement. (Konstantin Boudnik via suresh)
HADOOP-9276. Allow BoundedByteArrayOutputStream to be resettable.
(Arun Murthy via hitesh)
HADOOP-7688. Add servlet handler check in HttpServer.start().
(Uma Maheswara Rao G via szetszwo)
HADOOP-7886. Add toString to FileStatus. (SreeHari via jghoman)
OPTIMIZATIONS
HADOOP-8866. SampleQuantiles#query is O(N^2) instead of O(N). (Andrew Wang
via atm)
HADOOP-8926. hadoop.util.PureJavaCrc32 cache hit-ratio is low for static
data (Gopal V via bobby)
BUG FIXES
HADOOP-9041. FsUrlStreamHandlerFactory could cause an infinite loop in
FileSystem initialization. (Yanbo Liang and Radim Kolar via llu)
HADOOP-8418. Update UGI Principal classes name for running with
IBM JDK on 64 bits Windows. (Yu Gao via eyang)
HADOOP-8795. BASH tab completion doesn't look in PATH, assumes path to
executable is specified. (Sean Mackrory via atm)
HADOOP-8780. Update DeprecatedProperties apt file. (Ahmed Radwan via
tomwhite)
HADOOP-8833. fs -text should make sure to call inputstream.seek(0)
before using input stream. (tomwhite and harsh)
HADOOP-8791. Fix rm command documentation to indicte it deletes
files and not directories. (Jing Zhao via suresh)
HADOOP-8855. SSL-based image transfer does not work when Kerberos
is disabled. (todd via eli)
HADOOP-8616. ViewFS configuration requires a trailing slash. (Sandy Ryza
via atm)
HADOOP-8756. Fix SEGV when libsnappy is in java.library.path but
not LD_LIBRARY_PATH. (Colin Patrick McCabe via eli)
HADOOP-8881. FileBasedKeyStoresFactory initialization logging should
be debug not info. (tucu)
HADOOP-8913. hadoop-metrics2.properties should give units in comment
for sampling period. (Sandy Ryza via suresh)
HADOOP-8878. Uppercase namenode hostname causes hadoop dfs calls with
webhdfs filesystem and fsck to fail when security is on.
(Arpit Gupta via suresh)
HADOOP-8901. GZip and Snappy support may not work without unversioned
libraries (Colin Patrick McCabe via todd)
HADOOP-8883. Anonymous fallback in KerberosAuthenticator is broken.
(rkanter via tucu)
HADOOP-8900. BuiltInGzipDecompressor throws IOException - stored gzip size
doesn't match decompressed size. (Andy Isaacson via suresh)
HADOOP-8948. TestFileUtil.testGetDU fails on Windows due to incorrect
assumption of line separator. (Chris Nauroth via suresh)
HADOOP-8951. RunJar to fail with user-comprehensible error
message if jar missing. (stevel via suresh)
HADOOP-8713. TestRPCCompatibility fails intermittently with JDK7
(Trevor Robinson via tgraves)
HADOOP-9012. IPC Client sends wrong connection context (daryn via bobby)
HADOOP-7115. Add a cache for getpwuid_r and getpwgid_r calls (tucu)
HADOOP-6607. Add different variants of non caching HTTP headers. (tucu)
HADOOP-9049. DelegationTokenRenewer needs to be Singleton and FileSystems
should register/deregister to/from. (Karthik Kambatla via tomwhite)
HADOOP-9064. Augment DelegationTokenRenewer API to cancel the tokens on
calls to removeRenewAction. (kkambatl via tucu)
HADOOP-9103. UTF8 class does not properly decode Unicode characters
outside the basic multilingual plane. (todd)
HADOOP-9070. Kerberos SASL server cannot find kerberos key. (daryn via atm)
HADOOP-6762. Exception while doing RPC I/O closes channel
(Sam Rash and todd via todd)
HADOOP-9126. FormatZK and ZKFC startup can fail due to zkclient connection
establishment delay. (Rakesh R and todd via todd)
HADOOP-9113. o.a.h.fs.TestDelegationTokenRenewer is failing intermittently.
(Karthik Kambatla via eli)
HADOOP-9135. JniBasedUnixGroupsMappingWithFallback should log at debug
rather than info during fallback. (Colin Patrick McCabe via todd)
HADOOP-9152. HDFS can report negative DFS Used on clusters with very small
amounts of data. (Brock Noland via atm)
HADOOP-9153. Support createNonRecursive in ViewFileSystem.
(Sandy Ryza via tomwhite)
HADOOP-9181. Set daemon flag for HttpServer's QueuedThreadPool.
(Liang Xie via suresh)
HADOOP-9155. FsPermission should have different default value, 777 for
directory and 666 for file. (Binglin Chang via atm)
HADOOP-9183. Potential deadlock in ActiveStandbyElector. (tomwhite)
HADOOP-9203. RPCCallBenchmark should find a random available port.
(Andrew Purtell via suresh)
HADOOP-9178. src/main/conf is missing hadoop-policy.xml.
(Sandy Ryza via eli)
HADOOP-8816. HTTP Error 413 full HEAD if using kerberos authentication.
(moritzmoeller via tucu)
HADOOP-9212. Potential deadlock in FileSystem.Cache/IPC/UGI. (tomwhite)
HADOOP-8589 ViewFs tests fail when tests and home dirs are nested.
(sanjay Radia)
HADOOP-9193. hadoop script can inadvertently expand wildcard arguments
when delegating to hdfs script. (Andy Isaacson via todd)
HADOOP-9215. when using cmake-2.6, libhadoop.so doesn't get created
(only libhadoop.so.1.0.0) (Colin Patrick McCabe via todd)
HADOOP-8857. hadoop.http.authentication.signature.secret.file docs
should not state that secret is randomly generated. (tucu)
HADOOP-9190. packaging docs is broken. (Andy Isaacson via tgraves)
HADOOP-9221. Convert remaining xdocs to APT. (Andy Isaacson via atm)
HADOOP-8981. TestMetricsSystemImpl fails on Windows. (Xuan Gong via suresh)
HADOOP-9124. SortedMapWritable violates contract of Map interface for
equals() and hashCode(). (Surenkumar Nihalani via tomwhite)
HADOOP-9278. Fix the file handle leak in HarMetaData.parseMetaData() in
HarFileSystem. (Chris Nauroth via szetszwo)
HADOOP-9252. In StringUtils, humanReadableInt(..) has a race condition and
the synchronization of limitDecimalTo2(double) can be avoided. (szetszwo)
HADOOP-9260. Hadoop version may be not correct when starting name node or
data node. (Chris Nauroth via jlowe)
HADOOP-9289. FsShell rm -f fails for non-matching globs. (Daryn Sharp via
suresh)
Release 2.0.2-alpha - 2012-09-07
INCOMPATIBLE CHANGES
HADOOP-8388. Remove unused BlockLocation serialization.
(Colin Patrick McCabe via eli)
HADOOP-8689. Make trash a server side configuration option. (eli)
HADOOP-8710. Remove ability for users to easily run the trash emptire. (eli)
HADOOP-8794. Rename YARN_HOME to HADOOP_YARN_HOME. (vinodkv via acmurthy)
NEW FEATURES
HDFS-3042. Automatic failover support for NameNode HA (todd)
(see dedicated section below for breakdown of subtasks)
HADOOP-8135. Add ByteBufferReadable interface to FSDataInputStream. (Henry
Robinson via atm)
HADOOP-8458. Add management hook to AuthenticationHandler to enable
delegation token operations support (tucu)
HADOOP-8465. hadoop-auth should support ephemeral authentication (tucu)
HADOOP-8644. AuthenticatedURL should be able to use SSLFactory. (tucu)
HADOOP-8581. add support for HTTPS to the web UIs. (tucu)
HADOOP-7754. Expose file descriptors from Hadoop-wrapped local
FileSystems (todd and ahmed via tucu)
HADOOP-8240. Add a new API to allow users to specify a checksum type
on FileSystem.create(..). (Kihwal Lee via szetszwo)
IMPROVEMENTS
HADOOP-8340. SNAPSHOT build versions should compare as less than their eventual
final release. (todd)
HADOOP-8361. Avoid out-of-memory problems when deserializing strings.
(Colin Patrick McCabe via eli)
HADOOP-8224. Don't hardcode hdfs.audit.logger in the scripts.
(Tomohiko Kinebuchi via eli)
HADOOP-8398. Cleanup BlockLocation. (eli)
HADOOP-8422. Deprecate FileSystem#getDefault* and getServerDefault
methods that don't take a Path argument. (eli)
HADOOP-8323. Add javadoc and tests for Text.clear() behavior (harsh)
HADOOP-8358. Config-related WARN for dfs.web.ugi can be avoided. (harsh)
HADOOP-8450. Remove src/test/system. (eli)
HADOOP-8244. Improve comments on ByteBufferReadable.read. (Henry Robinson
via atm)
HADOOP-8368. Use CMake rather than autotools to build native code (ccccabe via tucu)
HADOOP-8524. Allow users to get source of a Configuration
parameter (harsh)
HADOOP-8449. hadoop fs -text fails with compressed sequence files
with the codec file extension (harsh)
HADOOP-6802. Remove FS_CLIENT_BUFFER_DIR_KEY = "fs.client.buffer.dir"
from CommonConfigurationKeys.java (not used, deprecated)
(Sho Shimauchi via harsh)
HADOOP-3450. Add tests to Local Directory Allocator for
asserting their URI-returning capability (Sho Shimauchi via harsh)
HADOOP-8463. hadoop.security.auth_to_local needs a key definition and doc.
(Madhukara Phatak via eli)
HADOOP-8533. Remove parallel call ununsed capability in RPC.
(Brandon Li via suresh)
HADOOP-8423. MapFile.Reader.get() crashes jvm or throws
EOFException on Snappy or LZO block-compressed data
(todd via harsh)
HADOOP-8541. Better high-percentile latency metrics. (Andrew Wang via atm)
HADOOP-8362. Improve exception message when Configuration.set() is
called with a null key or value. (Madhukara Phatak
and Suresh Srinivas via harsh)
HADOOP-7818. DiskChecker#checkDir should fail if the directory is
not executable. (Madhukara Phatak via harsh)
HADOOP-8531. SequenceFile Writer can throw out a better error if a
serializer or deserializer isn't available
(Madhukara Phatak via harsh)
HADOOP-8609. IPC server logs a useless message when shutting down socket.
(Jon Zuanich via atm)
HADOOP-8620. Add -Drequire.fuse and -Drequire.snappy. (Colin
Patrick McCabe via eli)
HADOOP-8687. Upgrade log4j to 1.2.17. (eli)
HADOOP-8278. Make sure components declare correct set of dependencies.
(tomwhite)
HADOOP-8700. Use enum to define the checksum constants in DataChecksum.
(szetszwo)
HADOOP-8686. Fix warnings in native code. (Colin Patrick McCabe via eli)
HADOOP-8239. Add subclasses of MD5MD5CRC32FileChecksum to support file
checksum with CRC32C. (Kihwal Lee via szetszwo)
HADOOP-8619. WritableComparator must implement no-arg constructor.
(Chris Douglas via Suresh)
HADOOP-8075. Lower native-hadoop library log from info to debug.
(Hızır Sefa İrken via eli)
HADOOP-8748. Refactor DFSClient retry utility methods to a new class
in org.apache.hadoop.io.retry. (Arun C Murthy via szetszwo)
HADOOP-8754. Deprecate all the RPC.getServer() variants. (Brandon Li
via szetszwo)
HADOOP-8801. ExitUtil#terminate should capture the exception stack trace. (eli)
HADOOP-8819. Incorrectly & is used instead of && in some file system
implementations. (Brandon Li via suresh)
HADOOP-7808. Port HADOOP-7510 - Add configurable option to use original
hostname in token instead of IP to allow server IP change.
(Daryn Sharp via suresh)
HADOOP-8367 Improve documentation of declaringClassProtocolName in
rpc headers. (Sanjay Radia)
HADOOP-8624. ProtobufRpcEngine should log all RPCs if TRACE logging is
enabled (todd)
BUG FIXES
HADOOP-8372. NetUtils.normalizeHostName() incorrectly handles hostname
starting with a numeric character. (Junping Du via suresh)
HADOOP-8393. hadoop-config.sh missing variable exports, causes Yarn jobs
to fail with ClassNotFoundException MRAppMaster. (phunt via tucu)
HADOOP-8316. Audit logging should be disabled by default. (eli)
HADOOP-8400. All commands warn "Kerberos krb5 configuration not found"
when security is not enabled. (tucu)
HADOOP-8406. CompressionCodecFactory.CODEC_PROVIDERS iteration is
thread-unsafe (todd)
HADOOP-8287. etc/hadoop is missing hadoop-env.sh (eli)
HADOOP-8408. MR doesn't work with a non-default ViewFS mount table
and security enabled. (atm via eli)
HADOOP-8329. Build fails with Java 7. (eli)
HADOOP-8268. A few pom.xml across Hadoop project
may fail XML validation. (Radim Kolar via harsh)
HADOOP-8444. Fix the tests FSMainOperationsBaseTest.java and
FileContextMainOperationsBaseTest.java to avoid potential
test failure (Madhukara Phatak via harsh)
HADOOP-8452. DN logs backtrace when running under jsvc and /jmx is loaded
(Andy Isaacson via bobby)
HADOOP-8460. Document proper setting of HADOOP_PID_DIR and
HADOOP_SECURE_DN_PID_DIR (bobby)
HADOOP-8466. hadoop-client POM incorrectly excludes avro. (bmahe via tucu)
HADOOP-8481. update BUILDING.txt to talk about cmake rather than autotools.
(Colin Patrick McCabe via eli)
HADOOP-8485. Don't hardcode "Apache Hadoop 0.23" in the docs. (eli)
HADOOP-8488. test-patch.sh gives +1 even if the native build fails.
(Colin Patrick McCabe via eli)
HADOOP-8507. Avoid OOM while deserializing DelegationTokenIdentifer.
(Colin Patrick McCabe via eli)
HADOOP-8433. Don't set HADOOP_LOG_DIR in hadoop-env.sh.
(Brahma Reddy Battula via eli)
HADOOP-8509. JarFinder duplicate entry: META-INF/MANIFEST.MF exception (tucu)
HADOOP-8512. AuthenticatedURL should reset the Token when the server returns
other than OK on authentication (tucu)
HADOOP-8168. empty-string owners or groups causes {{MissingFormatWidthException}}
in o.a.h.fs.shell.Ls.ProcessPath() (ekoontz via tucu)
HADOOP-8438. hadoop-validate-setup.sh refers to examples jar file which doesn't exist
(Devaraj K via umamahesh)
HADOOP-8538. CMake builds fail on ARM. (Trevor Robinson via eli)
HADOOP-8547. Package hadoop-pipes examples/bin directory (again).
(Colin Patrick McCabe via eli)
HADOOP-8563. don't package hadoop-pipes examples/bin
(Colin Patrick McCabe via tgraves)
HADOOP-8566. AvroReflectSerializer.accept(Class) throws a NPE if the class has no
package (primitive types and arrays). (tucu)
HADOOP-8586. Fixup a bunch of SPNEGO misspellings. (eli)
HADOOP-3886. Error in javadoc of Reporter, Mapper and Progressable
(Jingguo Yao via harsh)
HADOOP-8587. HarFileSystem access of harMetaCache isn't threadsafe. (eli)
HADOOP-8585. Fix initialization circularity between UserGroupInformation
and HadoopConfiguration. (Colin Patrick McCabe via atm)
HADOOP-8552. Conflict: Same security.log.file for multiple users.
(kkambatl via tucu)
HADOOP-8537. Fix TFile tests to pass even when native zlib support is not
compiled. (todd)
HADOOP-8626. Typo in default setting for
hadoop.security.group.mapping.ldap.search.filter.user. (Jonathan Natkins
via atm)
HADOOP-8480. The native build should honor -DskipTests.
(Colin Patrick McCabe via eli)
HADOOP-8659. Native libraries must build with soft-float ABI for Oracle JVM
on ARM. (Trevor Robinson via todd)
HADOOP-8654. TextInputFormat delimiter bug (Gelesh and Jason Lowe via
bobby)
HADOOP-8614. IOUtils#skipFully hangs forever on EOF.
(Colin Patrick McCabe via eli)
HADOOP-8720. TestLocalFileSystem should use test root subdirectory.
(Vlad Rozov via eli)
HADOOP-8721. ZKFC should not retry 45 times when attempting a graceful
fence during a failover. (Vinayakumar B via atm)
HADOOP-8632. Configuration leaking class-loaders (Costin Leau via bobby)
HADOOP-4572. Can not access user logs - Jetty is not configured by default
to serve aliases/symlinks (ahmed via tucu)
HADOOP-8660. TestPseudoAuthenticator failing with NPE. (tucu)
HADOOP-8699. some common testcases create core-site.xml in test-classes
making other testcases to fail. (tucu)
HADOOP-8031. Configuration class fails to find embedded .jar resources;
should use URL.openStream() (genman via tucu)
HADOOP-8737. cmake: always use JAVA_HOME to find libjvm.so, jni.h, jni_md.h.
(Colin Patrick McCabe via eli)
HADOOP-8747. Syntax error on cmake version 2.6 patch 2 in JNIFlags.cmake. (cmccabe via tucu)
HADOOP-8722. Update BUILDING.txt with latest snappy info.
(Colin Patrick McCabe via eli)
HADOOP-8764. CMake: HADOOP-8737 broke ARM build. (Trevor Robinson via eli)
HADOOP-8770. NN should not RPC to self to find trash defaults. (eli)
HADOOP-8648. libhadoop: native CRC32 validation crashes when
io.bytes.per.checksum=1. (Colin Patrick McCabe via eli)
HADOOP-8766. FileContextMainOperationsBaseTest should randomize the root
dir. (Colin Patrick McCabe via atm)
HADOOP-8749. HADOOP-8031 changed the way in which relative xincludes are handled in
Configuration. (ahmed via tucu)
HADOOP-8431. Running distcp wo args throws IllegalArgumentException.
(Sandy Ryza via eli)
HADOOP-8775. MR2 distcp permits non-positive value to -bandwidth option
which causes job never to complete. (Sandy Ryza via atm)
HADOOP-8781. hadoop-config.sh should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH. (tucu)
BREAKDOWN OF HDFS-3042 SUBTASKS
HADOOP-8220. ZKFailoverController doesn't handle failure to become active
correctly (todd)
HADOOP-8228. Auto HA: Refactor tests and add stress tests. (todd)
HADOOP-8215. Security support for ZK Failover controller (todd)
HADOOP-8245. Fix flakiness in TestZKFailoverController (todd)
HADOOP-8257. TestZKFailoverControllerStress occasionally fails with Mockito
error (todd)
HADOOP-8260. Replace ClientBaseWithFixes with our own modified copy of the
class (todd)
HADOOP-8246. Auto-HA: automatically scope znode by nameservice ID (todd)
HADOOP-8247. Add a config to enable auto-HA, which disables manual
FailoverController (todd)
HADOOP-8306. ZKFC: improve error message when ZK is not running. (todd)
HADOOP-8279. Allow manual failover to be invoked when auto-failover is
enabled. (todd)
HADOOP-8276. Auto-HA: add config for java options to pass to zkfc daemon
(todd via eli)
HADOOP-8405. ZKFC tests leak ZK instances. (todd)
Release 2.0.0-alpha - 05-23-2012
INCOMPATIBLE CHANGES
HADOOP-7920. Remove Avro Rpc. (suresh)
NEW FEATURES
HADOOP-7773. Add support for protocol buffer based RPC engine.
(suresh)
HADOOP-7875. Add helper class to unwrap protobuf ServiceException.
(suresh)
HADOOP-7454. Common side of High Availability Framework (HDFS-1623)
Contributed by Todd Lipcon, Aaron T. Myers, Eli Collins, Uma Maheswara Rao G,
Bikas Saha, Suresh Srinivas, Jitendra Nath Pandey, Hari Mankude, Brandon Li,
Sanjay Radia, Mingjie Lai, and Gregory Chanan
HADOOP-8121. Active Directory Group Mapping Service. (Jonathan Natkins via
atm)
HADOOP-7030. Add TableMapping topology implementation to read host to rack
mapping from a file. (Patrick Angeles and tomwhite via tomwhite)
HADOOP-8206. Common portion of a ZK-based failover controller (todd)
HADOOP-8210. Common side of HDFS-3148: The client should be able
to use multiple local interfaces for data transfer. (eli)
HADOOP-8343. Allow configuration of authorization for JmxJsonServlet and
MetricsServlet (tucu)
IMPROVEMENTS
HADOOP-7524. Change RPC to allow multiple protocols including multuple
versions of the same protocol (sanjay Radia)
HADOOP-7607. Simplify the RPC proxy cleanup process. (atm)
HADOOP-7687. Make getProtocolSignature public (sanjay)
HADOOP-7693. Enhance AvroRpcEngine to support the new #addProtocol
interface introduced in HADOOP-7524. (cutting)
HADOOP-7716. RPC protocol registration on SS does not log the protocol name
(only the class which may be different) (sanjay)
HADOOP-7776. Make the Ipc-Header in a RPC-Payload an explicit header.
(sanjay)
HADOOP-7862. Move the support for multiple protocols to lower layer so
that Writable, PB and Avro can all use it (Sanjay)
HADOOP-7876. Provided access to encoded key in DelegationKey for
use in protobuf based RPCs. (suresh)
HADOOP-7899. Generate proto java files as part of the build. (tucu)
HADOOP-7957. Classes deriving GetGroupsBase should be able to override
proxy creation. (jitendra)
HADOOP-7965. Support for protocol version and signature in PB. (jitendra)
HADOOP-8070. Add a standalone benchmark for RPC call performance. (todd)
HADOOP-8084. Updates ProtoBufRpc engine to not do an unnecessary copy
for RPC request/response. (ddas)
HADOOP-8085. Add RPC metrics to ProtobufRpcEngine. (Hari Mankude via
suresh)
HADOOP-8098. KerberosAuthenticatorHandler should use _HOST replacement to
resolve principal name (tucu)
HADOOP-8118. In metrics2.util.MBeans, change log level to trace for the
stack trace of InstanceAlreadyExistsException. (szetszwo)
HADOOP-8125. make hadoop-client set of curated jars available in a
distribution tarball (rvs via tucu)
HADOOP-7717. Move handling of concurrent client fail-overs to
RetryInvocationHandler (atm)
HADOOP-7728. Enable task memory management to be configurable in hadoop
config setup script. (ramya)
HADOOP-7358. Improve log levels when exceptions caught in RPC handler
(Todd Lipcon via shv)
HADOOP-7557 Make IPC header be extensible (sanjay radia)
HADOOP-7806. Support binding to sub-interfaces (eli)
HADOOP-6941. Adds support for building Hadoop with IBM's JDK
(Stephen Watt, Eli and ddas)
HADOOP-8183. Stop using "mapred.used.genericoptions.parser" (harsh)
HADOOP-6924. Adds a directory to the list of directories to search
for the libjvm.so file. The new directory is found by running a 'find'
command and the first output is taken. This was done to handle the
build of Hadoop with IBM's JDK. (Stephen Watt, Guillermo Cabrera and ddas)
HADOOP-8200. Remove HADOOP_[JOBTRACKER|TASKTRACKER]_OPTS. (eli)
HADOOP-8184. ProtoBuf RPC engine uses the IPC layer reply packet.
(Sanjay Radia via szetszwo)
HADOOP-8163. Improve ActiveStandbyElector to provide hooks for
fencing old active. (todd)
HADOOP-8193. Refactor FailoverController/HAAdmin code to add an abstract
class for "target" services. (todd)
HADOOP-8212. Improve ActiveStandbyElector's behavior when session expires
(todd)
HADOOP-8216. Address log4j.properties inconsistencies btw main and
template dirs. (Patrick Hunt via eli)
HADOOP-8149. Cap space usage of default log4j rolling policy.
(Patrick Hunt via eli)
HADOOP-8211. Update commons-net version to 3.1. (eli)
HADOOP-8236. haadmin should have configurable timeouts for failover
commands. (todd)
HADOOP-8242. AbstractDelegationTokenIdentifier: add getter methods
for owner and realuser. (Colin Patrick McCabe via eli)
HADOOP-8007. Use substitution tokens for fencing argument (todd)
HADOOP-8077. HA: fencing method should be able to be configured on
a per-NN or per-NS basis (todd)
HADOOP-8086. KerberosName silently sets defaultRealm to "" if the
Kerberos config is not found, it should log a WARN (tucu)
HADOOP-8280. Move VersionUtil/TestVersionUtil and GenericTestUtils from
HDFS into Common. (Ahmed Radwan via atm)
HADOOP-8117. Upgrade test build to Surefire 2.12 (todd)
HADOOP-8152. Expand public APIs for security library classes. (atm via eli)
HADOOP-7549. Use JDK ServiceLoader mechanism to find FileSystem implementations. (tucu)
HADOOP-8185. Update namenode -format documentation and add -nonInteractive
and -force. (Arpit Gupta via atm)
HADOOP-8214. make hadoop script recognize a full set of deprecated commands (rvs via tucu)
HADOOP-8347. Hadoop Common logs misspell 'successful'.
(Philip Zeyliger via eli)
HADOOP-8350. Improve NetUtils.getInputStream to return a stream which has
a tunable timeout. (todd)
HADOOP-8356. FileSystem service loading mechanism should print the FileSystem
impl it is failing to load (tucu)
HADOOP-8353. hadoop-daemon.sh and yarn-daemon.sh can be misleading on stop.
(Roman Shaposhnik via atm)
HADOOP-8113. Correction to BUILDING.txt: HDFS needs ProtocolBuffer, too
(not just MapReduce). Contributed by Eugene Koontz.
HADOOP-8285 Use ProtoBuf for RpcPayLoadHeader (sanjay radia)
HADOOP-8366 Use ProtoBuf for RpcResponseHeader (sanjay radia)
HADOOP-7729. Send back valid HTTP response if user hits IPC port with
HTTP GET. (todd)
HADOOP-7987. Support setting the run-as user in unsecure mode. (jitendra)
HADOOP-7994. Remove getProtocolVersion and getProtocolSignature from the
client side translator and server side implementation. (jitendra)
OPTIMIZATIONS
BUG FIXES
HADOOP-8199. Fix issues in start-all.sh and stop-all.sh (Devaraj K via umamahesh)
HADOOP-7635. RetryInvocationHandler should release underlying resources on
close. (atm)
HADOOP-7695. RPC.stopProxy can throw unintended exception while logging
error. (atm)
HADOOP-7833. Fix findbugs warnings in protobuf generated code.
(John Lee via suresh)
HADOOP-7897. ProtobufRpcEngine client side exception mechanism is not
consistent with WritableRpcEngine. (suresh)
HADOOP-7913. Fix bug in ProtoBufRpcEngine. (sanjay)
HADOOP-7892. IPC logs too verbose after "RpcKind" introduction. (todd)
HADOOP-7968. Errant println left in RPC.getHighestSupportedProtocol. (Sho
Shimauchi via harsh)
HADOOP-7931. o.a.h.ipc.WritableRpcEngine should have a way to force
initialization. (atm)
HADOOP-8104. Inconsistent Jackson versions (tucu)
HADOOP-8119. Fix javac warnings in TestAuthenticationFilter in hadoop-auth.
(szetszwo)
HADOOP-7888. TestFailoverProxy fails intermittently on trunk. (Jason Lowe
via atm)
HADOOP-8154. DNS#getIPs shouldn't silently return the local host
IP for bogus interface names. (eli)
HADOOP-8169. javadoc generation fails with java.lang.OutOfMemoryError:
Java heap space (tgraves via bobby)
HADOOP-8167. Configuration deprecation logic breaks backwards compatibility (tucu)
HADOOP-8189. LdapGroupsMapping shouldn't throw away IOException. (Jonathan Natkins via atm)
HADOOP-8191. SshFenceByTcpPort uses netcat incorrectly (todd)
HADOOP-8157. Fix race condition in Configuration that could cause spurious
ClassNotFoundExceptions after a GC. (todd)
HADOOP-8197. Configuration logs WARNs on every use of a deprecated key (tucu)
HADOOP-8159. NetworkTopology: getLeaf should check for invalid topologies.
(Colin Patrick McCabe via eli)
HADOOP-8204. TestHealthMonitor fails occasionally (todd)
HADOOP-8202. RPC stopProxy() does not close the proxy correctly.
(Hari Mankude via suresh)
HADOOP-8218. RPC.closeProxy shouldn't throw error when closing a mock
(todd)
HADOOP-8238. NetUtils#getHostNameOfIP blows up if given ip:port
string w/o port. (eli)
HADOOP-8243. Security support broken in CLI (manual) failover controller
(todd)
HADOOP-8251. Fix SecurityUtil.fetchServiceTicket after HADOOP-6941 (todd)
HADOOP-8249. invalid hadoop-auth cookies should trigger authentication
if info is avail before returning HTTP 401 (tucu)
HADOOP-8261. Har file system doesn't deal with FS URIs with a host but no
port. (atm)
HADOOP-8263. Stringification of IPC calls not useful (todd)
HADOOP-8264. Remove irritating double double quotes in front of hostname
(Bernd Fondermann via bobby)
HADOOP-8270. hadoop-daemon.sh stop action should return 0 for an
already stopped service. (Roman Shaposhnik via eli)
HADOOP-8144. pseudoSortByDistance in NetworkTopology doesn't work
properly if no local node and first node is local rack node.
(Junping Du)
HADOOP-8282. start-all.sh refers incorrectly start-dfs.sh
existence for starting start-yarn.sh. (Devaraj K via eli)
HADOOP-7350. Use ServiceLoader to discover compression codec classes.
(tomwhite)
HADOOP-8284. clover integration broken, also mapreduce poms are pulling
in clover as a dependency. (phunt via tucu)
HADOOP-8309. Pseudo & Kerberos AuthenticationHandler should use
getType() to create token (tucu)
HADOOP-8314. HttpServer#hasAdminAccess should return false if
authorization is enabled but user is not authenticated. (tucu)
HADOOP-8296. hadoop/yarn daemonlog usage wrong (Devaraj K via tgraves)
HADOOP-8310. FileContext#checkPath should handle URIs with no port. (atm)
HADOOP-8321. TestUrlStreamHandler fails. (tucu)
HADOOP-8325. Add a ShutdownHookManager to be used by different
components instead of the JVM shutdownhook (tucu)
HADOOP-8275. Range check DelegationKey length.
(Colin Patrick McCabe via eli)
HADOOP-8342. HDFS command fails with exception following merge of
HADOOP-8325 (tucu)
HADOOP-8346. Makes oid changes to make SPNEGO work. Was broken due
to fixes introduced by the IBM JDK compatibility patch. (ddas)
HADOOP-8355. SPNEGO filter throws/logs exception when authentication fails (tucu)
HADOOP-8349. ViewFS doesn't work when the root of a file system is mounted. (atm)
HADOOP-8328. Duplicate FileSystem Statistics object for 'file' scheme.
(tomwhite)
HADOOP-8359. Fix javadoc warnings in Configuration. (Anupam Seth via
szetszwo)
HADOOP-7988. Upper case in hostname part of the principals doesn't work with
kerberos. (jitendra)
BREAKDOWN OF HADOOP-7454 SUBTASKS
HADOOP-7455. HA: Introduce HA Service Protocol Interface. (suresh)
HADOOP-7774. HA: Administrative CLI to control HA daemons. (todd)
HADOOP-7896. HA: if both NNs are in Standby mode, client needs to try failing
back and forth several times with sleeps. (atm)
HADOOP-7922. Improve some logging for client IPC failovers and
StandbyExceptions (todd)
HADOOP-7921. StandbyException should extend IOException (todd)
HADOOP-7928. HA: Client failover policy is incorrectly trying to fail over all
IOExceptions (atm)
HADOOP-7925. Add interface and update CLI to query current state to
HAServiceProtocol (eli via todd)
HADOOP-7932. Make client connection retries on socket time outs configurable.
(Uma Maheswara Rao G via todd)
HADOOP-7924. FailoverController for client-based configuration (eli)
HADOOP-7961. Move HA fencing to common. (eli)
HADOOP-7970. HAServiceProtocol methods must throw IOException. (Hari Mankude
via suresh).
HADOOP-7992. Add ZKClient library to facilitate leader election. (Bikas Saha
via suresh).
HADOOP-7983. HA: failover should be able to pass args to fencers. (eli)
HADOOP-7938. HA: the FailoverController should optionally fence the active
during failover. (eli)
HADOOP-7991. HA: the FailoverController should check the standby is ready
before failing over. (eli)
HADOOP-8038. Add 'ipc.client.connect.max.retries.on.timeouts' entry in
core-default.xml file. (Uma Maheswara Rao G via atm)
HADOOP-8041. Log a warning when a failover is first attempted (todd)
HADOOP-8068. void methods can swallow exceptions when going through failover
path (todd)
HADOOP-8116. RetriableCommand is using RetryPolicy incorrectly after
HADOOP-7896. (atm)
HADOOP-8317. Update maven-assembly-plugin to 2.3 - fix build on FreeBSD
(Radim Kolar via bobby)
HADOOP-8172. Configuration no longer sets all keys in a deprecated key
list. (Anupam Seth via bobby)
HADOOP-7868. Hadoop native fails to compile when default linker
option is -Wl,--as-needed. (Trevor Robinson via eli)
HADOOP-8655. Fix TextInputFormat for large deliminators. (Gelesh via
bobby)