Apache Hadoop 2.5.0 Release Notes

These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements.

s3 server-side encryption is now supported.

To enable this feature, specify the following in your client-side configuration:

name: fs.s3n.server-side-encryption-algorithm value: AES256

  • HADOOP-10451 | Trivial | Remove unused field and imports from SaslRpcServer

SaslRpcServer.SASL_PROPS is removed. Any use of this variable should be replaced with the following code: SaslPropertiesResolver saslPropsResolver = SaslPropertiesResolver.getInstance(conf); Map<String, String> sasl_props = saslPropsResolver.getDefaultProperties();

  • HADOOP-10342 | Major | Extend UserGroupInformation to return a UGI given a preauthenticated kerberos Subject

Add getUGIFromSubject to leverage an external kerberos authentication

  • HADOOP-9919 | Major | Update hadoop-metrics2.properties examples to Yarn

Remove MRv1 settings from hadoop-metrics2.properties, add YARN settings instead.

  • HDFS-6293 | Blocker | Issues with OIV processing PB-based fsimages

Set “dfs.namenode.legacy-oiv-image.dir” to an appropriate directory to make standby name node or secondary name node save its file system state in the old fsimage format during checkpointing. This image can be used for offline analysis using the OfflineImageViewer. Use the “hdfs oiv_legacy” command to process the old fsimage format.

  • HDFS-6273 | Major | Config options to allow wildcard endpoints for namenode HTTP and HTTPS servers

HDFS-6273 introduces two new HDFS configuration keys:

  • dfs.namenode.http-bind-host
  • dfs.namenode.https-bind-host

The most common use case for these keys is to have the NameNode HTTP (or HTTPS) endpoints listen on all interfaces on multi-homed systems by setting the keys to 0.0.0.0 i.e. INADDR_ANY.

For the systems background on this usage of INADDR_ANY please refer to ip(7) in the Linux Programmer's Manual (web link: http://man7.org/linux/man-pages/man7/ip.7.html).

These keys complement the existing NameNode options:

  • dfs.namenode.rpc-bind-host
  • dfs.namenode.servicerpc-bind-host
  • HDFS-6168 | Major | Remove deprecated methods in DistributedFileSystem

WARNING: No release note provided for this incompatible change.

  • HDFS-6164 | Major | Remove lsr in OfflineImageViewer

The offlineimageviewer no longer generates lsr-style outputs. The functionality has been superseded by a tool that takes the fsimage and exposes WebHDFS-like API for user queries.

  • HDFS-6153 | Minor | Document “fileId” and “childrenNum” fields in the FileStatus Json schema

WARNING: No release note provided for this incompatible change.

  • HDFS-6110 | Major | adding more slow action log in critical write path

Log slow i/o. Set log thresholds in dfsclient and datanode via the below new configs:

dfs.client.slow.io.warning.threshold.ms (Default 30 seconds) dfs.datanode.slow.io.warning.threshold.ms (Default 300ms)

  • MAPREDUCE-5777 | Major | Support utf-8 text with BOM (byte order marker)

WARNING: No release note provided for this incompatible change.

  • YARN-2107 | Major | Refactor timeline classes into server.timeline package

WARNING: No release note provided for this incompatible change.