Apache Hadoop 2.7.0 发布

ybny 9年前

Apache Hadoop 2.7.0发布。一共修复了来自社区的535个JIRAs,其中:Hadoop Common有160个;HDFS有192个;YARN有148个;MapReduce有35个。Hadoop 2.7.0是2015年第一个Hadoop release版本,不过需要注意的是
(1)、不要将Hadoop 2.7.0用于生产环境,因为一些关键Bug还在测试中,如果需要在生产环境使用,需要等Hadoop 2.7.1/2.7.2,这些版本很快会发布。
(2)、这个版本的Hadoop已经不支持JDK 1.6运行环境,只支持JDK 7+。
此版本主要做的改进如下:

Hadoop Common

1、支持Windows Azure Storage,BLOB作为Hadoop中的文件系统。

Hadoop HDFS

1、支持文件截断(file truncate);
2、支持每个存储类型配额(Support for quotas per storage type);
3、支持可变长度的块文件

Hadoop YARN

1、YARN安全模块可插拔
2、YARN的本地化资源可以自动共享,全局缓存(测试版)

Hadoop MapReduce

1、能够限制运行的Map/Reduce作业的任务
2、为非常的大Job(有许多输出文件)加快了FileOutputCommitter。

英文原文

Hadoop Common
Windows Azure Storage Blob support (HADOOP-9629), available in trunk for a while, is now integrated into branch-2 and released as part of 2.7.0.
Hadoop HDFS
Enable new read/write scenarios in HDFS by adding support for truncate (HDFS-3107) and support for files with variable-length blocks (HDFS-3689)
Enforce quotas at the Heterogeneous Storage Type granularity in HDFS (HDFS-7584)
Enhance management (HDFS-7424) and monitoring (HDFS-7449) for the NFS Gateway Server
Hadoop YARN
YARN-3100 – Makes YARN authorization pluggable so that tools like Apache Ranger can provide authorization for YARN job submission operations
YARN-1492 – Automatic shared, global caching of YARN localized resources (beta)
Hadoop MapReduce
MAPREDUCE-5583 – Allows the ability to limit the size of a running MapReduce job by restricting the maximum number of Map or Reduce tasks running at any point in time
MAPREDUCE-4815 – Speed up Hive, Pig and MapReduce jobs that deal with many output files by making enhancements to the FileOutputCommitter

Hadoop2.7.0下载

http://hadoop.apache.org/releases.html#Download

来自:http://www.iteblog.com/archives/1329