Spring for Apache Hadoop 1.0 RC1 发布

jopen 11年前

Spring Hadoop为开发 Apache Hadoop 应用程序提供支持,充分利用 Spring, Spring Batch 和 Spring Integration 等项目提供的功能。

Changes in version 1.0 RC1 (2012-10-07)  ---------------------------------------    General  * 引入 Hive, Pig 运行器用于执行声明脚本  * 重构 (Cascading, M/R, Hive, Pig) 运行器,用 Callables 代替 FactoryBeans  * 重命名 'pig' 至 'pig-factory' 和 'pig-ref' 至 'pig-factory-ref'  * 重命名 'hive-client' 至 'hive-client-factory' 和 'hive-client-ref' 至 'hive-client-factory-ref'  * 引入所有运行器(Cascading, M/R, Hive, Pig) 的执行前和执行后的动作   * 引入嵌入式执行 Hadoop Jar  * 改进 spring-hadoop.xsd 命名空间  * 改进,完善和扩大的参考文档  * 改进 artifacts pom  * 升级至 Spring Batch 2.1.9  * 升级至 Hive 0.9.0  * 升级至 Pig 0.10.0  * 升级至 Gradle 1.2    Package o.s.data.hadoop.cascading  * Introduced FlowFactoryBean    Package o.s.data.hadoop.configuration  * Fixed potential cycle with FileSystem url registration    Package o.s.data.hadoop.fs  * Added codecs support to hdfs resources  * Refined DistributedCache fragment creation for CDH4/Hadoop 0.23 distros  * Introduced options for closing the FileSystem  * Fine-tuned the DistributedCache API for setting cache entries    Package o.s.data.hadoop.hbase  * Refined resource management of HBase tables    Package o.s.data.hadoop.hive  * Addressed swallowed exception occuring script execution  * Improved HiveQL parsing for multi-line statements  * Introduced variable binding and substitution per Hive script  * Refined namespace to preserve parameter ordering  * Introduced HiveClient factory (to deal with thread-safety issues)  * Introduced HiveTemplate & callback  * Introduced extended exception conversion to DataAccessException  * Introduced HiveRunner    Package o.s.data.hadoop.mapreduce  * Introduced scope attribute for job definitions  * Introduced verbose flag to job tasklet  * Introduced more options for job and streaming namespace  * Introduced jar executor  * Refined Tool and Jar execution to prevent class loading leaks  * Refactored JobRunner FactoryBean into a Callable  * Introduced namespace for job-runner  * Removed path validation from JobFactoryBean    Package o.s.data.hadoop.pig  * Refined namespace to preserve parameter ordering  * Introduced PigServer factory (to deal with thread-safety issues)  * Introduced PigTemplate & callback  * Introduced extended exception conversion to DataAccessException  * Refined execution of Pig scripts  * Introduced PigRunner    Package o.s.data.hadoop.scripting  * Refactored HdfsScriptFactoryBean into HdfsScriptRunner  * Script definitions no longer cause execution on container lookup