Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

MAPREDUCE-5796. Use current version of the archive name in DistributedCacheDeploy document (Akira AJISAKA via aw)

This commit is contained in:
Allen Wittenauer 2014-09-26 12:59:07 -07:00
parent f154ebe8c4
commit b40f433cf8
2 changed files with 10 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-mapreduce-project/CHANGES.txt
View File

@ -384,6 +384,9 @@ Release 2.6.0 - UNRELEASED
in mapred-default should be moved into description tags (Tsuyoshi OZAWA in mapred-default should be moved into description tags (Tsuyoshi OZAWA
via aw) via aw)
MAPREDUCE-5796. Use current version of the archive name in
DistributedCacheDeploy document (Akira AJISAKA via aw)
Release 2.5.1 - 2014-09-05 Release 2.5.1 - 2014-09-05
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES

14
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/DistributedCacheDeploy.apt.vm
View File

@ -67,9 +67,9 @@ Hadoop MapReduce Next Generation - Distributed Cache Deploy
location where the archive is located. As when specifying distributed cache location where the archive is located. As when specifying distributed cache
files for a job, this is a URL that also supports creating an alias for the files for a job, this is a URL that also supports creating an alias for the
archive if a URL fragment is specified. For example, archive if a URL fragment is specified. For example,
<<<hdfs:/mapred/framework/hadoop-mapreduce-2.1.1.tar.gz#mrframework>>> will <<<hdfs:/mapred/framework/hadoop-mapreduce-${project.version}.tar.gz#mrframework>>>
be localized as <<<mrframework>>> rather than will be localized as <<<mrframework>>> rather than
<<<hadoop-mapreduce-2.1.1.tar.gz>>>. <<<hadoop-mapreduce-${project.version}.tar.gz>>>.
[[3]] Configure <<<mapreduce.application.classpath>>> to set the proper [[3]] Configure <<<mapreduce.application.classpath>>> to set the proper
classpath to use with the MapReduce archive configured above. NOTE: An error classpath to use with the MapReduce archive configured above. NOTE: An error
@ -105,14 +105,14 @@ Hadoop MapReduce Next Generation - Distributed Cache Deploy
necessary YARN, HDFS, and Hadoop Common jars and all other dependencies. In necessary YARN, HDFS, and Hadoop Common jars and all other dependencies. In
that case, <<<mapreduce.application.classpath>>> would be configured to that case, <<<mapreduce.application.classpath>>> would be configured to
something like the following example, where the archive basename is something like the following example, where the archive basename is
hadoop-mapreduce-2.1.1.tar.gz and the archive is organized internally similar hadoop-mapreduce-${project.version}.tar.gz and the archive is organized
to the standard Hadoop distribution archive: internally similar to the standard Hadoop distribution archive:
<<<$HADOOP_CONF_DIR,$PWD/hadoop-mapreduce-2.1.1.tar.gz/hadoop-mapreduce-2.1.1/share/hadoop/mapreduce/*,$PWD/hadoop-mapreduce-2.1.1.tar.gz/hadoop-mapreduce-2.1.1/share/hadoop/mapreduce/lib/*,$PWD/hadoop-mapreduce-2.1.1.tar.gz/hadoop-mapreduce-2.1.1/share/hadoop/common/*,$PWD/hadoop-mapreduce-2.1.1.tar.gz/hadoop-mapreduce-2.1.1/share/hadoop/common/lib/*,$PWD/hadoop-mapreduce-2.1.1.tar.gz/hadoop-mapreduce-2.1.1/share/hadoop/yarn/*,$PWD/hadoop-mapreduce-2.1.1.tar.gz/hadoop-mapreduce-2.1.1/share/hadoop/yarn/lib/*,$PWD/hadoop-mapreduce-2.1.1.tar.gz/hadoop-mapreduce-2.1.1/share/hadoop/hdfs/*,$PWD/hadoop-mapreduce-2.1.1.tar.gz/hadoop-mapreduce-2.1.1/share/hadoop/hdfs/lib/*>>> <<<$HADOOP_CONF_DIR,$PWD/hadoop-mapreduce-${project.version}.tar.gz/hadoop-mapreduce-${project.version}/share/hadoop/mapreduce/*,$PWD/hadoop-mapreduce-${project.version}.tar.gz/hadoop-mapreduce-${project.version}/share/hadoop/mapreduce/lib/*,$PWD/hadoop-mapreduce-${project.version}.tar.gz/hadoop-mapreduce-${project.version}/share/hadoop/common/*,$PWD/hadoop-mapreduce-${project.version}.tar.gz/hadoop-mapreduce-${project.version}/share/hadoop/common/lib/*,$PWD/hadoop-mapreduce-${project.version}.tar.gz/hadoop-mapreduce-${project.version}/share/hadoop/yarn/*,$PWD/hadoop-mapreduce-${project.version}.tar.gz/hadoop-mapreduce-${project.version}/share/hadoop/yarn/lib/*,$PWD/hadoop-mapreduce-${project.version}.tar.gz/hadoop-mapreduce-${project.version}/share/hadoop/hdfs/*,$PWD/hadoop-mapreduce-${project.version}.tar.gz/hadoop-mapreduce-${project.version}/share/hadoop/hdfs/lib/*>>>
Another possible approach is to have the archive consist of just the Another possible approach is to have the archive consist of just the
MapReduce jars and have the remaining dependencies picked up from the Hadoop MapReduce jars and have the remaining dependencies picked up from the Hadoop
distribution installed on the nodes. In that case, the above example would distribution installed on the nodes. In that case, the above example would
change to something like the following: change to something like the following:
<<<$HADOOP_CONF_DIR,$PWD/hadoop-mapreduce-2.1.1.tar.gz/hadoop-mapreduce-2.1.1/share/hadoop/mapreduce/*,$PWD/hadoop-mapreduce-2.1.1.tar.gz/hadoop-mapreduce-2.1.1/share/hadoop/mapreduce/lib/*,$HADOOP_COMMON_HOME/share/hadoop/common/*,$HADOOP_COMMON_HOME/share/hadoop/common/lib/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*,$HADOOP_YARN_HOME/share/hadoop/yarn/*,$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*>>> <<<$HADOOP_CONF_DIR,$PWD/hadoop-mapreduce-${project.version}.tar.gz/hadoop-mapreduce-${project.version}/share/hadoop/mapreduce/*,$PWD/hadoop-mapreduce-${project.version}.tar.gz/hadoop-mapreduce-${project.version}/share/hadoop/mapreduce/lib/*,$HADOOP_COMMON_HOME/share/hadoop/common/*,$HADOOP_COMMON_HOME/share/hadoop/common/lib/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*,$HADOOP_YARN_HOME/share/hadoop/yarn/*,$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*>>>