Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge -c 1195579 from trunk to branch-0.23 to fix MAPREDUCE-3275. 

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23@1195581 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Arun Murthy 2011-10-31 17:34:25 +00:00
parent 07c755a394
commit d6d2ff8206
2 changed files with 67 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-mapreduce-project/CHANGES.txt
View File

@ -407,6 +407,9 @@ Release 0.23.0 - Unreleased
MAPREDUCE-3146. Added a MR specific command line to dump logs for a
given TaskAttemptID. (Siddharth Seth via vinodkv)
MAPREDUCE-3275. Added documentation for AM WebApp Proxy. (Robert Evans via
acmurthy)
OPTIMIZATIONS
MAPREDUCE-2026. Make JobTracker.getJobCounters() and

70
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
View File

@ -100,6 +100,8 @@ Hadoop MapReduce Next Generation - Cluster Setup
| ResourceManager | YARN_RESOURCEMANAGER_OPTS |
*--------------------------------------+--------------------------------------+
| NodeManager | YARN_NODEMANAGER_OPTS |
*--------------------------------------+--------------------------------------+
| WebAppProxy | YARN_PROXYSERVER_OPTS |
*--------------------------------------+--------------------------------------+
For example, To configure Namenode to use parallelGC, the following
@ -450,9 +452,16 @@ Hadoop MapReduce Next Generation - Cluster Setup
Run a script to start NodeManagers on all slaves:
----
$ $YARN_HOME/bin/hdfs start nodemanager --config $HADOOP_CONF_DIR
$ $YARN_HOME/bin/yarn start nodemanager --config $HADOOP_CONF_DIR
----
Start a standalone WebAppProxy server. If multiple servers
are used with load balancing it should be run on each of them:
----
$ $YARN_HOME/bin/yarn start proxyserver --config $HADOOP_CONF_DIR
----
Start the MapReduce JobHistory Server with the following command, run on the
designated server:
@ -485,9 +494,17 @@ Hadoop MapReduce Next Generation - Cluster Setup
Run a script to stop NodeManagers on all slaves:
----
$ $YARN_HOME/bin/hdfs stop nodemanager --config $HADOOP_CONF_DIR
$ $YARN_HOME/bin/yarn stop nodemanager --config $HADOOP_CONF_DIR
----
Stop the WebAppProxy server. If multiple servers are used with load
balancing it should be run on each of them:
----
$ $YARN_HOME/bin/yarn stop proxyserver --config $HADOOP_CONF_DIR
----
Stop the MapReduce JobHistory Server with the following command, run on the
designated server:
@ -502,7 +519,7 @@ Hadoop MapReduce Next Generation - Cluster Setup
to run Hadoop in <<secure mode>> with strong, Kerberos-based
authentication.
* <<<User Acccounts for Hadoop Daemons>>>
* <<<User Accounts for Hadoop Daemons>>>
Ensure that HDFS and YARN daemons run as different Unix users, for e.g.
<<<hdfs>>> and <<<yarn>>>. Also, ensure that the MapReduce JobHistory
@ -751,6 +768,31 @@ KVNO Timestamp Principal
* <<<conf/yarn-site.xml>>>
* WebAppProxy
The <<<WebAppProxy>>> provides a proxy between the web applications
exported by an application and an end user. If security is enabled
it will warn users before accessing a potentially unsafe web application.
Authentication and authorization using the proxy is handled just like
any other privileged web application.
*-------------------------+-------------------------+------------------------+
|| Parameter || Value || Notes |
*-------------------------+-------------------------+------------------------+
| <<<yarn.web-proxy.address>>> | | |
| | <<<WebAppProxy>>> host:port for proxy to AM web apps. | |
| | | <host:port> if this is the same as <<<yarn.resourcemanager.webapp.address>>>|
| | | or it is not defined then the <<<ResourceManager>>> will run the proxy|
| | | otherwise a standalone proxy server will need to be launched.|
*-------------------------+-------------------------+------------------------+
| <<<yarn.web-proxy.keytab>>> | | |
| | </etc/security/keytab/web-app.service.keytab> | |
| | | Kerberos keytab file for the WebAppProxy. |
*-------------------------+-------------------------+------------------------+
| <<<yarn.web-proxy.principal>>> | wap/_HOST@REALM.TLD | |
| | | Kerberos principal name for the WebAppProxy. |
*-------------------------+-------------------------+------------------------+
* LinuxContainerExecutor
A <<<ContainerExecutor>>> used by YARN framework which define how any
@ -968,7 +1010,15 @@ KVNO Timestamp Principal
Run a script to start NodeManagers on all slaves as <yarn>:
----
[yarn]$ $YARN_HOME/bin/hdfs start nodemanager --config $HADOOP_CONF_DIR
[yarn]$ $YARN_HOME/bin/yarn start nodemanager --config $HADOOP_CONF_DIR
----
Start a standalone WebAppProxy server. Run on the WebAppProxy
server as <yarn>. If multiple servers are used with load balancing
it should be run on each of them:
----
[yarn]$ $YARN_HOME/bin/yarn start proxyserver --config $HADOOP_CONF_DIR
----
Start the MapReduce JobHistory Server with the following command, run on the
@ -1003,12 +1053,20 @@ KVNO Timestamp Principal
Run a script to stop NodeManagers on all slaves as <yarn>:
----
[yarn]$ $YARN_HOME/bin/hdfs stop nodemanager --config $HADOOP_CONF_DIR
[yarn]$ $YARN_HOME/bin/yarn stop nodemanager --config $HADOOP_CONF_DIR
----
Stop the WebAppProxy server. Run on the WebAppProxy server as
<yarn>. If multiple servers are used with load balancing it
should be run on each of them:
----
[yarn]$ $YARN_HOME/bin/yarn stop proxyserver --config $HADOOP_CONF_DIR
----
Stop the MapReduce JobHistory Server with the following command, run on the
designated server as <mapred>:
----
[mapred]$ $YARN_HOME/bin/yarn stop historyserver --config $HADOOP_CONF_DIR
----