YARN-1393. SLS: Add how-to-use instructions. (Wei Yan via kasha)

This commit is contained in:
Karthik Kambatla 2015-02-03 13:20:41 -08:00
parent 5bd984691b
commit c559df2219
3 changed files with 27 additions and 4 deletions

View File

@ -8,5 +8,25 @@ SLS runs a regular RM without RPC endpoints and uses a NodeManager and
Application Manager simulators to send and receive events simulating cluster Application Manager simulators to send and receive events simulating cluster
and application load behavior. and application load behavior.
The size of the cluster and the application load is scripted in a configuration ==== Quick Start ====
file.
Let $HADOOP_ROOT represent the Hadoop install directory. If you build Hadoop
yourself, $HADOOP_ROOT is hadoop-dist/target/hadoop-$VERSION. The simulator
is located at $HADOOP_ROOT/share/hadoop/tools/sls. The folder sls contains
four directories: bin (running scripts), html (web portal to view progress),
sample-conf (some example configurations), and sample-data (an example rumen
trace).
STEP 1: Copy all configuration files (under sample-conf) to $HADOOP_ROOT/etc/hadoop.
STEP 2: Go to the $HADOOP_ROOT/share/hadoop/tools/sls directory, and run the simulator
using the sample rumen trace (under sample-data).
bin/slsrun.sh —-input-rumen=sample-data/2jobs2min-rumen-jh.json —-output-dir=sample-output
The simulator will start to run, and you can track the running progress
using its web portal (http://$HOST:10001/simulate, where $HOST is the place
where you run the simulator.). All collected scheduler metrics are stored
under the output-dir during running. This trace takes about 3 mins to finish.
For more detailed setup, you can check out the document
(http://issues.apache.org/jira/secure/attachment/12604817/YARN-1021.pdf)

View File

@ -225,7 +225,8 @@ Yarn Scheduler Load Simulator (SLS)
input traces. The script to start the simulator is <<<slsrun.sh>>>. input traces. The script to start the simulator is <<<slsrun.sh>>>.
+----+ +----+
$ $HADOOP_ROOT/share/hadoop/tools/sls/bin/slsrun.sh $ cd $HADOOP_ROOT/share/hadoop/tools/sls
$ bin/slsrun.sh
--input-rumen|--input-sls=<TRACE_FILE1,TRACE_FILE2,...> --input-rumen|--input-sls=<TRACE_FILE1,TRACE_FILE2,...>
--output-dir=<SLS_SIMULATION_OUTPUT_DIRECTORY> [--nodes=<SLS_NODES_FILE>] --output-dir=<SLS_SIMULATION_OUTPUT_DIRECTORY> [--nodes=<SLS_NODES_FILE>]
[--track-jobs=<JOBID1,JOBID2,...>] [--print-simulation] [--track-jobs=<JOBID1,JOBID2,...>] [--print-simulation]
@ -258,7 +259,7 @@ $ $HADOOP_ROOT/share/hadoop/tools/sls/bin/slsrun.sh
convert rumen traces to sls traces. convert rumen traces to sls traces.
+----+ +----+
$ $HADOOP_ROOT/share/hadoop/tools/sls/bin/rumen2sls.sh $ bin/rumen2sls.sh
--rumen-file=<RUMEN_FILE> --rumen-file=<RUMEN_FILE>
--output-dir=<SLS_OUTPUT_DIRECTORY> --output-dir=<SLS_OUTPUT_DIRECTORY>
[--output-prefix=<SLS_FILE_PREFIX>] [--output-prefix=<SLS_FILE_PREFIX>]

View File

@ -237,6 +237,8 @@ Release 2.7.0 - UNRELEASED
YARN-3075. NodeLabelsManager implementation to retrieve label to node YARN-3075. NodeLabelsManager implementation to retrieve label to node
mapping (Varun Saxena via wangda) mapping (Varun Saxena via wangda)
YARN-1393. SLS: Add how-to-use instructions. (Wei Yan via kasha)
OPTIMIZATIONS OPTIMIZATIONS
BUG FIXES BUG FIXES