YARN-1393. SLS: Add how-to-use instructions. (Wei Yan via kasha)
This commit is contained in:
parent
5bd984691b
commit
c559df2219
|
@ -8,5 +8,25 @@ SLS runs a regular RM without RPC endpoints and uses a NodeManager and
|
||||||
Application Manager simulators to send and receive events simulating cluster
|
Application Manager simulators to send and receive events simulating cluster
|
||||||
and application load behavior.
|
and application load behavior.
|
||||||
|
|
||||||
The size of the cluster and the application load is scripted in a configuration
|
==== Quick Start ====
|
||||||
file.
|
|
||||||
|
Let $HADOOP_ROOT represent the Hadoop install directory. If you build Hadoop
|
||||||
|
yourself, $HADOOP_ROOT is hadoop-dist/target/hadoop-$VERSION. The simulator
|
||||||
|
is located at $HADOOP_ROOT/share/hadoop/tools/sls. The folder sls contains
|
||||||
|
four directories: bin (running scripts), html (web portal to view progress),
|
||||||
|
sample-conf (some example configurations), and sample-data (an example rumen
|
||||||
|
trace).
|
||||||
|
|
||||||
|
STEP 1: Copy all configuration files (under sample-conf) to $HADOOP_ROOT/etc/hadoop.
|
||||||
|
STEP 2: Go to the $HADOOP_ROOT/share/hadoop/tools/sls directory, and run the simulator
|
||||||
|
using the sample rumen trace (under sample-data).
|
||||||
|
|
||||||
|
bin/slsrun.sh —-input-rumen=sample-data/2jobs2min-rumen-jh.json —-output-dir=sample-output
|
||||||
|
|
||||||
|
The simulator will start to run, and you can track the running progress
|
||||||
|
using its web portal (http://$HOST:10001/simulate, where $HOST is the place
|
||||||
|
where you run the simulator.). All collected scheduler metrics are stored
|
||||||
|
under the output-dir during running. This trace takes about 3 mins to finish.
|
||||||
|
|
||||||
|
For more detailed setup, you can check out the document
|
||||||
|
(http://issues.apache.org/jira/secure/attachment/12604817/YARN-1021.pdf)
|
||||||
|
|
|
@ -225,7 +225,8 @@ Yarn Scheduler Load Simulator (SLS)
|
||||||
input traces. The script to start the simulator is <<<slsrun.sh>>>.
|
input traces. The script to start the simulator is <<<slsrun.sh>>>.
|
||||||
|
|
||||||
+----+
|
+----+
|
||||||
$ $HADOOP_ROOT/share/hadoop/tools/sls/bin/slsrun.sh
|
$ cd $HADOOP_ROOT/share/hadoop/tools/sls
|
||||||
|
$ bin/slsrun.sh
|
||||||
--input-rumen|--input-sls=<TRACE_FILE1,TRACE_FILE2,...>
|
--input-rumen|--input-sls=<TRACE_FILE1,TRACE_FILE2,...>
|
||||||
--output-dir=<SLS_SIMULATION_OUTPUT_DIRECTORY> [--nodes=<SLS_NODES_FILE>]
|
--output-dir=<SLS_SIMULATION_OUTPUT_DIRECTORY> [--nodes=<SLS_NODES_FILE>]
|
||||||
[--track-jobs=<JOBID1,JOBID2,...>] [--print-simulation]
|
[--track-jobs=<JOBID1,JOBID2,...>] [--print-simulation]
|
||||||
|
@ -258,7 +259,7 @@ $ $HADOOP_ROOT/share/hadoop/tools/sls/bin/slsrun.sh
|
||||||
convert rumen traces to sls traces.
|
convert rumen traces to sls traces.
|
||||||
|
|
||||||
+----+
|
+----+
|
||||||
$ $HADOOP_ROOT/share/hadoop/tools/sls/bin/rumen2sls.sh
|
$ bin/rumen2sls.sh
|
||||||
--rumen-file=<RUMEN_FILE>
|
--rumen-file=<RUMEN_FILE>
|
||||||
--output-dir=<SLS_OUTPUT_DIRECTORY>
|
--output-dir=<SLS_OUTPUT_DIRECTORY>
|
||||||
[--output-prefix=<SLS_FILE_PREFIX>]
|
[--output-prefix=<SLS_FILE_PREFIX>]
|
||||||
|
|
|
@ -237,6 +237,8 @@ Release 2.7.0 - UNRELEASED
|
||||||
YARN-3075. NodeLabelsManager implementation to retrieve label to node
|
YARN-3075. NodeLabelsManager implementation to retrieve label to node
|
||||||
mapping (Varun Saxena via wangda)
|
mapping (Varun Saxena via wangda)
|
||||||
|
|
||||||
|
YARN-1393. SLS: Add how-to-use instructions. (Wei Yan via kasha)
|
||||||
|
|
||||||
OPTIMIZATIONS
|
OPTIMIZATIONS
|
||||||
|
|
||||||
BUG FIXES
|
BUG FIXES
|
||||||
|
|
Loading…
Reference in New Issue