2013-09-27 16:23:19 -04:00
|
|
|
Yarn Scheduler Load Simulator (SLS)
|
|
|
|
|
|
|
|
|
|
SLS is a stress and performance harness for the Yarn Resource Manager Scheduler
|
|
|
|
|
that exercises the scheduler implementation simulating the cluster size and the
|
|
|
|
|
applications load without having to have a cluster nor applications.
|
|
|
|
|
|
|
|
|
|
SLS runs a regular RM without RPC endpoints and uses a NodeManager and
|
|
|
|
|
Application Manager simulators to send and receive events simulating cluster
|
|
|
|
|
and application load behavior.
|
|
|
|
|
|
2015-02-03 16:20:41 -05:00
|
|
|
==== Quick Start ====
|
|
|
|
|
|
|
|
|
|
Let $HADOOP_ROOT represent the Hadoop install directory. If you build Hadoop
|
|
|
|
|
yourself, $HADOOP_ROOT is hadoop-dist/target/hadoop-$VERSION. The simulator
|
|
|
|
|
is located at $HADOOP_ROOT/share/hadoop/tools/sls. The folder sls contains
|
|
|
|
|
four directories: bin (running scripts), html (web portal to view progress),
|
|
|
|
|
sample-conf (some example configurations), and sample-data (an example rumen
|
|
|
|
|
trace).
|
|
|
|
|
|
|
|
|
|
STEP 1: Copy all configuration files (under sample-conf) to $HADOOP_ROOT/etc/hadoop.
|
|
|
|
|
STEP 2: Go to the $HADOOP_ROOT/share/hadoop/tools/sls directory, and run the simulator
|
|
|
|
|
using the sample rumen trace (under sample-data).
|
|
|
|
|
|
|
|
|
|
bin/slsrun.sh —-input-rumen=sample-data/2jobs2min-rumen-jh.json —-output-dir=sample-output
|
|
|
|
|
|
|
|
|
|
The simulator will start to run, and you can track the running progress
|
|
|
|
|
using its web portal (http://$HOST:10001/simulate, where $HOST is the place
|
|
|
|
|
where you run the simulator.). All collected scheduler metrics are stored
|
|
|
|
|
under the output-dir during running. This trace takes about 3 mins to finish.
|
|
|
|
|
|
|
|
|
|
For more detailed setup, you can check out the document
|
|
|
|
|
(http://issues.apache.org/jira/secure/attachment/12604817/YARN-1021.pdf)
|
