Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

YARN-4708. Missing default mapper type in TimelineServer performance test tool usage. Contributed by Kai Sasaki. 

(cherry picked from commit b68901d7dd)
This commit is contained in:
Tsuyoshi Ozawa 2016-02-22 07:49:50 +09:00
parent 7660cfd91a
commit 929e869ae0
3 changed files with 10 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TimelineServicePerformance.java
View File

@ -48,7 +48,7 @@ protected static int printUsage() {
")\n" + ")\n" +
" [-v] timeline service version\n" + " [-v] timeline service version\n" +
" [-mtype <mapper type in integer>]\n" + " [-mtype <mapper type in integer>]\n" +
" 1. simple entity write mapper\n" + " 1. simple entity write mapper (default)\n" +
" 2. jobhistory files replay mapper\n" + " 2. jobhistory files replay mapper\n" +
" [-s <(KBs)test>] number of KB per put (mtype=1, default: " + " [-s <(KBs)test>] number of KB per put (mtype=1, default: " +
SimpleEntityWriterV1.KBS_SENT_DEFAULT + " KB)\n" + SimpleEntityWriterV1.KBS_SENT_DEFAULT + " KB)\n" +

2
hadoop-yarn-project/CHANGES.txt
View File

@ -76,6 +76,8 @@ Release 2.9.0 - UNRELEASED
YARN-4689. FairScheduler: Cleanup preemptContainer to be more readable. YARN-4689. FairScheduler: Cleanup preemptContainer to be more readable.
(Kai Sasaki via kasha) (Kai Sasaki via kasha)
YARN-4708. Missing default mapper type in TimelineServer performance test tool
usage. (Kai Sasaki via ozawa)
OPTIMIZATIONS OPTIMIZATIONS

13
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServer.md
View File

@ -783,8 +783,8 @@ String.
| `entitytype` | string | The entity type | | `entitytype` | string | The entity type |
| `relatedentities` | map | The related entities' identifiers, which are organized in a map of entityType : [entity1, entity2, ...] | | `relatedentities` | map | The related entities' identifiers, which are organized in a map of entityType : [entity1, entity2, ...] |
| `events` | list | The events of the entity | | `events` | list | The events of the entity |
| `primaryfilters` | map | The primary filters of the entity, which are orgainzied in a map of key : [value1, value2, ...] | | `primaryfilters` | map | The primary filters of the entity, which are organized in a map of key : [value1, value2, ...] |
| `otherinfo` | map | The other information of the entity, which is orgainzied in a map of key : value | | `otherinfo` | map | The other information of the entity, which is organized in a map of key : value |
| `starttime` | long | The start time of the entity | | `starttime` | long | The start time of the entity |
### Response Examples: ### Response Examples:
@ -2038,7 +2038,7 @@ this failure *will not* result in an HTTP error code being retured.
A status code of 200 will be returned —however, there will be an error code A status code of 200 will be returned —however, there will be an error code
in the list of failed entities for each entity which could not be added. in the list of failed entities for each entity which could not be added.
<a name="TIMELINE_SERVER_PERFORMANCE_TEST_TOOL"></a> Timelnine Server Performance Test Tool <a name="TIMELINE_SERVER_PERFORMANCE_TEST_TOOL"></a> Timeline Server Performance Test Tool
---------- ----------
###<a name="HIGHLIGHTS"></a>Highlights ###<a name="HIGHLIGHTS"></a>Highlights
@ -2054,7 +2054,8 @@ Mapper Types Description:
1. SimpleEntityWriter mapper 1. SimpleEntityWriter mapper
Each mapper writes a user-specified number of timeline entities Each mapper writes a user-specified number of timeline entities
with a user-specified size to the timeline server. with a user-specified size to the timeline server. SimpleEntityWrite
is a default mapper of the performance test tool.
2. JobHistoryFileReplay mapper 2. JobHistoryFileReplay mapper
Each mapper replays jobhistory files under a specified directory Each mapper replays jobhistory files under a specified directory
@ -2071,7 +2072,7 @@ Options:
[-m <maps>] number of mappers (default: 1) [-m <maps>] number of mappers (default: 1)
[-v] timeline service version [-v] timeline service version
[-mtype <mapper type in integer>] [-mtype <mapper type in integer>]
1. simple entity write mapper 1. simple entity write mapper (default)
2. jobhistory files replay mapper 2. jobhistory files replay mapper
[-s <(KBs)test>] number of KB per put (mtype=1, default: 1 KB) [-s <(KBs)test>] number of KB per put (mtype=1, default: 1 KB)
[-t] package sending iterations per mapper (mtype=1, default: 100) [-t] package sending iterations per mapper (mtype=1, default: 100)
@ -2105,7 +2106,7 @@ Example input of JobHistoryFileReplay mapper test:
/testInput/job_2.jhist /testInput/job_2.jhist
/testInput/job_2_conf.xml /testInput/job_2_conf.xml
Eample output of JobHistoryFileReplay test: Example output of JobHistoryFileReplay test:
TRANSACTION RATE (per mapper): 4000.0 ops/s TRANSACTION RATE (per mapper): 4000.0 ops/s
IO RATE (per mapper): 0.0 KB/s IO RATE (per mapper): 0.0 KB/s