Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

YARN-6159. Documentation changes for TimelineV2Client (Naganarasimha G R via Varun Saxena) 

(cherry picked from commit 6ba61d20d3)
This commit is contained in:
Varun Saxena 2017-02-21 12:25:37 +05:30
parent 74e43bf287
commit 4a3a4aaf7e
1 changed files with 18 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

44
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServiceV2.md
View File

@ -340,56 +340,48 @@ To write MapReduce framework data to Timeline Service v.2, enable the following
This section is for YARN application developers that want to integrate with Timeline Service v.2. This section is for YARN application developers that want to integrate with Timeline Service v.2.
Developers can continue to use the `TimelineClient` API to publish per-framework data to the Developers need to use the `TimelineV2Client` API to publish per-framework data to the
Timeline Service v.2. You only need to instantiate the right type of the client to write to v.2. Timeline Service v.2. The entity/object API for v.2 is different than v.1 as
On the other hand, the entity/object API for v.2 is different than v.1 as the object model is the object model is significantly changed. The v.2 timeline entity class is
significantly changed. The v.2 timeline entity class is `org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity`.
`org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity` whereas the v.1 class is
`org.apache.hadoop.yarn.api.records.timeline.TimelineEntity`. The methods on `TimelineClient`
suitable for writing to Timeline Service v.2 are clearly delineated, and they use the v.2
types as arguments.
Timeline Service v.2 `putEntities` methods come in 2 varieties: `putEntities` and Timeline Service v.2 `putEntities` methods come in 2 varieties: `putEntities` and
`putEntitiesAsync`. The former is a blocking operation which must be used for writing more `putEntitiesAsync`. The former is a blocking operation which must be used for writing more
critical data (e.g. lifecycle events). The latter is a non-blocking operation. Note that neither critical data (e.g. lifecycle events). The latter is a non-blocking operation. Note that neither
has a return value. has a return value.
Creating a `TimelineClient` for v.2 involves passing in the application id to the factory method. Creating a `TimelineV2Client` involves passing in the application id to the static method
`TimelineV2Client.createTimelineClient`.
For example: For example:
// Create and start the Timeline client v.2 // Create and start the Timeline client v.2
TimelineClient client = TimelineClient.createTimelineClient(appId); TimelineV2Client timelineClient =
client.init(conf); TimelineV2Client.createTimelineClient(appId);
client.start(); timelineClient.init(conf);
timelineClient.start();
try { try {
TimelineEntity myEntity = new TimelineEntity(); TimelineEntity myEntity = new TimelineEntity();
myEntity.setEntityType("MY_APPLICATION"); myEntity.setType("MY_APPLICATION");
myEntity.setEntityId("MyApp1") myEntity.setId("MyApp1");
// Compose other entity info // Compose other entity info
// Blocking write // Blocking write
client.putEntities(entity); timelineClient.putEntities(myEntity);
TimelineEntity myEntity2 = new TimelineEntity(); TimelineEntity myEntity2 = new TimelineEntity();
// Compose other info // Compose other info
// Non-blocking write // Non-blocking write
timelineClient.putEntitiesAsync(entity); timelineClient.putEntitiesAsync(myEntity2);
} catch (IOException e) { } catch (IOException | YarnException e) {
// Handle the exception
} catch (RuntimeException e) {
// In Hadoop 2.6, if attempts submit information to the Timeline Server fail more than the retry limit,
// a RuntimeException will be raised. This may change in future releases, being
// replaced with a IOException that is (or wraps) that which triggered retry failures.
} catch (YarnException e) {
// Handle the exception // Handle the exception
} finally { } finally {
// Stop the Timeline client // Stop the Timeline client
client.stop(); timelineClient.stop();
} }
As evidenced above, you need to specify the YARN application id to be able to write to the Timeline As evidenced above, you need to specify the YARN application id to be able to write to the Timeline
@ -397,9 +389,9 @@ Service v.2. Note that currently you need to be on the cluster to be able to wri
Service. For example, an application master or code in the container can write to the Timeline Service. For example, an application master or code in the container can write to the Timeline
Service, while an off-cluster MapReduce job submitter cannot. Service, while an off-cluster MapReduce job submitter cannot.
After creating the timeline client, user also needs to set the timeline collector address for the application. If `AMRMClient` is used then by registering the timeline client by calling `AMRMClient#registerTimelineClient` is sufficient. After creating the timeline v2 client, user also needs to set the timeline collector address for the application. If `AMRMClient` is used then by registering the timeline client by calling `AMRMClient#registerTimelineV2Client` is sufficient.
amRMClient.registerTimelineClient(timelineClient); amRMClient.registerTimelineV2Client(timelineClient);
Else address needs to be retrieved from the AM allocate response and need to be set in timeline client explicitly. Else address needs to be retrieved from the AM allocate response and need to be set in timeline client explicitly.