Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

YARN-5174. [documentation] several updates/corrections to timeline service documentation (Sangjin Lee, Varun Saxena, Naganarasimha G R, and Li Lu via sjlee)

This commit is contained in:
Sangjin Lee 2016-07-01 16:23:00 -07:00
parent 8bf87eede2
commit b313ce06b2
2 changed files with 219 additions and 159 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

378
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/TimelineServiceV2.md
View File

@ -21,7 +21,7 @@ The YARN Timeline Service v.2
* [Current Status](#Current_Status) * [Current Status](#Current_Status)
* [Deployment](#Deployment) * [Deployment](#Deployment)
* [Configurations](#Configurations) * [Configurations](#Configurations)
* [Enabling the Timeline Service v.2](#Enabling_Timeline_Service_v2) * [Enabling Timeline Service v.2](#Enabling_Timeline_Service_v2)
* [Publishing of application specific data](#Publishing_of_application_specific_data) * [Publishing of application specific data](#Publishing_of_application_specific_data)
* [Timeline Service v.2 REST API](#Timeline_Service_REST_API_v2) * [Timeline Service v.2 REST API](#Timeline_Service_REST_API_v2)
* [Query Flows](#REST_API_LIST_FLOWS) * [Query Flows](#REST_API_LIST_FLOWS)
@ -55,11 +55,15 @@ scales well to a large size while maintaining good response times for reads and
#### Usability improvements #### Usability improvements
In many cases, users are interested in information at the level of "flows" or logical groups of In many cases, users are interested in information at the level of "flows" or logical groups of
YARN applications. It is much more common to launch a set or series of YARN applications to YARN applications. It is much more common to launch a set or series of YARN applications to
complete a logic application. Timeline Service v.2 supports the notion of flows explicitly. In complete a logical application. Timeline Service v.2 supports the notion of flows explicitly. In
addition, it supports aggregating metrics at the flow level. addition, it supports aggregating metrics at the flow level.
Also, information such as configuration and metrics is treated and supported as a first-class Also, information such as configuration and metrics is treated and supported as first-class
citizen. citizens.
The following diagrams illustrates the relationship between different YARN entities modelling flows.
![Flow Hierarchy](./images/flow_hierarchy.png)
###<a name="Architecture"></a>Architecture ###<a name="Architecture"></a>Architecture
@ -85,30 +89,32 @@ The following diagram illustrates the design at a high level.
### <a name="Current_Status"></a>Current Status and Future Plans ### <a name="Current_Status"></a>Current Status and Future Plans
YARN Timeline Service v.2 is currently in alpha. It is very much work in progress, and many things YARN Timeline Service v.2 is currently in alpha ("alpha 1"). It is very much work in progress, and
can and will change rapidly. Users must enable Timeline Service v.2 only on a test or many things can and will change rapidly. Users must enable Timeline Service v.2 only on a test or
experimental cluster to test the feature. experimental cluster to test the feature.
A complete end-to-end flow of writes and reads must be functional, with Apache HBase as the Most importantly, **security is not enabled**. Do not set up or use Timeline Service v.2 until
backend. You must be able to start generating data. When enabled, all YARN-generic events are security is implemented if security is a requirement.
A complete end-to-end flow of writes and reads is functional, with Apache HBase as the backend.
You should be able to start generating data. When enabled, all YARN-generic events are
published as well as YARN system metrics such as CPU and memory. Furthermore, some applications published as well as YARN system metrics such as CPU and memory. Furthermore, some applications
including Distributed Shell and MapReduce write per-framework data to YARN Timeline Service v.2. including Distributed Shell and MapReduce can write per-framework data to YARN Timeline Service
v.2.
The REST API comes with a good number of useful and flexible query patterns (see below for more The basic mode of accessing data is via REST. Currently there is no support for command line
information). access. The REST API comes with a good number of useful and flexible query patterns (see below for
more information).
Although the basic mode of accessing data is via REST, it also comes with a basic web UI based on
the proposed new YARN UI framework. Currently there is no support for command line access, however.
The collectors (writers) are currently embedded in the node managers as auxiliary services. The The collectors (writers) are currently embedded in the node managers as auxiliary services. The
resource manager also has its dedicated in-process collector. The reader is currently a single resource manager also has its dedicated in-process collector. The reader is currently a single
instance. Currently, it is not possible to write to Timeline Service outside the context of a YARN instance. Currently, it is not possible to write to Timeline Service outside the context of a YARN
application (i.e. no off-cluster client). application (i.e. no off-cluster client).
When YARN Timeline Service v.2 is disabled, one must expect no functional or performance impact When YARN Timeline Service v.2 is disabled, one can expect no functional or performance impact
on any other existing functionality. on any other existing functionality.
The work to make it production-ready continues. Some key items include The work to make it truly production-ready continues. Some key items include
* More robust storage fault tolerance * More robust storage fault tolerance
* Security * Security
@ -134,10 +140,9 @@ New configuration parameters that are introduced with v.2 are marked bold.
|:---- |:---- | |:---- |:---- |
| `yarn.timeline-service.enabled` | Indicate to clients whether Timeline service is enabled or not. If enabled, the `TimelineClient` library used by applications will post entities and events to the Timeline server. Defaults to `false`. | | `yarn.timeline-service.enabled` | Indicate to clients whether Timeline service is enabled or not. If enabled, the `TimelineClient` library used by applications will post entities and events to the Timeline server. Defaults to `false`. |
| `yarn.timeline-service.version` | Indicate what is the current version of the running timeline service. For example, if "yarn.timeline-service.version" is 1.5, and "yarn.timeline-service.enabled" is true, it means the cluster will and must bring up the timeline service v.1.5 (and nothing else). On the client side, if the client uses the same version of timeline service, it must succeed. If the client chooses to use a smaller version in spite of this, then depending on how robust the compatibility story is between versions, the results may vary. Defaults to `1.0f`. | | `yarn.timeline-service.version` | Indicate what is the current version of the running timeline service. For example, if "yarn.timeline-service.version" is 1.5, and "yarn.timeline-service.enabled" is true, it means the cluster will and must bring up the timeline service v.1.5 (and nothing else). On the client side, if the client uses the same version of timeline service, it must succeed. If the client chooses to use a smaller version in spite of this, then depending on how robust the compatibility story is between versions, the results may vary. Defaults to `1.0f`. |
| **`yarn.timeline-service.writer.class`** | The class for the backend storage writer. Defaults to a filesystem storage writer, therefore it must be overridden. | | **`yarn.timeline-service.writer.class`** | The class for the backend storage writer. Defaults to HBase storage writer. |
| **`yarn.timeline-service.reader.class`** | The class for the backend storage reader. Defaults to a filesystem storage reader, therefore it must be overridden. | | **`yarn.timeline-service.reader.class`** | The class for the backend storage reader. Defaults to HBase storage reader. |
| **`yarn.system-metrics-publisher.enabled`** | The setting that controls whether yarn system metrics is published on the Timeline service or not by RM And NM. Defaults to `false`. | | **`yarn.system-metrics-publisher.enabled`** | The setting that controls whether yarn system metrics is published on the Timeline service or not by RM And NM. Defaults to `false`. |
| **`yarn.rm.system-metrics-publisher.emit-container-events`** | The setting that controls whether yarn container metrics is published to the timeline server or not by RM. This configuration setting is for ATS V2. Defaults to `false`. |
#### Advanced configuration #### Advanced configuration
@ -150,20 +155,24 @@ New configuration parameters that are introduced with v.2 are marked bold.
| **`yarn.timeline-service.writer.flush-interval-seconds`** | The setting that controls how often the timeline collector flushes the timeline writer. Defaults to `60`. | | **`yarn.timeline-service.writer.flush-interval-seconds`** | The setting that controls how often the timeline collector flushes the timeline writer. Defaults to `60`. |
| **`yarn.timeline-service.app-collector.linger-period.ms`** | Time period till which the application collector will be alive in NM, after the application master container finishes. Defaults to `1000` (1 second). | | **`yarn.timeline-service.app-collector.linger-period.ms`** | Time period till which the application collector will be alive in NM, after the application master container finishes. Defaults to `1000` (1 second). |
| **`yarn.timeline-service.timeline-client.number-of-async-entities-to-merge`** | Time line V2 client tries to merge these many number of async entities (if available) and then call the REST ATS V2 API to submit. Defaults to `10`. | | **`yarn.timeline-service.timeline-client.number-of-async-entities-to-merge`** | Time line V2 client tries to merge these many number of async entities (if available) and then call the REST ATS V2 API to submit. Defaults to `10`. |
| **`yarn.timeline-service.coprocessor.app-final-value-retention-milliseconds`** | The setting that controls how long the final value of a metric of a completed app is retained before merging into the flow sum. Defaults to `259200000` (3 days). | | **`yarn.timeline-service.hbase.coprocessor.app-final-value-retention-milliseconds`** | The setting that controls how long the final value of a metric of a completed app is retained before merging into the flow sum. Defaults to `259200000` (3 days). This should be set in the HBase cluster. |
| **`yarn.rm.system-metrics-publisher.emit-container-events`** | The setting that controls whether yarn container metrics is published to the timeline server or not by RM. This configuration setting is for ATS V2. Defaults to `false`. |
### <a name="Enabling_Timeline_Service_v2"></a>Enabling the Timeline Service v.2 ### <a name="Enabling_Timeline_Service_v2"></a>Enabling Timeline Service v.2
#### Preparing Apache HBase cluster for storage #### Preparing Apache HBase cluster for storage
The first part is to set up or pick an Apache HBase cluster to use as the storage cluster. Once The first part is to set up or pick an Apache HBase cluster to use as the storage cluster. The
you have an HBase cluster ready to use for this purpose, perform the following steps. version of Apache HBase that is supported with Timeline Service v.2 is 1.1.x. The 1.0.x versions
do not work with Timeline Service v.2. The 1.2.x versions have not been tested.
Once you have an Apache HBase cluster ready to use for this purpose, perform the following steps.
First, add the timeline service jar to the HBase classpath in all HBase machines in the cluster. It First, add the timeline service jar to the HBase classpath in all HBase machines in the cluster. It
is needed for the coprocessor as well as the schema creator. For example, is needed for the coprocessor as well as the schema creator. For example,
cp hadoop-yarn-server-timelineservice-3.0.0-SNAPSHOT.jar /usr/hbase/lib/ cp hadoop-yarn-server-timelineservice-3.0.0-alpha1-SNAPSHOT.jar /usr/hbase/lib/
Then, enable the coprocessor that handles the aggregation. To enable it, add the following entry in Then, enable the coprocessor that handles the aggregation. To enable it, add the following entry in
region servers' `hbase-site.xml` file (generally located in the `conf` directory) as follows: region servers' `hbase-site.xml` file (generally located in the `conf` directory) as follows:
@ -201,16 +210,6 @@ Following are the basic configurations to start Timeline service v.2:
<value>true</value> <value>true</value>
</property> </property>
<property>
<name>yarn.timeline-service.writer.class</name>
<value>org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineWriterImpl</value>
</property>
<property>
<name>yarn.timeline-service.reader.class</name>
<value>org.apache.hadoop.yarn.server.timelineservice.storage.HBaseTimelineReaderImpl</value>
</property>
<property> <property>
<name>yarn.nodemanager.aux-services</name> <name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle,timeline_collector</value> <value>mapreduce_shuffle,timeline_collector</value>
@ -237,7 +236,7 @@ Following are the basic configurations to start Timeline service v.2:
</property> </property>
``` ```
In addition, you may want to set the YARN cluster name to a reasonably unique name in case you In addition, you may want to set the YARN cluster name to a reasonably unique value in case you
are using multiple clusters to store data in the same Apache HBase storage: are using multiple clusters to store data in the same Apache HBase storage:
``` ```
@ -277,10 +276,11 @@ This section is for YARN application developers that want to integrate with Time
Developers can continue to use the `TimelineClient` API to publish per-framework data to the Developers can continue to use the `TimelineClient` API to publish per-framework data to the
Timeline Service v.2. You only need to instantiate the right type of the client to write to v.2. Timeline Service v.2. You only need to instantiate the right type of the client to write to v.2.
On the other hand, the entity/object API for v.2 is different than v.1 as the object model is On the other hand, the entity/object API for v.2 is different than v.1 as the object model is
significantly changed. The v.2 timeline entity class is `org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity` significantly changed. The v.2 timeline entity class is
whereas the v.1 class is `org.apache.hadoop.yarn.api.records.timeline.TimelineEntity`. The methods `org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity` whereas the v.1 class is
on `TimelineClient` suitable for writing to the Timeline Service v.2 are clearly delineated, and `org.apache.hadoop.yarn.api.records.timeline.TimelineEntity`. The methods on `TimelineClient`
they use the v.2 types as arguments. suitable for writing to Timeline Service v.2 are clearly delineated, and they use the v.2
types as arguments.
Timeline Service v.2 `putEntities` methods come in 2 varieties: `putEntities` and Timeline Service v.2 `putEntities` methods come in 2 varieties: `putEntities` and
`putEntitiesAsync`. The former is a blocking operation which must be used for writing more `putEntitiesAsync`. The former is a blocking operation which must be used for writing more
@ -330,16 +330,58 @@ Service v.2. Note that currently you need to be on the cluster to be able to wri
Service. For example, an application master or code in the container can write to the Timeline Service. For example, an application master or code in the container can write to the Timeline
Service, while an off-cluster MapReduce job submitter cannot. Service, while an off-cluster MapReduce job submitter cannot.
After creating the timeline client, user also needs to set the timeline collector address for the application. If `AMRMClient` is used then by registering the timeline client by calling `AMRMClient#registerTimelineClient` is sufficient.
amRMClient.registerTimelineClient(timelineClient);
Else address needs to be retreived from the AM allocate response and need to be set in timeline client explicitly.
timelineClient.setTimelineServiceAddress(response.getCollectorAddr());
You can create and publish your own entities, events, and metrics as with previous versions. You can create and publish your own entities, events, and metrics as with previous versions.
TimelineEntity objects have the following fields to hold timeline data:
* events: A set of TimelineEvents, ordered by the timestamp of the events in descending
order. Each event contains one id and a map to store related information and is
associated with one timestamp.
* configs: A map from a string (config name) to a string (config value) representing all
configs associated with the entity. Users can post the whole config or a part of it in the
configs field. Supported for application and generic entities. Supported for application and
generic entities.
* metrics: A set of metrics related to this entity. There are two types of metrics: single
value metric and time series metric. Each metric item contains metric name (id), value, and what
kind of aggregation operation should be performed in this metric (no­op by default). Supported for
flow run, application and generic entities.
* info: A map from a string (info key name) to an object (info value) to hold up related
information for this entity. Supported for application and generic entities.
* isrelatedtoEntities and relatestoEntities: It is also possible to represent relationships between
entities. Each entity contains a relatestoEntities and isrelatedtoEntities fields to represent
relationships to other entities. Both fields are represented by a map from a string (the name of
the relationship) to a timeline entity. In this way relationships among entities can be
represented as a DAG.
Note that when posting timeline metrics, one may choose how each metric should be aggregated
through the `TimelineMetric#setRealtimeAggregationOp()` method. The word "aggregate" here means
applying one of the `TimelineMetricOperation` for a set of entities. Timeline service v2 provides
built-in application level aggregation, which means aggregating metrics from different timeline
entities within one YARN application. Right now, there are two kinds of operations supported in
`TimelineMetricOperation`:
* `MAX`: Getting the maximum value among all `TimelineMetric` objects.
* `SUM`: Getting the sum of all `TimelineMetric` objects.
By default, the `NOP` operation means not performing any real-time aggregation operation.
Application frameworks must set the "flow context" whenever possible in order to take advantage Application frameworks must set the "flow context" whenever possible in order to take advantage
of the flow support Timeline Service v.2 provides. The flow context consists of the following: of the flow support Timeline Service v.2 provides. The flow context consists of the following:
* Flow name: a string that identifies the high-level flow (e.g. "distributed grep" or any * Flow name: a string that identifies the high-level flow (e.g. "distributed grep" or any
identifiable name that can uniquely represent the app) identifiable name that can uniquely represent the app)
* Flow run id: a monotonically-increasing sequence of numbers that distinguish different runs of * Flow run id: a monotonically-increasing sequence of numbers that distinguish different runs of
the same flow the same flow.
* (optional) Flow version: a string identifier that denotes a version of the flow * (optional) Flow version: a string identifier that denotes a version of the flow. Flow version can
be used to identify changes in the flows, such as code changes or script changes.
If the flow context is not specified, defaults are supplied for these attributes: If the flow context is not specified, defaults are supplied for these attributes:
@ -359,10 +401,11 @@ You can provide the flow context via YARN application tags:
appContext.setApplicationTags(tags); appContext.setApplicationTags(tags);
# <a name="Timeline_Service_REST_API_v2"></a>Timeline Service v.2 REST API # <a name="Timeline_Service_REST_API_v2"></a>Timeline Service v.2 REST API
Querying the Timeline Service v.2 is currently only supported via REST API; there is no API Querying Timeline Service v.2 is currently only supported via REST API; there is no API client
client implemented in the YARN libraries. implemented in the YARN libraries.
The v.2 REST API is implemented at under the path, `/ws/v2/timeline/` on the Timeline Service web The v.2 REST API is implemented at under the path, `/ws/v2/timeline/` on the Timeline Service web
service. service.
@ -377,11 +420,11 @@ Returns a JSON object describing the service instance and version information.
{ {
"About":"Timeline Reader API", "About":"Timeline Reader API",
"timeline-service-version":"3.0.0-SNAPSHOT", "timeline-service-version":"3.0.0-alpha1-SNAPSHOT",
"timeline-service-build-version":"3.0.0-SNAPSHOT from fb0acd08e6f0b030d82eeb7cbfa5404376313e60 by sjlee source checksum be6cba0e42417d53be16459e1685e7", "timeline-service-build-version":"3.0.0-alpha1-SNAPSHOT from fb0acd08e6f0b030d82eeb7cbfa5404376313e60 by sjlee source checksum be6cba0e42417d53be16459e1685e7",
"timeline-service-version-built-on":"2016-04-11T23:15Z", "timeline-service-version-built-on":"2016-04-11T23:15Z",
"hadoop-version":"3.0.0-SNAPSHOT", "hadoop-version":"3.0.0-alpha1-SNAPSHOT",
"hadoop-build-version":"3.0.0-SNAPSHOT from fb0acd08e6f0b030d82eeb7cbfa5404376313e60 by sjlee source checksum ee968fd0aedcc7384230ee3ca216e790", "hadoop-build-version":"3.0.0-alpha1-SNAPSHOT from fb0acd08e6f0b030d82eeb7cbfa5404376313e60 by sjlee source checksum ee968fd0aedcc7384230ee3ca216e790",
"hadoop-version-built-on":"2016-04-11T23:14Z" "hadoop-version-built-on":"2016-04-11T23:14Z"
} }
@ -390,8 +433,9 @@ The following shows the supported queries on the REST API.
## <a name="REST_API_LIST_FLOWS"></a>Query Flows ## <a name="REST_API_LIST_FLOWS"></a>Query Flows
With Query Flows API, you can retrieve a list of active flows that had runs most recently. With Query Flows API, you can retrieve a list of active flows that had runs most recently.
If REST endpoint without cluster name is used, cluster specified by the configuration yarn.resourcemanager.cluster-id in yarn-site.xml is taken. If the REST endpoint without the cluster name is used, the cluster specified by the configuration
If none of the flows match the predicates, an empty list will be returned. `yarn.resourcemanager.cluster-id` in `yarn-site.xml` is taken. If none of the flows match the
predicates, an empty list will be returned.
### HTTP request: ### HTTP request:
@ -406,8 +450,8 @@ If none of the flows match the predicates, an empty list will be returned.
1. `limit` - If specified, defines the number of flows to return. The maximum possible value for limit 1. `limit` - If specified, defines the number of flows to return. The maximum possible value for limit
is maximum value of Long. If it is not specified or has a value less than 0, then limit will be is maximum value of Long. If it is not specified or has a value less than 0, then limit will be
considered as 100. considered as 100.
1. `dateRange` - If specified is given as "[startdate]-[enddate]"(i.e. start and end date separated by 1. `daterange` - If specified is given as "[startdate]-[enddate]"(i.e. start and end date separated by
"-") or single date. Dates are interpreted in yyyyMMdd format and are assumed to be in GMT. "-") or single date. Dates are interpreted in the yyyyMMdd format and are assumed to be in UTC.
If a single date is specified, all flows active on that date are returned. If both startdate and enddate If a single date is specified, all flows active on that date are returned. If both startdate and enddate
is given, all flows active between start and end date will be returned. If only startdate is given, flows is given, all flows active between start and end date will be returned. If only startdate is given, flows
active on and after startdate are returned. If only enddate is given, flows active on and before enddate active on and after startdate are returned. If only enddate is given, flows active on and before enddate
@ -480,10 +524,11 @@ If none of the flows match the predicates, an empty list will be returned.
## <a name="REST_API_LIST_FLOW_RUNS"></a>Query Flow Runs ## <a name="REST_API_LIST_FLOW_RUNS"></a>Query Flow Runs
With Query Flow Runs API, you can drill further down to get the runs (specific instances) of a given flow. With Query Flow Runs API, you can drill further down to get the runs (specific instances) of a
This returns the most recent runs that belong to the given flow. If REST endpoint without cluster name is used, given flow. This returns the most recent runs that belong to the given flow. If the REST
cluster specified by the configuration yarn.resourcemanager.cluster-id in yarn-site.xml is taken. endpoint without the cluster name is used, the cluster specified by the configuration
If none of the flow runs match the predicates, an empty list will be returned. `yarn.resourcemanager.cluster-id` in `yarn-site.xml` is taken. If none of the flow runs match the
predicates, an empty list will be returned.
### HTTP request: ### HTTP request:
@ -498,16 +543,16 @@ If none of the flow runs match the predicates, an empty list will be returned.
1. `limit` - If specified, defines the number of flows to return. The maximum possible value for limit 1. `limit` - If specified, defines the number of flows to return. The maximum possible value for limit
is maximum value of Long. If it is not specified or has a value less than 0, then limit will be is maximum value of Long. If it is not specified or has a value less than 0, then limit will be
considered as 100. considered as 100.
1. `createdTimeStart` - If specified, then only flow runs started after this timestamp are returned. 1. `createdtimestart` - If specified, then only flow runs started after this timestamp are returned.
1. `createdTimeEnd` - If specified, then only flow runs started before this timestamp are returned. 1. `createdtimeend` - If specified, then only flow runs started before this timestamp are returned.
1. `metricsToRetrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response. 1. `metricstoretrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response.
metricsToRetrieve can be an expression of the form :<br/> metricstoretrieve can be an expression of the form :<br/>
(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> (&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional for the This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional for the
simple expression. Alternatively, expressions can be of the form :<br/> simple expression. Alternatively, expressions can be of the form :<br/>
!(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> !(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics not matching any of the prefixes will be retrieved.<br/> This specifies a comma separated list of metric id prefixes. Only metrics not matching any of the prefixes will be retrieved.<br/>
If metricsToRetrieve is specified, metrics will be retrieved irrespective of whether `METRICS` is specified in fields query param If metricstoretrieve is specified, metrics will be retrieved irrespective of whether `METRICS` is specified in fields query param
or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded. or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `fields` - Specifies which fields to retrieve. For querying flow runs, only `ALL` or `METRICS` are valid fields. 1. `fields` - Specifies which fields to retrieve. For querying flow runs, only `ALL` or `METRICS` are valid fields.
Other fields will lead to HTTP 400 (Bad Request) response. If not specified, in response, id, type, createdtime and info fields Other fields will lead to HTTP 400 (Bad Request) response. If not specified, in response, id, type, createdtime and info fields
@ -559,9 +604,10 @@ If none of the flow runs match the predicates, an empty list will be returned.
## <a name="REST_API_LIST_FLOW_RUN"></a>Query Flow Run ## <a name="REST_API_LIST_FLOW_RUN"></a>Query Flow Run
With this API, you can query a specific flow run identified by cluster, user, flow name and run id.If REST endpoint without cluster name is With this API, you can query a specific flow run identified by cluster, user, flow name and run id.
used, cluster specified by the configuration yarn.resourcemanager.cluster-id in yarn-site.xml is taken. Metrics are returned by default while If the REST endpoint without the cluster name is used, the cluster specified by the configuration
querying individual flow run. `yarn.resourcemanager.cluster-id` in `yarn-site.xml` is taken. Metrics are returned by default
while querying individual flow runs.
### HTTP request: ### HTTP request:
@ -573,8 +619,8 @@ querying individual flow run.
### Query Parameters Supported: ### Query Parameters Supported:
1. `metricsToRetrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response. 1. `metricstoretrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response.
metricsToRetrieve can be an expression of the form :<br/> metricstoretrieve can be an expression of the form :<br/>
(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> (&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional for the This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional for the
simple expression. Alternatively, expressions can be of the form :<br/> simple expression. Alternatively, expressions can be of the form :<br/>
@ -627,9 +673,11 @@ querying individual flow run.
## <a name="REST_API_LIST_FLOW_APPS"></a>Query Apps for a flow ## <a name="REST_API_LIST_FLOW_APPS"></a>Query Apps for a flow
With this API, you can query all the YARN applications that are part of a specific flow.If REST endpoint without cluster name With this API, you can query all the YARN applications that are part of a specific flow. If the
is used, cluster specified by the configuration yarn.resourcemanager.cluster-id in yarn-site.xml is taken. If number of matching applications are REST endpoint without the cluster name is used, the cluster specified by the configuration
more than the limit, the most recent apps up to the limit will be returned. If none of the apps match the predicates, an empty list will be returned. `yarn.resourcemanager.cluster-id` in `yarn-site.xml` is taken. If the number of matching
applications are more than the limit, the most recent apps up to the limit will be returned. If
none of the apps match the predicates, an empty list will be returned.
### HTTP request: ### HTTP request:
@ -644,19 +692,19 @@ more than the limit, the most recent apps up to the limit will be returned. If n
1. `limit` - If specified, defines the number of applications to return. The maximum possible value for limit 1. `limit` - If specified, defines the number of applications to return. The maximum possible value for limit
is maximum value of Long. If it is not specified or has a value less than 0, then limit will be is maximum value of Long. If it is not specified or has a value less than 0, then limit will be
considered as 100. considered as 100.
1. `createdTimeStart` - If specified, then only applications created after this timestamp are returned. 1. `createdtimestart` - If specified, then only applications created after this timestamp are returned.
1. `createdTimeEnd` - If specified, then only applications created before this timestamp are returned. 1. `createdtimeend` - If specified, then only applications created before this timestamp are returned.
1. `relatesTo` - If specified, matched applications must relate to or not relate to given entities associated with a entity type. 1. `relatesto` - If specified, matched applications must relate to or not relate to given entities associated with a entity type.
relatesto is represented as an expression of the form :<br/> relatesto is represented as an expression of the form :<br/>
"(&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...,&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...) &lt;op&gt; !(&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...,&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...)".<br/> "(&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...,&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...) &lt;op&gt; !(&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...,&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...)".<br/>
If relatesTo expression has entity type - entity id(s) relations specified within enclosing brackets proceeding "!", this means apps with If relatesto expression has entity type - entity id(s) relations specified within enclosing brackets proceeding "!", this means apps with
these relations in its relatesTo field, will not be returned. For expressions or subexpressions without "!", all apps which have the specified these relations in its relatesto field, will not be returned. For expressions or subexpressions without "!", all apps which have the specified
relations in its relatesTo field, will be returned. "op" is a logical operator and can be either AND or OR. entity type can be followed by any number relations in its relatesto field, will be returned. "op" is a logical operator and can be either AND or OR. entity type can be followed by any number
of entity id(s). And we can combine any number of ANDs' and ORs' to create complex expressions. Brackets can be used to club expressions together.<br/> of entity id(s). And we can combine any number of ANDs' and ORs' to create complex expressions. Brackets can be used to club expressions together.<br/>
_For example_ : relatesTo can be "(((type1:id1:id2:id3,type3:id9) AND !(type2:id7:id8)) OR (type1:id4))".<br/> _For example_ : relatesto can be "(((type1:id1:id2:id3,type3:id9) AND !(type2:id7:id8)) OR (type1:id4))".<br/>
Please note that URL unsafe characters such as spaces will have to be suitably encoded. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `isRelatedTo` - If specified, matched applications must be related to or not related to given entities associated with a entity type. isRelatedTo is 1. `isrelatedto` - If specified, matched applications must be related to or not related to given entities associated with a entity type. isrelatedto is
represented in the same form as relatesTo. represented in the same form as relatesto.
1. `infofilters` - If specified, matched applications must have exact matches to the given info key and must be either equal or not equal to 1. `infofilters` - If specified, matched applications must have exact matches to the given info key and must be either equal or not equal to
given value. The info key is a string but value can be any object. infofilters are represented as an expression of the form :<br/> given value. The info key is a string but value can be any object. infofilters are represented as an expression of the form :<br/>
"(&lt;key&gt; &lt;compareop&gt; &lt;value&gt;) &lt;op&gt; (&lt;key&gt; &lt;compareop&gt; &lt;value&gt;)".<br/> "(&lt;key&gt; &lt;compareop&gt; &lt;value&gt;) &lt;op&gt; (&lt;key&gt; &lt;compareop&gt; &lt;value&gt;)".<br/>
@ -685,30 +733,30 @@ more than the limit, the most recent apps up to the limit will be returned. If n
We can combine any number of ANDs' and ORs' to create complex expressions. Brackets can be used to club expressions together.<br/> We can combine any number of ANDs' and ORs' to create complex expressions. Brackets can be used to club expressions together.<br/>
_For example_ : eventfilters can be "(((event1,event2) AND !(event4)) OR (event3,event7,event5))".<br/> _For example_ : eventfilters can be "(((event1,event2) AND !(event4)) OR (event3,event7,event5))".<br/>
Please note that URL unsafe characters such as spaces will have to be suitably encoded. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `metricsToRetrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response. 1. `metricstoretrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response.
metricsToRetrieve can be an expression of the form :<br/> metricstoretrieve can be an expression of the form :<br/>
(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> (&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional for This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional for
the simple expression. Alternatively, expressions can be of the form:<br/> the simple expression. Alternatively, expressions can be of the form:<br/>
!(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> !(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics not matching any of the prefixes will be retrieved.<br/> This specifies a comma separated list of metric id prefixes. Only metrics not matching any of the prefixes will be retrieved.<br/>
If metricsToRetrieve is specified, metrics will be retrieved irrespective of whether `METRICS` is specified in fields query param If metricstoretrieve is specified, metrics will be retrieved irrespective of whether `METRICS` is specified in fields query param
or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded. or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `confsToRetrieve` - If specified, defines which configs to retrieve or which ones not to retrieve and send back in response. 1. `confstoretrieve` - If specified, defines which configs to retrieve or which ones not to retrieve and send back in response.
confsToRetrieve can be an expression of the form :<br/> confstoretrieve can be an expression of the form :<br/>
(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/> (&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/>
This specifies a comma separated list of config name prefixes. Only configs matching any of the prefixes will be retrieved. Brackets are optional for This specifies a comma separated list of config name prefixes. Only configs matching any of the prefixes will be retrieved. Brackets are optional for
the simple expression. Alternatively, expressions can be of the form:<br/> the simple expression. Alternatively, expressions can be of the form:<br/>
!(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/> !(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/>
This specifies a comma separated list of config name prefixes. Only configs not matching any of the prefixes will be retrieved.<br/> This specifies a comma separated list of config name prefixes. Only configs not matching any of the prefixes will be retrieved.<br/>
If confsToRetrieve is specified, configs will be retrieved irrespective of whether `CONFIGS` is specified in fields query param If confstoretrieve is specified, configs will be retrieved irrespective of whether `CONFIGS` is specified in fields query param
or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded. or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `fields` - Specifies which fields to retrieve. Possible values for fields can be `EVENTS`, `INFO`, `CONFIGS`, `METRICS`, `RELATES_TO`, 1. `fields` - Specifies which fields to retrieve. Possible values for fields can be `EVENTS`, `INFO`, `CONFIGS`, `METRICS`, `RELATES_TO`,
`IS_RELATED_TO` and `ALL`. All fields will be retrieved if `ALL` is specified. Multiple fields can be specified as a comma-separated list. `IS_RELATED_TO` and `ALL`. All fields will be retrieved if `ALL` is specified. Multiple fields can be specified as a comma-separated list.
If fields is not specified, in response, app id, type (equivalent to YARN_APPLICATION), app createdtime and UID in info field will be returned. If fields is not specified, in response, app id, type (equivalent to YARN_APPLICATION), app createdtime and UID in info field will be returned.
1. `metricsLimit` - If specified, defines the number of metrics to return. Considered only if fields contains METRICS/ALL 1. `metricslimit` - If specified, defines the number of metrics to return. Considered only if fields contains METRICS/ALL
or metricsToRetrieve is specified. Ignored otherwise. The maximum possible value for metricsLimit can be maximum value of or metricstoretrieve is specified. Ignored otherwise. The maximum possible value for metricslimit can be maximum value of
Integer. If it is not specified or has a value less than 1, and metrics have to be retrieved, then metricsLimit will be Integer. If it is not specified or has a value less than 1, and metrics have to be retrieved, then metricslimit will be
considered as 1 i.e. latest single value of metric(s) will be returned. considered as 1 i.e. latest single value of metric(s) will be returned.
### Example JSON Response: ### Example JSON Response:
@ -750,9 +798,11 @@ more than the limit, the most recent apps up to the limit will be returned. If n
## <a name="REST_API_LIST_FLOWRUN_APPS"></a>Query Apps for a flow run ## <a name="REST_API_LIST_FLOWRUN_APPS"></a>Query Apps for a flow run
With this API, you can query all the YARN applications that are part of a specific flow run.If REST endpoint without cluster name With this API, you can query all the YARN applications that are part of a specific flow run. If the
is used, cluster specified by the configuration yarn.resourcemanager.cluster-id in yarn-site.xml is taken. If number of matching applications are REST endpoint without the cluster name is used, the cluster specified by the configuration
more than the limit, the most recent apps up to the limit will be returned. If none of the apps match the predicates, an empty list will be returned. `yarn.resourcemanager.cluster-id` in `yarn-site.xml` is taken. If number of matching applications
are more than the limit, the most recent apps up to the limit will be returned. If none of the apps
match the predicates, an empty list will be returned.
### HTTP request: ### HTTP request:
@ -767,19 +817,19 @@ more than the limit, the most recent apps up to the limit will be returned. If n
1. `limit` - If specified, defines the number of applications to return. The maximum possible value for limit 1. `limit` - If specified, defines the number of applications to return. The maximum possible value for limit
is maximum value of Long. If it is not specified or has a value less than 0, then limit will be is maximum value of Long. If it is not specified or has a value less than 0, then limit will be
considered as 100. considered as 100.
1. `createdTimeStart` - If specified, then only applications created after this timestamp are returned. 1. `createdtimestart` - If specified, then only applications created after this timestamp are returned.
1. `createdTimeEnd` - If specified, then only applications created before this timestamp are returned. 1. `createdtimeend` - If specified, then only applications created before this timestamp are returned.
1. `relatesTo` - If specified, matched applications must relate to or not relate to given entities associated with a entity type. 1. `relatesto` - If specified, matched applications must relate to or not relate to given entities associated with a entity type.
relatesto is represented as an expression of the form :<br/> relatesto is represented as an expression of the form :<br/>
"(&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...,&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...) &lt;op&gt; !(&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...,&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...)".<br/> "(&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...,&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...) &lt;op&gt; !(&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...,&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...)".<br/>
If relatesTo expression has entity type - entity id(s) relations specified within enclosing brackets proceeding "!", this means apps with If relatesto expression has entity type - entity id(s) relations specified within enclosing brackets proceeding "!", this means apps with
these relations in its relatesTo field, will not be returned. For expressions or subexpressions without "!", all apps which have the specified these relations in its relatesto field, will not be returned. For expressions or subexpressions without "!", all apps which have the specified
relations in its relatesTo field, will be returned. "op" is a logical operator and can be either AND or OR. entity type can be followed by any number relations in its relatesto field, will be returned. "op" is a logical operator and can be either AND or OR. entity type can be followed by any number
of entity id(s). And we can combine any number of ANDs' and ORs' to create complex expressions. Brackets can be used to club expressions together.<br/> of entity id(s). And we can combine any number of ANDs' and ORs' to create complex expressions. Brackets can be used to club expressions together.<br/>
_For example_ : relatesTo can be "(((type1:id1:id2:id3,type3:id9) AND !(type2:id7:id8)) OR (type1:id4))".<br/> _For example_ : relatesto can be "(((type1:id1:id2:id3,type3:id9) AND !(type2:id7:id8)) OR (type1:id4))".<br/>
Please note that URL unsafe characters such as spaces will have to be suitably encoded. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `isRelatedTo` - If specified, matched applications must be related to or not related to given entities associated with a entity type. isRelatedTo is 1. `isrelatedto` - If specified, matched applications must be related to or not related to given entities associated with a entity type. isrelatedto is
represented in the same form as relatesTo. represented in the same form as relatesto.
1. `infofilters` - If specified, matched applications must have exact matches to the given info key and must be either equal or not equal to 1. `infofilters` - If specified, matched applications must have exact matches to the given info key and must be either equal or not equal to
given value. The info key is a string but value can be any object. infofilters are represented as an expression of the form :<br/> given value. The info key is a string but value can be any object. infofilters are represented as an expression of the form :<br/>
"(&lt;key&gt; &lt;compareop&gt; &lt;value&gt;) &lt;op&gt; (&lt;key&gt; &lt;compareop&gt; &lt;value&gt;)".<br/> "(&lt;key&gt; &lt;compareop&gt; &lt;value&gt;) &lt;op&gt; (&lt;key&gt; &lt;compareop&gt; &lt;value&gt;)".<br/>
@ -808,30 +858,30 @@ more than the limit, the most recent apps up to the limit will be returned. If n
We can combine any number of ANDs' and ORs' to create complex expressions. Brackets can be used to club expressions together.<br/> We can combine any number of ANDs' and ORs' to create complex expressions. Brackets can be used to club expressions together.<br/>
_For example_ : eventfilters can be "(((event1,event2) AND !(event4)) OR (event3,event7,event5))".<br/> _For example_ : eventfilters can be "(((event1,event2) AND !(event4)) OR (event3,event7,event5))".<br/>
Please note that URL unsafe characters such as spaces will have to be suitably encoded. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `metricsToRetrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response. 1. `metricstoretrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response.
metricsToRetrieve can be an expression of the form :<br/> metricstoretrieve can be an expression of the form :<br/>
(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> (&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional for the This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional for the
simple expression. Alternatively, expressions can be of the form :<br/> simple expression. Alternatively, expressions can be of the form :<br/>
!(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> !(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics not matching any of the prefixes will be retrieved.<br/> This specifies a comma separated list of metric id prefixes. Only metrics not matching any of the prefixes will be retrieved.<br/>
If metricsToRetrieve is specified, metrics will be retrieved irrespective of whether `METRICS` is specified in fields query param If metricstoretrieve is specified, metrics will be retrieved irrespective of whether `METRICS` is specified in fields query param
or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded. or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `confsToRetrieve` - If specified, defines which configs to retrieve or which ones not to retrieve and send back in response. 1. `confstoretrieve` - If specified, defines which configs to retrieve or which ones not to retrieve and send back in response.
confsToRetrieve can be an expression of the form :<br/> confstoretrieve can be an expression of the form :<br/>
(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/> (&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/>
This specifies a comma separated list of config name prefixes. Only configs matching any of the prefixes will be retrieved. Brackets are optional for the This specifies a comma separated list of config name prefixes. Only configs matching any of the prefixes will be retrieved. Brackets are optional for the
simple expression. Alternatively, expressions can be of the form :<br/> simple expression. Alternatively, expressions can be of the form :<br/>
!(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/> !(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/>
This specifies a comma separated list of config name prefixes. Only configs not matching any of the prefixes will be retrieved.<br/> This specifies a comma separated list of config name prefixes. Only configs not matching any of the prefixes will be retrieved.<br/>
If confsToRetrieve is specified, configs will be retrieved irrespective of whether `CONFIGS` is specified in fields query param If confstoretrieve is specified, configs will be retrieved irrespective of whether `CONFIGS` is specified in fields query param
or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded. or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `fields` - Specifies which fields to retrieve. Possible values for fields can be `EVENTS`, `INFO`, `CONFIGS`, `METRICS`, `RELATES_TO`, 1. `fields` - Specifies which fields to retrieve. Possible values for fields can be `EVENTS`, `INFO`, `CONFIGS`, `METRICS`, `RELATES_TO`,
`IS_RELATED_TO` and `ALL`. All fields will be retrieved if `ALL` is specified. Multiple fields can be specified as a comma-separated list. `IS_RELATED_TO` and `ALL`. All fields will be retrieved if `ALL` is specified. Multiple fields can be specified as a comma-separated list.
If fields is not specified, in response, app id, type (equivalent to YARN_APPLICATION), app createdtime and UID in info field will be returned. If fields is not specified, in response, app id, type (equivalent to YARN_APPLICATION), app createdtime and UID in info field will be returned.
1. `metricsLimit` - If specified, defines the number of metrics to return. Considered only if fields contains METRICS/ALL 1. `metricslimit` - If specified, defines the number of metrics to return. Considered only if fields contains METRICS/ALL
or metricsToRetrieve is specified. Ignored otherwise. The maximum possible value for metricsLimit can be maximum value of or metricstoretrieve is specified. Ignored otherwise. The maximum possible value for metricslimit can be maximum value of
Integer. If it is not specified or has a value less than 1, and metrics have to be retrieved, then metricsLimit will be Integer. If it is not specified or has a value less than 1, and metrics have to be retrieved, then metricslimit will be
considered as 1 i.e. latest single value of metric(s) will be returned. considered as 1 i.e. latest single value of metric(s) will be returned.
### Example JSON Response: ### Example JSON Response:
@ -861,10 +911,12 @@ more than the limit, the most recent apps up to the limit will be returned. If n
## <a name="REST_API_LIST_APP"></a>Query app ## <a name="REST_API_LIST_APP"></a>Query app
With this API, you can query a single YARN application identified by cluster and application ID. If REST endpoint without cluster name With this API, you can query a single YARN application identified by the cluster and the
is used, cluster specified by the configuration yarn.resourcemanager.cluster-id in yarn-site.xml is taken. Flow context information i.e. application ID. If the REST endpoint without the cluster name is used, the cluster specified by the
user, flow name and run id are not mandatory but if specified in query param can preclude the need for an additional operation to fetch configuration `yarn.resourcemanager.cluster-id` in `yarn-site.xml` is taken. Flow context
flow context information based on cluster and app id. information i.e. user, flow name and run id are not mandatory but if specified in query param can
preclude the need for an additional operation to fetch flow context information based on cluster
and app id.
### HTTP request: ### HTTP request:
@ -882,30 +934,30 @@ flow context information based on cluster and app id.
If userid, flowname and flowrunid are not specified, we would have to fetch flow context information based on cluster and appid while executing the query. If userid, flowname and flowrunid are not specified, we would have to fetch flow context information based on cluster and appid while executing the query.
1. `flowrunid` - Only applications belonging to this flow run id will be returned. This query param must be specified along with userid and flowname query params, otherwise it will be ignored. 1. `flowrunid` - Only applications belonging to this flow run id will be returned. This query param must be specified along with userid and flowname query params, otherwise it will be ignored.
If userid, flowname and flowrunid are not specified, we would have to fetch flow context information based on cluster and appid while executing the query. If userid, flowname and flowrunid are not specified, we would have to fetch flow context information based on cluster and appid while executing the query.
1. `metricsToRetrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response. 1. `metricstoretrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response.
metricsToRetrieve can be an expression of the form :<br/> metricstoretrieve can be an expression of the form :<br/>
(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> (&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional for the This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional for the
simple expression. Alternatively, expressions can be of the form :<br/> simple expression. Alternatively, expressions can be of the form :<br/>
!(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> !(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics not matching any of the prefixes will be retrieved.<br/> This specifies a comma separated list of metric id prefixes. Only metrics not matching any of the prefixes will be retrieved.<br/>
If metricsToRetrieve is specified, metrics will be retrieved irrespective of whether `METRICS` is specified in fields query param If metricstoretrieve is specified, metrics will be retrieved irrespective of whether `METRICS` is specified in fields query param
or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded. or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `confsToRetrieve` - If specified, defines which configs to retrieve or which ones not to retrieve and send back in response. 1. `confstoretrieve` - If specified, defines which configs to retrieve or which ones not to retrieve and send back in response.
confsToRetrieve can be an expression of the form :<br/> confstoretrieve can be an expression of the form :<br/>
(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/> (&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/>
This specifies a comma separated list of config name prefixes. Only configs matching any of the prefixes will be retrieved. Brackets are optional for the This specifies a comma separated list of config name prefixes. Only configs matching any of the prefixes will be retrieved. Brackets are optional for the
simple expression. Alternatively, expressions can be of the form :<br/> simple expression. Alternatively, expressions can be of the form :<br/>
!(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/> !(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/>
This specifies a comma separated list of config name prefixes. Only configs not matching any of the prefixes will be retrieved.<br/> This specifies a comma separated list of config name prefixes. Only configs not matching any of the prefixes will be retrieved.<br/>
If confsToRetrieve is specified, configs will be retrieved irrespective of whether `CONFIGS` is specified in fields query param If confstoretrieve is specified, configs will be retrieved irrespective of whether `CONFIGS` is specified in fields query param
or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded. or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `fields` - Specifies which fields to retrieve. Possible values for fields can be `EVENTS`, `INFO`, `CONFIGS`, `METRICS`, `RELATES_TO`, 1. `fields` - Specifies which fields to retrieve. Possible values for fields can be `EVENTS`, `INFO`, `CONFIGS`, `METRICS`, `RELATES_TO`,
`IS_RELATED_TO` and `ALL`. All fields will be retrieved if `ALL` is specified. Multiple fields can be specified as a comma-separated list. `IS_RELATED_TO` and `ALL`. All fields will be retrieved if `ALL` is specified. Multiple fields can be specified as a comma-separated list.
If fields is not specified, in response, app id, type (equivalent to YARN_APPLICATION), app createdtime and UID in info field will be returned. If fields is not specified, in response, app id, type (equivalent to YARN_APPLICATION), app createdtime and UID in info field will be returned.
1. `metricsLimit` - If specified, defines the number of metrics to return. Considered only if fields contains METRICS/ALL 1. `metricslimit` - If specified, defines the number of metrics to return. Considered only if fields contains METRICS/ALL
or metricsToRetrieve is specified. Ignored otherwise. The maximum possible value for metricsLimit can be maximum value of or metricstoretrieve is specified. Ignored otherwise. The maximum possible value for metricslimit can be maximum value of
Integer. If it is not specified or has a value less than 1, and metrics have to be retrieved, then metricsLimit will be Integer. If it is not specified or has a value less than 1, and metrics have to be retrieved, then metricslimit will be
considered as 1 i.e. latest single value of metric(s) will be returned. considered as 1 i.e. latest single value of metric(s) will be returned.
### Example JSON Response: ### Example JSON Response:
@ -933,12 +985,16 @@ flow context information based on cluster and app id.
## <a name="REST_API_LIST_ENTITIES"></a>Query generic entities ## <a name="REST_API_LIST_ENTITIES"></a>Query generic entities
With this API, you can query generic entities identified by cluster ID, application ID and per-framework entity type. If REST endpoint without cluster name With this API, you can query generic entities identified by cluster ID, application ID and
is used, cluster specified by the configuration yarn.resourcemanager.cluster-id in yarn-site.xml is taken. Flow context information i.e. per-framework entity type. If the REST endpoint without the cluster name is used, the cluster
user, flow name and run id are not mandatory but if specified in query param can preclude the need for an additional operation to fetch specified by the configuration `yarn.resourcemanager.cluster-id` in `yarn-site.xml` is taken. Flow
flow context information based on cluster and app id. If number of matching entities are more than the limit, the most recent entities up to the limit context information i.e. user, flow name and run id are not mandatory but if specified in query
will be returned. This endpoint can be used to query containers, application attempts or any other generic entity which clients put into the backend. param can preclude the need for an additional operation to fetch flow context information based on
For instance, we can query containers by specifying entity type as `YARN_CONTAINER` and application attempts by specifying entity type as `YARN_APPLICATION_ATTEMPT`. cluster and app id. If number of matching entities are more than the limit, the most recent
entities up to the limit will be returned. This endpoint can be used to query containers,
application attempts or any other generic entity which clients put into the backend.
For instance, we can query containers by specifying entity type as `YARN_CONTAINER` and application
attempts by specifying entity type as `YARN_APPLICATION_ATTEMPT`.
If none of the entities match the predicates, an empty list will be returned. If none of the entities match the predicates, an empty list will be returned.
### HTTP request: ### HTTP request:
@ -959,19 +1015,19 @@ If none of the entities match the predicates, an empty list will be returned.
If userid, flowname and flowrunid are not specified, we would have to fetch flow context information based on cluster and appid while executing the query. If userid, flowname and flowrunid are not specified, we would have to fetch flow context information based on cluster and appid while executing the query.
1. `limit` - If specified, defines the number of entities to return. The maximum possible value for limit is maximum value of Long. If it is not specified 1. `limit` - If specified, defines the number of entities to return. The maximum possible value for limit is maximum value of Long. If it is not specified
or has a value less than 0, then limit will be considered as 100. or has a value less than 0, then limit will be considered as 100.
1. `createdTimeStart` - If specified, then only entities created after this timestamp are returned. 1. `createdtimestart` - If specified, then only entities created after this timestamp are returned.
1. `createdTimeEnd` - If specified, then only entities created before this timestamp are returned. 1. `createdtimeend` - If specified, then only entities created before this timestamp are returned.
1. `relatesTo` - If specified, matched entities must relate to or not relate to given entities associated with a entity type. 1. `relatesto` - If specified, matched entities must relate to or not relate to given entities associated with a entity type.
relatesto is represented as an expression of the form :<br/> relatesto is represented as an expression of the form :<br/>
"(&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...,&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...) &lt;op&gt; !(&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...,&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...)".<br/> "(&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...,&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...) &lt;op&gt; !(&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...,&lt;entitytype&gt;:&lt;entityid&gt;:&lt;entityid&gt;...)".<br/>
If relatesTo expression has entity type - entity id(s) relations specified within enclosing brackets proceeding "!", this means entities with If relatesto expression has entity type - entity id(s) relations specified within enclosing brackets proceeding "!", this means entities with
these relations in its relatesTo field, will not be returned. For expressions or subexpressions without "!", all entities which have the specified these relations in its relatesto field, will not be returned. For expressions or subexpressions without "!", all entities which have the specified
relations in its relatesTo field, will be returned. "op" is a logical operator and can be either AND or OR. entity type can be followed by any number relations in its relatesto field, will be returned. "op" is a logical operator and can be either AND or OR. entity type can be followed by any number
of entity id(s). And we can combine any number of ANDs' and ORs' to create complex expressions. Brackets can be used to club expressions together.<br/> of entity id(s). And we can combine any number of ANDs' and ORs' to create complex expressions. Brackets can be used to club expressions together.<br/>
_For example_ : relatesTo can be "(((type1:id1:id2:id3,type3:id9) AND !(type2:id7:id8)) OR (type1:id4))".<br/> _For example_ : relatesto can be "(((type1:id1:id2:id3,type3:id9) AND !(type2:id7:id8)) OR (type1:id4))".<br/>
Please note that URL unsafe characters such as spaces will have to be suitably encoded. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `isRelatedTo` - If specified, matched entities must be related to or not related to given entities associated with a entity type. isRelatedTo is 1. `isrelatedto` - If specified, matched entities must be related to or not related to given entities associated with a entity type. isrelatedto is
represented in the same form as relatesTo. represented in the same form as relatesto.
1. `infofilters` - If specified, matched entities must have exact matches to the given info key and must be either equal or not equal to 1. `infofilters` - If specified, matched entities must have exact matches to the given info key and must be either equal or not equal to
given value. The info key is a string but value can be any object. infofilters are represented as an expression of the form :<br/> given value. The info key is a string but value can be any object. infofilters are represented as an expression of the form :<br/>
"(&lt;key&gt; &lt;compareop&gt; &lt;value&gt;) &lt;op&gt; (&lt;key&gt; &lt;compareop&gt; &lt;value&gt;)".<br/> "(&lt;key&gt; &lt;compareop&gt; &lt;value&gt;) &lt;op&gt; (&lt;key&gt; &lt;compareop&gt; &lt;value&gt;)".<br/>
@ -1000,30 +1056,30 @@ If none of the entities match the predicates, an empty list will be returned.
We can combine any number of ANDs' and ORs' to create complex expressions. Brackets can be used to club expressions together.<br/> We can combine any number of ANDs' and ORs' to create complex expressions. Brackets can be used to club expressions together.<br/>
_For example_ : eventfilters can be "(((event1,event2) AND !(event4)) OR (event3,event7,event5))".<br/> _For example_ : eventfilters can be "(((event1,event2) AND !(event4)) OR (event3,event7,event5))".<br/>
Please note that URL unsafe characters such as spaces will have to be suitably encoded. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `metricsToRetrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response. 1. `metricstoretrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response.
metricsToRetrieve can be an expression of the form :<br/> metricstoretrieve can be an expression of the form :<br/>
(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> (&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional for This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional for
the simple expression. Alternatively, expressions can be of the form:<br/> the simple expression. Alternatively, expressions can be of the form:<br/>
!(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> !(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics not matching any of the prefixes will be retrieved.<br/> This specifies a comma separated list of metric id prefixes. Only metrics not matching any of the prefixes will be retrieved.<br/>
If metricsToRetrieve is specified, metrics will be retrieved irrespective of whether `METRICS` is specified in fields query param If metricstoretrieve is specified, metrics will be retrieved irrespective of whether `METRICS` is specified in fields query param
or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded. or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `confsToRetrieve` - If specified, defines which configs to retrieve or which ones not to retrieve and send back in response. 1. `confstoretrieve` - If specified, defines which configs to retrieve or which ones not to retrieve and send back in response.
confsToRetrieve can be an expression of the form :<br/> confstoretrieve can be an expression of the form :<br/>
(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/> (&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/>
This specifies a comma separated list of config name prefixes. Only configs matching any of the prefixes will be retrieved. Brackets are optional for This specifies a comma separated list of config name prefixes. Only configs matching any of the prefixes will be retrieved. Brackets are optional for
the simple expression. Alternatively, expressions can be of the form:<br/> the simple expression. Alternatively, expressions can be of the form:<br/>
!(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/> !(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/>
This specifies a comma separated list of config name prefixes. Only configs not matching any of the prefixes will be retrieved.<br/> This specifies a comma separated list of config name prefixes. Only configs not matching any of the prefixes will be retrieved.<br/>
If confsToRetrieve is specified, configs will be retrieved irrespective of whether `CONFIGS` is specified in fields query param If confstoretrieve is specified, configs will be retrieved irrespective of whether `CONFIGS` is specified in fields query param
or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded. or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `fields` - Specifies which fields to retrieve. Possible values for fields can be `EVENTS`, `INFO`, `CONFIGS`, `METRICS`, `RELATES_TO`, 1. `fields` - Specifies which fields to retrieve. Possible values for fields can be `EVENTS`, `INFO`, `CONFIGS`, `METRICS`, `RELATES_TO`,
`IS_RELATED_TO` and `ALL`. All fields will be retrieved if `ALL` is specified. Multiple fields can be specified as a comma-separated list. `IS_RELATED_TO` and `ALL`. All fields will be retrieved if `ALL` is specified. Multiple fields can be specified as a comma-separated list.
If fields is not specified, in response, entity id, entity type, createdtime and UID in info field will be returned. If fields is not specified, in response, entity id, entity type, createdtime and UID in info field will be returned.
1. `metricsLimit` - If specified, defines the number of metrics to return. Considered only if fields contains METRICS/ALL 1. `metricslimit` - If specified, defines the number of metrics to return. Considered only if fields contains METRICS/ALL
or metricsToRetrieve is specified. Ignored otherwise. The maximum possible value for metricsLimit can be maximum value of or metricstoretrieve is specified. Ignored otherwise. The maximum possible value for metricslimit can be maximum value of
Integer. If it is not specified or has a value less than 1, and metrics have to be retrieved, then metricsLimit will be Integer. If it is not specified or has a value less than 1, and metrics have to be retrieved, then metricslimit will be
considered as 1 i.e. latest single value of metric(s) will be returned. considered as 1 i.e. latest single value of metric(s) will be returned.
### Example JSON Response: ### Example JSON Response:
@ -1066,12 +1122,16 @@ If none of the entities match the predicates, an empty list will be returned.
## <a name="REST_API_LIST_ENTITY"></a>Query generic entity ## <a name="REST_API_LIST_ENTITY"></a>Query generic entity
With this API, you can query a specific generic entity identified by cluster ID, application ID, per-framework entity type and entity ID. If REST endpoint without With this API, you can query a specific generic entity identified by cluster ID, application ID,
cluster name is used, cluster specified by the configuration yarn.resourcemanager.cluster-id in yarn-site.xml is taken. Flow context information i.e. per-framework entity type and entity ID. If the REST endpoint without the cluster name is used, the
user, flow name and run id are not mandatory but if specified in query param can preclude the need for an additional operation to fetch cluster specified by the configuration `yarn.resourcemanager.cluster-id` in `yarn-site.xml` is
flow context information based on cluster and app id. This endpoint can be used to query a single container, application attempt or any other generic entity which taken. Flow context information i.e. user, flow name and run id are not mandatory but if specified
clients put into the backend. For instance, we can query a specific YARN container by specifying entity type as `YARN_CONTAINER` and giving entity ID as container ID. in query param can preclude the need for an additional operation to fetch flow context information
Similarly, application attempt can be queried by specifying entity type as `YARN_APPLICATION_ATTEMPT` and entity ID being the application attempt ID. based on cluster and app id. This endpoint can be used to query a single container, application
attempt or any other generic entity which clients put into the backend. For instance, we can query
a specific YARN container by specifying entity type as `YARN_CONTAINER` and giving entity ID as
container ID. Similarly, application attempt can be queried by specifying entity type as
`YARN_APPLICATION_ATTEMPT` and entity ID being the application attempt ID.
### HTTP request: ### HTTP request:
@ -1089,30 +1149,30 @@ Similarly, application attempt can be queried by specifying entity type as `YARN
If userid, flowname and flowrunid are not specified, we would have to fetch flow context information based on cluster and appid while executing the query. If userid, flowname and flowrunid are not specified, we would have to fetch flow context information based on cluster and appid while executing the query.
1. `flowrunid` - If specified, entity must belong to this flow run id. This query param must be specified along with userid and flowname query params, otherwise it will be ignored. 1. `flowrunid` - If specified, entity must belong to this flow run id. This query param must be specified along with userid and flowname query params, otherwise it will be ignored.
If userid, flowname and flowrunid are not specified, we would have to fetch flow context information based on cluster and appid while executing the query. If userid, flowname and flowrunid are not specified, we would have to fetch flow context information based on cluster and appid while executing the query.
1. `metricsToRetrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response. 1. `metricstoretrieve` - If specified, defines which metrics to retrieve or which ones not to retrieve and send back in response.
metricsToRetrieve can be an expression of the form :<br/> metricstoretrieve can be an expression of the form :<br/>
(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> (&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional This specifies a comma separated list of metric id prefixes. Only metrics matching any of the prefixes will be retrieved. Brackets are optional
for the simple expression. Alternatively, expressions can be of the form:<br/> for the simple expression. Alternatively, expressions can be of the form:<br/>
!(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/> !(&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;,&lt;metricprefix&gt;...)<br/>
This specifies a comma separated list of metric id prefixes. Only metrics not matching any of the prefixes will be retrieved.<br/> This specifies a comma separated list of metric id prefixes. Only metrics not matching any of the prefixes will be retrieved.<br/>
If metricsToRetrieve is specified, metrics will be retrieved irrespective of whether `METRICS` is specified in fields query param If metricstoretrieve is specified, metrics will be retrieved irrespective of whether `METRICS` is specified in fields query param
or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded. or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `confsToRetrieve` - If specified, defines which configs to retrieve or which ones not to retrieve and send back in response. 1. `confstoretrieve` - If specified, defines which configs to retrieve or which ones not to retrieve and send back in response.
confsToRetrieve can be an expression of the form :<br/> confstoretrieve can be an expression of the form :<br/>
(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/> (&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/>
This specifies a comma separated list of config name prefixes. Only configs matching any of the prefixes will be retrieved. Brackets are optional for This specifies a comma separated list of config name prefixes. Only configs matching any of the prefixes will be retrieved. Brackets are optional for
the simple expression. Alternatively, expressions can be of the form :<br/> the simple expression. Alternatively, expressions can be of the form :<br/>
!(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/> !(&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;,&lt;config\_name\_prefix&gt;...)<br/>
This specifies a comma separated list of config name prefixes. Only configs not matching any of the prefixes will be retrieved.<br/> This specifies a comma separated list of config name prefixes. Only configs not matching any of the prefixes will be retrieved.<br/>
If confsToRetrieve is specified, configs will be retrieved irrespective of whether `CONFIGS` is specified in fields query param If confstoretrieve is specified, configs will be retrieved irrespective of whether `CONFIGS` is specified in fields query param
or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded. or not. Please note that URL unsafe characters such as spaces will have to be suitably encoded.
1. `fields` - Specifies which fields to retrieve. Possible values for fields can be `EVENTS`, `INFO`, `CONFIGS`, `METRICS`, `RELATES_TO`, 1. `fields` - Specifies which fields to retrieve. Possible values for fields can be `EVENTS`, `INFO`, `CONFIGS`, `METRICS`, `RELATES_TO`,
`IS_RELATED_TO` and `ALL`. All fields will be retrieved if `ALL` is specified. Multiple fields can be specified as a comma-separated list. `IS_RELATED_TO` and `ALL`. All fields will be retrieved if `ALL` is specified. Multiple fields can be specified as a comma-separated list.
If fields is not specified, in response, entity id, entity type, createdtime and UID in info field will be returned. If fields is not specified, in response, entity id, entity type, createdtime and UID in info field will be returned.
1. `metricsLimit` - If specified, defines the number of metrics to return. Considered only if fields contains METRICS/ALL 1. `metricslimit` - If specified, defines the number of metrics to return. Considered only if fields contains METRICS/ALL
or metricsToRetrieve is specified. Ignored otherwise. The maximum possible value for metricsLimit can be maximum value of or metricstoretrieve is specified. Ignored otherwise. The maximum possible value for metricslimit can be maximum value of
Integer. If it is not specified or has a value less than 1, and metrics have to be retrieved, then metricsLimit will be Integer. If it is not specified or has a value less than 1, and metrics have to be retrieved, then metricslimit will be
considered as 1 i.e. latest single value of metric(s) will be returned. considered as 1 i.e. latest single value of metric(s) will be returned.
### Example JSON Response: ### Example JSON Response:

BIN
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/resources/images/flow_hierarchy.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 41 KiB