mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-03-09 14:34:43 +00:00
791b9b147c
This introduces a new GetRollupIndexCaps API which allows the user to retrieve rollup capabilities of a specific rollup index (or index pattern). This is distinct from the existing RollupCaps endpoint. - Multiple jobs can be stored in multiple indices and point to a single target data index pattern (logstash-*). The existing API finds capabilities/config of all jobs matching that data index pattern. - One rollup index can hold data from multiple jobs, targeting multiple data index patterns. This new API finds the capabilities based on the concrete rollup indices.
162 lines
4.5 KiB
Plaintext
162 lines
4.5 KiB
Plaintext
[role="xpack"]
|
|
[[rollup-get-rollup-index-caps]]
|
|
=== Get Rollup Index Capabilities
|
|
++++
|
|
<titleabbrev>Get Rollup Index Caps</titleabbrev>
|
|
++++
|
|
|
|
experimental[]
|
|
|
|
This API returns the rollup capabilities of all jobs inside of a rollup index (e.g. the index where rollup data is stored).
|
|
A single rollup index may store the data for multiple rollup jobs, and may have a variety of capabilities depending on those jobs.
|
|
|
|
This API will allow you to determine:
|
|
|
|
1. What jobs are stored in an index (or indices specified via a pattern)?
|
|
2. What target indices were rolled up, what fields were used in those rollups and what aggregations can be performed on each job?
|
|
|
|
==== Request
|
|
|
|
`GET {index}/_xpack/rollup/data`
|
|
|
|
//===== Description
|
|
|
|
==== Path Parameters
|
|
|
|
`index`::
|
|
(string) Index or index-pattern of concrete rollup indices to check for capabilities.
|
|
|
|
|
|
|
|
==== Request Body
|
|
|
|
There is no request body for the Get Jobs API.
|
|
|
|
==== Authorization
|
|
|
|
You must have `monitor`, `monitor_rollup`, `manage` or `manage_rollup` cluster privileges to use this API.
|
|
For more information, see
|
|
{xpack-ref}/security-privileges.html[Security Privileges].
|
|
|
|
==== Examples
|
|
|
|
Imagine we have an index named `sensor-1` full of raw data. We know that the data will grow over time, so there
|
|
will be a `sensor-2`, `sensor-3`, etc. Let's create a Rollup job, which stores it's data in `sensor_rollup`:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT _xpack/rollup/job/sensor
|
|
{
|
|
"index_pattern": "sensor-*",
|
|
"rollup_index": "sensor_rollup",
|
|
"cron": "*/30 * * * * ?",
|
|
"page_size" :1000,
|
|
"groups" : {
|
|
"date_histogram": {
|
|
"field": "timestamp",
|
|
"interval": "1h",
|
|
"delay": "7d"
|
|
},
|
|
"terms": {
|
|
"fields": ["node"]
|
|
}
|
|
},
|
|
"metrics": [
|
|
{
|
|
"field": "temperature",
|
|
"metrics": ["min", "max", "sum"]
|
|
},
|
|
{
|
|
"field": "voltage",
|
|
"metrics": ["avg"]
|
|
}
|
|
]
|
|
}
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
// TEST[setup:sensor_index]
|
|
|
|
If at a later date, we'd like to determine what jobs and capabilities were stored in the `sensor_rollup` index, we can use the Get Rollup
|
|
Index API:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /sensor_rollup/_xpack/rollup/data
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
// TEST[continued]
|
|
|
|
Note how we are requesting the concrete rollup index name (`sensor_rollup`) as the first part of the URL.
|
|
This will yield the following response:
|
|
|
|
[source,js]
|
|
----
|
|
{
|
|
"sensor_rollup" : {
|
|
"rollup_jobs" : [
|
|
{
|
|
"job_id" : "sensor",
|
|
"rollup_index" : "sensor_rollup",
|
|
"index_pattern" : "sensor-*",
|
|
"fields" : {
|
|
"node" : [
|
|
{
|
|
"agg" : "terms"
|
|
}
|
|
],
|
|
"temperature" : [
|
|
{
|
|
"agg" : "min"
|
|
},
|
|
{
|
|
"agg" : "max"
|
|
},
|
|
{
|
|
"agg" : "sum"
|
|
}
|
|
],
|
|
"timestamp" : [
|
|
{
|
|
"agg" : "date_histogram",
|
|
"time_zone" : "UTC",
|
|
"interval" : "1h",
|
|
"delay": "7d"
|
|
}
|
|
],
|
|
"voltage" : [
|
|
{
|
|
"agg" : "avg"
|
|
}
|
|
]
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
----
|
|
// TESTRESPONSE
|
|
|
|
|
|
The response that is returned contains information that is similar to the original Rollup configuration, but formatted
|
|
differently. First, there are some house-keeping details: the Rollup job's ID, the index that holds the rolled data,
|
|
the index pattern that the job was targeting.
|
|
|
|
Next it shows a list of fields that contain data eligible for rollup searches. Here we see four fields: `node`, `temperature`,
|
|
`timestamp` and `voltage`. Each of these fields list the aggregations that are possible. For example, you can use a min, max
|
|
or sum aggregation on the `temperature` field, but only a `date_histogram` on `timestamp`.
|
|
|
|
Note that the `rollup_jobs` element is an array; there can be multiple, independent jobs configured for a single index
|
|
or index pattern. Each of these jobs may have different configurations, so the API returns a list of all the various
|
|
configurations available.
|
|
|
|
|
|
Like other APIs that interact with indices, you can specify index patterns instead of explicit indices:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /*_rollup/_xpack/rollup/data
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
// TEST[continued]
|
|
|