honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[7.x] [DOCS] Update get data stream API response (#59197) (#59221) 

Updates docs and snippets for changes made to the get data stream API with
PR #59128.
This commit is contained in:
James Rodewig 2020-07-08 14:04:14 -04:00 committed by GitHub
parent 6a0f7411e2
commit d2c5a4c5e9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 220 additions and 119 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

52
docs/reference/data-streams/change-mappings-and-settings.asciidoc
View File

@ -84,6 +84,8 @@ PUT /_index_template/new_logs_data_stream
PUT /_data_stream/logs PUT /_data_stream/logs
POST /logs/_rollover/
PUT /_data_stream/new_logs PUT /_data_stream/new_logs
---- ----
// TESTSETUP // TESTSETUP
@ -565,33 +567,43 @@ data stream, including a list of its backing indices.
---- ----
GET /_data_stream/logs GET /_data_stream/logs
---- ----
// TEST[skip: shard failures]
The API returns the following response. Note the `indices` property contains an The API returns the following response. Note the `indices` property contains an
array of the stream's current backing indices. The oldest backing index, array of the stream's current backing indices. The first item in the array
`.ds-logs-000001`, is the first item in the array. contains information about the stream's oldest backing index, `.ds-logs-000001`.
[source,console-result] [source,console-result]
---- ----
[ {
{ "data_streams": [
"name": "logs", {
"timestamp_field": "@timestamp", "name": "logs",
"indices": [ "timestamp_field": {
{ "name": "@timestamp"
"index_name": ".ds-logs-000001",
"index_uuid": "DXAE-xcCQTKF93bMm9iawA"
}, },
{ "indices": [
"index_name": ".ds-logs-000002", {
"index_uuid": "Wzxq0VhsQKyPxHhaK3WYAg" "index_name": ".ds-logs-000001", <1>
} "index_uuid": "Gpdiyq8sRuK9WuthvAdFbw"
], },
"generation": 2 {
} "index_name": ".ds-logs-000002",
] "index_uuid": "_eEfRrFHS9OyhqWntkgHAQ"
}
],
"generation": 2,
"status": "GREEN",
"template": "logs_data_stream"
}
]
}
---- ----
// TESTRESPONSE[skip:unable to assert responses with top level array] // TESTRESPONSE[s/"index_uuid": "Gpdiyq8sRuK9WuthvAdFbw"/"index_uuid": $body.data_streams.0.indices.0.index_uuid/]
// TESTRESPONSE[s/"index_uuid": "_eEfRrFHS9OyhqWntkgHAQ"/"index_uuid": $body.data_streams.0.indices.1.index_uuid/]
// TESTRESPONSE[s/"status": "GREEN"/"status": "YELLOW"/]
<1> First item in the `indices` array for the `logs` data stream. This item
contains information about the stream's oldest backing index, `.ds-logs-000001`.
The following <<docs-reindex,reindex API>> request copies documents from The following <<docs-reindex,reindex API>> request copies documents from
`.ds-logs-000001` to the `new_logs` data stream. Note the request's `op_type` is `.ds-logs-000001` to the `new_logs` data stream. Note the request's `op_type` is

115
docs/reference/data-streams/set-up-a-data-stream.asciidoc
View File

@ -278,20 +278,6 @@ PUT /_data_stream/logs_alt
==== ====
-- --
////
[source,console]
----
DELETE /_data_stream/logs
DELETE /_data_stream/logs_alt
DELETE /_index_template/logs_data_stream
DELETE /_ilm/policy/logs_policy
----
// TEST[continued]
////
[discrete] [discrete]
[[get-info-about-a-data-stream]] [[get-info-about-a-data-stream]]
=== Get information about a data stream === Get information about a data stream
@ -303,47 +289,70 @@ information about one or more data streams, including:
* The current backing indices, which is returned as an array. The last item in * The current backing indices, which is returned as an array. The last item in
the array contains information about the stream's current write index. the array contains information about the stream's current write index.
* The current generation * The current generation
* The data stream's health status
* The index template used to create the stream's backing indices
* The current {ilm-init} lifecycle policy in the stream's matching index
template
This is also handy way to verify that a recently created data stream exists. This is also handy way to verify that a recently created data stream exists.
.*Example* .*Example*
[%collapsible] [%collapsible]
==== ====
The following get data stream API request retrieves information about any data The following get data stream API request retrieves information about the
streams starting with `logs`. `logs` data stream.
////
[source,console]
----
POST /logs/_rollover/
----
// TEST[continued]
////
[source,console] [source,console]
---- ----
GET /_data_stream/logs* GET /_data_stream/logs
---- ----
// TEST[skip: shard failures] // TEST[continued]
The API returns the following response, which includes information about the The API returns the following response. Note the `indices` property contains an
`logs` data stream. Note the `indices` property contains an array of the array of the stream's current backing indices. The last item in this array
stream's current backing indices. The last item in this array contains contains information about the stream's write index, `.ds-logs-000002`.
information for the `logs` stream's write index, `.ds-logs-000002`.
[source,console-result] [source,console-result]
---- ----
[ {
{ "data_streams": [
"name": "logs", {
"timestamp_field": "@timestamp", "name": "logs",
"indices": [ "timestamp_field": {
{ "name": "@timestamp"
"index_name": ".ds-logs-000001",
"index_uuid": "DXAE-xcCQTKF93bMm9iawA"
}, },
{ "indices": [
"index_name": ".ds-logs-000002", {
"index_uuid": "Wzxq0VhsQKyPxHhaK3WYAg" "index_name": ".ds-logs-000001",
} "index_uuid": "krR78LfvTOe6gr5dj2_1xQ"
], },
"generation": 2 {
} "index_name": ".ds-logs-000002", <1>
] "index_uuid": "C6LWyNJHQWmA08aQGvqRkA"
}
],
"generation": 2,
"status": "GREEN",
"template": "logs_data_stream",
"ilm_policy": "logs_policy"
}
]
}
---- ----
// TESTRESPONSE[skip:unable to assert responses with top level array] // TESTRESPONSE[s/"index_uuid": "krR78LfvTOe6gr5dj2_1xQ"/"index_uuid": $body.data_streams.0.indices.0.index_uuid/]
// TESTRESPONSE[s/"index_uuid": "C6LWyNJHQWmA08aQGvqRkA"/"index_uuid": $body.data_streams.0.indices.1.index_uuid/]
// TESTRESPONSE[s/"status": "GREEN"/"status": "YELLOW"/]
<1> Last item in the `indices` array for the `logs` data stream. This item
contains information about the stream's current write index, `.ds-logs-000002`.
==== ====
[discrete] [discrete]
@ -366,30 +375,6 @@ a data stream and its backing indices.
The following delete data stream API request deletes the `logs` data stream. This The following delete data stream API request deletes the `logs` data stream. This
request also deletes the stream's backing indices and any data they contain. request also deletes the stream's backing indices and any data they contain.
////
[source,console]
----
PUT /_index_template/logs_data_stream
{
"index_patterns": [ "logs*" ],
"data_stream": {
"timestamp_field": "@timestamp"
},
"template": {
"mappings": {
"properties": {
"@timestamp": {
"type": "date"
}
}
}
}
}
PUT /_data_stream/logs
----
////
[source,console] [source,console]
---- ----
DELETE /_data_stream/logs DELETE /_data_stream/logs
@ -400,7 +385,9 @@ DELETE /_data_stream/logs
//// ////
[source,console] [source,console]
---- ----
DELETE /_index_template/logs_data_stream DELETE /_data_stream/*
DELETE /_index_template/*
DELETE /_ilm/policy/logs_policy
---- ----
// TEST[continued] // TEST[continued]
//// ////

172
docs/reference/indices/get-data-stream.asciidoc
View File

@ -10,9 +10,33 @@ See <<get-info-about-a-data-stream>>.
//// ////
[source,console] [source,console]
---- ----
PUT _index_template/template PUT /_ilm/policy/my-lifecycle-policy
{ {
"index_patterns": ["my-data-stream*"], "policy": {
"phases": {
"hot": {
"actions": {
"rollover": {
"max_size": "25GB"
}
}
},
"delete": {
"min_age": "30d",
"actions": {
"delete": {}
}
}
}
}
}
PUT /_index_template/my-index-template
{
"index_patterns": [ "my-data-stream*" ],
"data_stream": {
"timestamp_field": "@timestamp"
},
"template": { "template": {
"mappings": { "mappings": {
"properties": { "properties": {
@ -20,14 +44,18 @@ PUT _index_template/template
"type": "date" "type": "date"
} }
} }
},
"settings": {
"index.lifecycle.name": "my-lifecycle-policy"
} }
},
"data_stream": {
"timestamp_field": "@timestamp"
} }
} }
PUT /_data_stream/my-data-stream PUT /_data_stream/my-data-stream
POST /my-data-stream/_rollover
PUT /_data_stream/my-data-stream_two
---- ----
// TESTSETUP // TESTSETUP
//// ////
@ -35,8 +63,9 @@ PUT /_data_stream/my-data-stream
//// ////
[source,console] [source,console]
---- ----
DELETE /_data_stream/my-data-stream DELETE /_data_stream/*
DELETE /_index_template/template DELETE /_index_template/*
DELETE /_ilm/policy/my-lifecycle-policy
---- ----
// TEARDOWN // TEARDOWN
//// ////
@ -45,7 +74,6 @@ DELETE /_index_template/template
---- ----
GET /_data_stream/my-data-stream GET /_data_stream/my-data-stream
---- ----
// TEST[skip_shard_failures]
[[get-data-stream-api-request]] [[get-data-stream-api-request]]
==== {api-request-title} ==== {api-request-title}
@ -64,14 +92,29 @@ Wildcard (`*`) expressions are supported.
[[get-data-stream-api-response-body]] [[get-data-stream-api-response-body]]
==== {api-response-body-title} ==== {api-response-body-title}
`data_streams`::
(array of objects)
Contains information about retrieved data streams.
+
.Properties of objects in `data_streams`
[%collapsible%open]
====
`name`:: `name`::
(string) (string)
Name of the data stream. Name of the data stream.
`timestamp_field`:: `timestamp_field`::
(object)
Contains information about the data stream's timestamp field.
+
.Properties of `timestamp_field`
[%collapsible%open]
=====
`name`::
(string) (string)
Name of the data stream's timestamp field. This field must be included in every Name of the data stream's timestamp field. This field must be included in every
document indexed to the data stream. document indexed to the data stream.
=====
`indices`:: `indices`::
(array of objects) (array of objects)
@ -83,7 +126,7 @@ The last item in this array contains information about the stream's current
+ +
.Properties of `indices` objects .Properties of `indices` objects
[%collapsible%open] [%collapsible%open]
==== =====
`index_name`:: `index_name`::
(string) (string)
Name of the backing index. For naming conventions, see Name of the backing index. For naming conventions, see
@ -92,7 +135,7 @@ Name of the backing index. For naming conventions, see
`index_uuid`:: `index_uuid`::
(string) (string)
Universally unique identifier (UUID) for the index. Universally unique identifier (UUID) for the index.
==== =====
`generation`:: `generation`::
(integer) (integer)
@ -100,6 +143,47 @@ Current <<data-streams-generation,generation>> for the data stream. This number
acts as a cumulative count of the stream's backing indices, including acts as a cumulative count of the stream's backing indices, including
deleted indices. deleted indices.
`status`::
(string)
<<cluster-health,Health status>> of the data stream.
+
This health status is based on the state of the primary and replica shards of
the stream's backing indices.
+
.Values for `status`
[%collapsible%open]
=====
`green`:::
All shards are assigned.
`yellow`:::
All primary shards are assigned, but one or more replica shards are
unassigned.
`red`:::
One or more primary shards are unassigned, so some data is unavailable.
=====
`template`::
(string)
Name of the index template used to create the data stream's backing indices.
+
The template's index pattern must match the name of this data stream. See
<<create-a-data-stream-template>>.
`ilm_policy`::
(string)
Name of the current {ilm-init} lifecycle policy in the stream's matching index
template. This lifecycle policy is set in the `index.lifecycle.name` setting.
+
If the template does not include a lifecycle policy, this property is not
included in the response.
+
NOTE: A data stream's backing indices may be assigned different lifecycle
policies. To retrieve the lifecycle policy for individual backing indices,
use the <<indices-get-settings,get index settings API>>.
====
[[get-data-stream-api-example]] [[get-data-stream-api-example]]
==== {api-examples-title} ==== {api-examples-title}
@ -107,35 +191,53 @@ deleted indices.
---- ----
GET _data_stream/my-data-stream* GET _data_stream/my-data-stream*
---- ----
// TEST[continued]
// TEST[skip_shard_failures]
The API returns the following response: The API returns the following response:
[source,console-result] [source,console-result]
---- ----
[ {
{ "data_streams": [
"name" : "my-data-stream", <1> {
"timestamp_field" : "@timestamp", <2> "name": "my-data-stream",
"indices" : [ <3> "timestamp_field": {
{ "name": "@timestamp"
"index_name" : ".ds-my-data-stream-000001",
"index_uuid" : "DXAE-xcCQTKF93bMm9iawA"
}, },
{ "indices": [
"index_name" : ".ds-my-data-stream-000002", {
"index_uuid" : "Wzxq0VhsQKyPxHhaK3WYAg" "index_name": ".ds-my-data-stream-000001",
} "index_uuid": "xCEhwsp8Tey0-FLNFYVwSg"
], },
"generation" : 2 <4> {
} "index_name": ".ds-my-data-stream-000002",
] "index_uuid": "PA_JquKGSiKcAKBA8DJ5gw"
}
],
"generation": 2,
"status": "GREEN",
"template": "my-index-template",
"ilm_policy": "my-lifecycle-policy"
},
{
"name": "my-data-stream_two",
"timestamp_field": {
"name": "@timestamp"
},
"indices": [
{
"index_name": ".ds-my-data-stream_two-000001",
"index_uuid": "3liBu2SYS5axasRt6fUIpA"
}
],
"generation": 1,
"status": "YELLOW",
"template": "my-index-template",
"ilm_policy": "my-lifecycle-policy"
}
]
}
---- ----
// TESTRESPONSE[skip:unable to assert responses with top level array] // TESTRESPONSE[s/"index_uuid": "xCEhwsp8Tey0-FLNFYVwSg"/"index_uuid": $body.data_streams.0.indices.0.index_uuid/]
// TESTRESPONSE[s/"index_uuid": "PA_JquKGSiKcAKBA8DJ5gw"/"index_uuid": $body.data_streams.0.indices.1.index_uuid/]
<1> Name of the data stream // TESTRESPONSE[s/"index_uuid": "3liBu2SYS5axasRt6fUIpA"/"index_uuid": $body.data_streams.1.indices.0.index_uuid/]
<2> The name of the timestamp field for the data stream // TESTRESPONSE[s/"status": "GREEN"/"status": "YELLOW"/]
<3> List of backing indices
<4> Current generation for the data stream