parent
ea3c49979e
commit
747f8bfe79
|
@ -56,20 +56,19 @@ image::images/data-streams/data-streams-diagram.svg[align="center"]
|
|||
|
||||
To create backing indices, each data stream requires a matching
|
||||
<<indices-templates,index template>>. This template acts as a blueprint for the
|
||||
stream's backing indices. It contains:
|
||||
stream's backing indices. It specifies:
|
||||
|
||||
* The mappings and settings applied to each backing index when it's created.
|
||||
* One or more wildcard (`*`) patterns that match the name of the stream.
|
||||
|
||||
* A name or wildcard (`*`) pattern that matches the data stream's name.
|
||||
* The mappings and settings for the stream's backing indices.
|
||||
|
||||
* A `data_stream` object with an empty body (`{ }`). This object indicates the
|
||||
template is used for data streams.
|
||||
* That the template is used exclusively for data streams.
|
||||
|
||||
A `@timestamp` field must be included in every document indexed to the data
|
||||
stream. This field can be mapped as a <<date,`date`>> or
|
||||
<<date_nanos,`date_nanos`>> field data type in the stream's matching index
|
||||
template. If no mapping is specified in the template, the `date` field data type
|
||||
with default options is used.
|
||||
Every document indexed to a data stream must have a `@timestamp` field. This
|
||||
field can be mapped as a <<date,`date`>> or <<date_nanos,`date_nanos`>> field
|
||||
data type by the stream's index template. If the template does not specify a
|
||||
mapping, the `@timestamp` field is mapped as a `date` field with default
|
||||
options.
|
||||
|
||||
The same index template can be used to create multiple data streams.
|
||||
|
||||
|
|
|
@ -55,10 +55,18 @@ triggers for such actions.
|
|||
TIP: While optional, we recommend using {ilm-init} to manage the backing indices
|
||||
associated with a data stream.
|
||||
|
||||
The following <<ilm-put-lifecycle,create lifecycle policy API>> request
|
||||
configures the `logs_policy` lifecycle policy.
|
||||
You can create the policy through the Kibana UI. In Kibana, open the menu and go
|
||||
to *Stack Management > Index Lifecycle Policies*. Click *Index Lifecycle
|
||||
Policies*.
|
||||
|
||||
The `logs_policy` policy uses the <<ilm-rollover,`rollover` action>> to create a
|
||||
[role="screenshot"]
|
||||
image::images/ilm/create-policy.png[Index Lifecycle Policies page]
|
||||
|
||||
You can also create a policy using the <<ilm-put-lifecycle,create lifecycle
|
||||
policy API>>.
|
||||
|
||||
The following request configures the `logs_policy` lifecycle policy. The
|
||||
`logs_policy` policy uses the <<ilm-rollover,`rollover` action>> to create a
|
||||
new <<data-stream-write-index,write index>> for the data stream when the current
|
||||
one reaches 25GB in size. The policy also deletes backing indices 30 days after
|
||||
their rollover.
|
||||
|
@ -95,19 +103,18 @@ PUT /_ilm/policy/logs_policy
|
|||
A data stream uses an index template to configure its backing indices. A
|
||||
template for a data stream must specify:
|
||||
|
||||
* An index pattern that matches the name of the stream.
|
||||
|
||||
* An empty `data_stream` object that indicates the template is used for data
|
||||
streams.
|
||||
* One or more wildcard (`*`) patterns that match the name of the stream.
|
||||
|
||||
* The mappings and settings for the stream's backing indices.
|
||||
|
||||
* That the template is used exclusively for data streams.
|
||||
|
||||
Every document indexed to a data stream must have a `@timestamp` field. This
|
||||
field can be mapped as a <<date,`date`>> or <<date_nanos,`date_nanos`>> field
|
||||
data type by the stream's index template. This mapping can include other
|
||||
<<mapping-params,mapping parameters>>, such as <<mapping-date-format,`format`>>.
|
||||
If the template does not specify a mapping is specified in the template, the
|
||||
`@timestamp` field is mapped as a `date` field with default options.
|
||||
If the template does not specify a mapping, the `@timestamp` field is mapped as
|
||||
a `date` field with default options.
|
||||
|
||||
We recommend using {ilm-init} to manage a data stream's backing indices. Specify
|
||||
the name of the lifecycle policy with the `index.lifecycle.name` setting.
|
||||
|
@ -117,11 +124,24 @@ in this template before creating a data stream. Later changes to the mappings or
|
|||
settings of a stream's backing indices may require reindexing. See
|
||||
<<data-streams-change-mappings-and-settings>>.
|
||||
|
||||
The following <<indices-templates,put index template API>> request
|
||||
configures the `logs_data_stream` template.
|
||||
You can create an index template through the Kibana UI:
|
||||
|
||||
Because no field mapping is specified, the `@timestamp` field uses the `date`
|
||||
field data type by default.
|
||||
. From Kibana, open the menu and go to *Stack Management > Index Management*.
|
||||
. In the *Index Templates* tab, click *Create template*.
|
||||
. In the Create template wizard, use the *Data stream* toggle to indicate the
|
||||
template is used exclusively for data streams.
|
||||
|
||||
[role="screenshot"]
|
||||
image::images/data-streams/create-index-template.png[Create template page]
|
||||
|
||||
You can also create a template using the <<indices-put-template,put index
|
||||
template API>>. The template must include a `data_stream` object with an empty
|
||||
body (`{ }`). This object indicates the template is used exclusively for data
|
||||
streams.
|
||||
|
||||
The following request configures the `logs_data_stream` index template. Because
|
||||
no field mapping is specified, the `@timestamp` field uses the `date` field data
|
||||
type by default.
|
||||
|
||||
[source,console]
|
||||
----
|
||||
|
@ -179,7 +199,7 @@ You can create a data stream using one of two methods:
|
|||
[[index-documents-to-create-a-data-stream]]
|
||||
==== Index documents to create a data stream
|
||||
|
||||
You can automatically generate a data stream using an indexing request. Submit
|
||||
You can automatically create a data stream using an indexing request. Submit
|
||||
an <<add-documents-to-a-data-stream,indexing request>> to a target
|
||||
matching the name or wildcard pattern defined in the template's `index_patterns`
|
||||
property.
|
||||
|
@ -254,10 +274,16 @@ PUT /_data_stream/logs_alt
|
|||
[[get-info-about-a-data-stream]]
|
||||
=== Get information about a data stream
|
||||
|
||||
You can use the <<indices-get-data-stream,get data stream API>> to get
|
||||
information about one or more data streams, including:
|
||||
To view information about a data stream in Kibana, open the menu and go to
|
||||
*Stack Management > Index Management*. In the *Data Streams* tab, click a data
|
||||
stream's name to view information about the stream.
|
||||
|
||||
[role="screenshot"]
|
||||
image::images/data-streams/data-streams-list.png[Data Streams tab]
|
||||
|
||||
You can also use the <<indices-get-data-stream,get data stream API>> to retrieve
|
||||
the following information about one or more data streams:
|
||||
|
||||
* The timestamp field
|
||||
* The current backing indices, which is returned as an array. The last item in
|
||||
the array contains information about the stream's current write index.
|
||||
* The current generation
|
||||
|
@ -266,8 +292,6 @@ information about one or more data streams, including:
|
|||
* The current {ilm-init} lifecycle policy in the stream's matching index
|
||||
template
|
||||
|
||||
This is also handy way to verify that a recently created data stream exists.
|
||||
|
||||
The following get data stream API request retrieves information about the
|
||||
`logs` data stream.
|
||||
|
||||
|
@ -334,11 +358,18 @@ data. See <<data-stream-privileges>>.
|
|||
[[delete-a-data-stream]]
|
||||
=== Delete a data stream
|
||||
|
||||
You can use the <<indices-delete-data-stream,delete data stream API>> to delete
|
||||
a data stream and its backing indices.
|
||||
You can use the Kibana UI to delete a data stream and its backing indices. In
|
||||
Kibana, open the menu and go to *Stack Management > Index Management*. In the
|
||||
*Data Streams* tab, click the trash can icon to delete a stream and its backing
|
||||
indices.
|
||||
|
||||
The following delete data stream API request deletes the `logs` data stream. This
|
||||
request also deletes the stream's backing indices and any data they contain.
|
||||
[role="screenshot"]
|
||||
image::images/data-streams/data-streams-list.png[Data Streams tab]
|
||||
|
||||
You can also use the the <<indices-delete-data-stream,delete data stream API>>
|
||||
to delete a data stream. The following delete data stream API request deletes
|
||||
the `logs` data stream. This request also deletes the stream's backing indices
|
||||
and any data they contain.
|
||||
|
||||
[source,console]
|
||||
----
|
||||
|
|
|
@ -55,9 +55,10 @@ reaches either a `max_size` of 50 gigabytes or a `max_age` of 30 days.
|
|||
* A `delete` phase that sets `min_age` to remove the index 90 days after rollover.
|
||||
Note that this value is relative to the rollover time, not the index creation time.
|
||||
|
||||
You can create the policy through {kib} Management or with the
|
||||
You can create the policy through {kib} or with the
|
||||
<<ilm-put-lifecycle, put policy>> API.
|
||||
To create the policy from {kib}, go to Management and click **Index Lifecycle Policies**.
|
||||
To create the policy from {kib}, open the menu and go to *Stack Management >
|
||||
Index Lifecycle Policies*. Click *Index Lifecycle Policies*.
|
||||
|
||||
[role="screenshot"]
|
||||
image:images/ilm/create-policy.png[]
|
||||
|
@ -109,9 +110,14 @@ To enable the {ilm-init} to manage the data stream, the template configures one
|
|||
|
||||
* `index.lifecycle.name` specifies the name of the lifecycle policy to apply to the data stream.
|
||||
|
||||
You can use the {kib} Create template wizard to add the template.
|
||||
This wizard invokes the put _index_template API to create the <<indices-templates,index template>>
|
||||
with the options you specify.
|
||||
You can use the {kib} Create template wizard to add the template. From Kibana,
|
||||
open the menu and go to *Stack Management > Index Management*. In the *Index
|
||||
Templates* tab, click *Create template*.
|
||||
|
||||
image::images/data-streams/create-index-template.png[Create template page]
|
||||
|
||||
This wizard invokes the <<indices-put-template,put index template API>> to create
|
||||
the index template with the options you specify.
|
||||
|
||||
.API example
|
||||
[%collapsible]
|
||||
|
@ -303,9 +309,9 @@ that match the index pattern.
|
|||
* `index.lifecycle.rollover_alias` specifies the index alias to be rolled over
|
||||
when the rollover action is triggered for an index.
|
||||
|
||||
You can use the {kib} Create template wizard to add the template.
|
||||
To access the wizard, go to Management, click **Index Management**,
|
||||
and select the **Index Templates** view.
|
||||
You can use the {kib} Create template wizard to add the template. To access the
|
||||
wizard, open the menu, go to *Stack Management > Index Management*, and click
|
||||
the *Index Templates* tab.
|
||||
|
||||
[role="screenshot"]
|
||||
image:images/ilm/create-template-wizard.png[]
|
||||
|
|
Binary file not shown.
After Width: | Height: | Size: 86 KiB |
Binary file not shown.
After Width: | Height: | Size: 16 KiB |
Loading…
Reference in New Issue