Remove "About data streams," "Date math support for index names," and "Simulate multi-component templates"
This commit is contained in:
aetter
parent
62611a7605
commit
60b1859f1e
|
|
@ -15,32 +15,6 @@ Data streams simplify this bootstrapping process and enforce a setup that best s
|
|||
A data stream is internally composed of multiple backing indices. Search requests are routed to all the backing indices, while indexing requests are routed to the latest write index. You can use [ISM]({{site.url}}{{site.baseurl}}/im-plugin/ism/index/) policies to automatically handle rollovers or deletion of indices in a data stream, based on your use case.
|
||||
|
||||
|
||||
## About data streams
|
||||
|
||||
A data stream consists of one or more hidden auto-generated backing indices. These backing indices are named using the following convention:
|
||||
|
||||
```
|
||||
.ds-<data-stream-name>-<generation-id>
|
||||
```
|
||||
|
||||
For example, `.ds-logs-redis-000003`, where generation-id is a six-digit, zero-padded integer that acts as a cumulative count of the data stream’s rollovers, starting at `000001`.
|
||||
|
||||
The most recently created backing index is the data stream’s write index. You can’t add documents directly to any of the backing indices. You can only add them via the data stream handle:
|
||||
|
||||
|
||||
|
||||
The data stream routes search requests to all of its backing indices. It uses the timestamp field to intelligently route search requests to the right set of indices and shards:
|
||||
|
||||
|
||||
|
||||
The following operations are not supported on the write index because they might hinder the indexing operation:
|
||||
|
||||
- close
|
||||
- clone
|
||||
- delete
|
||||
- shrink
|
||||
- split
|
||||
|
||||
## Get started with data streams
|
||||
|
||||
### Step 1: Create an index template
|
||||
|
|
|
|||
|
|
@ -1,60 +0,0 @@
|
|||
---
|
||||
layout: default
|
||||
title: Date math support for index names
|
||||
nav_order: 92
|
||||
---
|
||||
|
||||
# Date math support for index names
|
||||
|
||||
Date math is shorthand arithmetic for finding relative dates.
|
||||
|
||||
If you're indexing time-series data with the dates mapped in the index names, you can use date math in your queries to filter index names and limit the number of searched indices.
|
||||
|
||||
## Date math syntax
|
||||
|
||||
The date math syntax for an index name is as follows:
|
||||
|
||||
```
|
||||
<static_name{date_math_expr{date_format|time_zone}}>
|
||||
```
|
||||
|
||||
- `static_name`: The unchanged or static portion of the index name. To use the characters `{` and `}` in the static part of an index name, escape them with a backslash `\`.
|
||||
- `date_math_expr`: The changing or dynamic portion of the index name that’s computed by the date math expression. For example, `now+1h` adds one hour, `now-1d` subtracts one hour, and `now/d` rounds down to the nearest day, where `now` represents the current timestamp.
|
||||
- `date_format`: (Optional) Specify the format for the computed date. The default value is `YYYY.MM.dd`. Make sure that you’re using the correct small or capital letters in the date format. For example, `mm` denotes minute of hour, while `MM` denotes month of year. Similarly, `hh` denotes the hour in the `1-12` range in combination with AM/PM, while `HH` denotes the hour in the `0-23` 24-hour range.
|
||||
- `time_zone`: (Optional) Specify the timezone offset. The default value is UTC. For example, the UTC time offset for PST is `-08:00`.
|
||||
|
||||
## Date math example
|
||||
|
||||
You must enclose date math index names within angle brackets.
|
||||
|
||||
If today is 22nd March, 2024:
|
||||
|
||||
- `<logstash-{now/d}>` resolves to `logstash-2024.03.22`
|
||||
- `<logstash-{now/M}>` resolves to `logstash-2024.03.01`
|
||||
- `<logstash-{now/M{YYYY.MM}}>` resolves to `logstash-2024.03`
|
||||
- `<logstash-{now/M-1M{YYYY.MM}}>` resolves to `logstash-2024.02`
|
||||
- `<logstash-{now/d{yyyy.MM.dd|+12:00}}>` resolves to `logstash-2024.03.23`
|
||||
|
||||
You need to encode all special characters in URI format:
|
||||
|
||||
Special characters | URI format
|
||||
:--- | :---
|
||||
`<` | %3C
|
||||
`>` | %3E
|
||||
`/` | %2F
|
||||
`{` | %7B
|
||||
`}` | %7D
|
||||
`|` | %7C
|
||||
`+` | %2B
|
||||
`:` | %3A
|
||||
`,` | %2C
|
||||
`\` | %5C
|
||||
|
||||
If you are searching for errors in your daily logs with the default Logstash index name format `logstash-YYYY.MM.dd`, you can use date math to restrict the search to indices of the past three days:
|
||||
|
||||
```
|
||||
# GET <logstash-{now/d-2d}>,<logstash-{now/d-1d}>,<logstash-{now/d}>/_search
|
||||
GET %3Clogstash-%7Bnow%2Fd-2d%7D%3E%2C%3Clogstash-%7Bnow%2Fd-1d%7D%3E%2C%3Clogstash-%7Bnow%2Fd%7D%3E/_search
|
||||
```
|
||||
|
||||
This date math expression is evaluated at runtime.
|
||||
|
|
@ -328,71 +328,6 @@ GET logs-2020-01-01
|
|||
}
|
||||
```
|
||||
|
||||
### Simulate multi-component templates
|
||||
|
||||
For index templates composed of multiple component templates, you can simulate applying a new template to verify whether the settings are applied as you expect.
|
||||
|
||||
To simulate the settings that would be applied to a specific index name:
|
||||
|
||||
```json
|
||||
POST _index_template/_simulate_index/<index_name>
|
||||
```
|
||||
|
||||
To simulate the settings that would be applied from an existing template:
|
||||
|
||||
```json
|
||||
POST _index_template/_simulate/<index_template>
|
||||
```
|
||||
|
||||
You can also specify a template definition in the simulate request:
|
||||
|
||||
```json
|
||||
POST _index_template/_simulate
|
||||
{
|
||||
"index_patterns": [
|
||||
"logs-2020-01-*"
|
||||
],
|
||||
"template": {
|
||||
"settings" : {
|
||||
"index.number_of_shards" : 3
|
||||
}
|
||||
},
|
||||
"composed_of": ["component_template_1", "component_template_2"]
|
||||
}
|
||||
```
|
||||
|
||||
The `_simulate` API returns the final settings, mappings, and aliases that will be applied to indices that match the index pattern. You can also see any overlapping templates whose configuration is superseded by the simulated template body or higher priority templates:
|
||||
|
||||
```json
|
||||
{
|
||||
"template" : {
|
||||
"settings" : {
|
||||
"index" : {
|
||||
"number_of_shards" : "3"
|
||||
}
|
||||
},
|
||||
"mappings" : {
|
||||
"properties" : {
|
||||
"@timestamp" : {
|
||||
"type" : "date"
|
||||
},
|
||||
"ip_address" : {
|
||||
"type" : "ip"
|
||||
}
|
||||
}
|
||||
},
|
||||
"aliases" : { }
|
||||
},
|
||||
"overlapping" : [
|
||||
{
|
||||
"name" : "daily_logs",
|
||||
"index_patterns" : [
|
||||
"logs-2020-01-*"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Index template options
|
||||
|
||||
|
|
|
|||
Binary file not shown.
|
Before Width: | Height: | Size: 49 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 44 KiB |
Loading…
Reference in New Issue