[7.x] Initial documentation for index templates V2 (#55755) (#55898)

Backports the following commits to 7.x:
 - Initial documentation for index templates V2 (#55755)
This commit is contained in:
Lee Hinman 2020-04-28 16:10:50 -06:00 committed by GitHub
parent f8db1a56f8
commit 4315a55a1c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 725 additions and 20 deletions

View File

@ -81,6 +81,7 @@ testClusters.integTest {
// TODO: remove this once cname is prepended to transport.publish_address by default in 8.0
systemProperty 'es.transport.cname_in_publish_address', 'true'
systemProperty 'es.itv2_feature_enabled', 'true'
}
// build the cluster with all plugins

View File

@ -55,6 +55,11 @@ index settings, aliases, mappings, and index templates.
* <<indices-get-template>>
* <<indices-template-exists>>
[float]
[[component-template-apis]]
=== Component templates:
* <<indices-component-template>>
[float]
[[monitoring]]
=== Monitoring:
@ -125,6 +130,10 @@ include::indices/template-exists.asciidoc[]
include::indices/open-close.asciidoc[]
include::indices/index-templates.asciidoc[]
include::indices/component-templates.asciidoc[]
include::indices/templates.asciidoc[]
include::indices/put-mapping.asciidoc[]

View File

@ -0,0 +1,272 @@
[[indices-component-template]]
=== Component template API
++++
<titleabbrev>Component template</titleabbrev>
++++
Component templates are building blocks that specify mappings, settings, or alias configuration, but
don't apply to a set of indices themselves. To be used a component template must be specified in the
`composed_of` of an <<indices-templates,index template>>.
[source,console]
--------------------------------------------------
PUT _component_template/template_1
{
"template": {
"settings": {
"number_of_shards": 1
},
"mappings": {
"_source": {
"enabled": false
},
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date",
"format": "EEE MMM dd HH:mm:ss Z yyyy"
}
}
}
}
}
--------------------------------------------------
// TESTSETUP
//////////////////////////
[source,console]
--------------------------------------------------
DELETE _component_template/template_*
--------------------------------------------------
// TEARDOWN
//////////////////////////
[[put-component-template-api-request]]
==== {api-request-title}
`PUT /_component_template/<component-template>`
[[put-component-template-api-desc]]
==== {api-description-title}
Use the PUT component template API to create or update a component template.
// tag::component-template-def[]
Component templates define <<index-modules-settings,settings>>, <<mapping,mappings>>, and
<<indices-aliases,aliases>> that you can automatically apply when creating new indices. On their own
component templates are not matched or applied to an index, instead they are used in the
`composed_of` definition of an <<indices-templates,index template>> to make up the final index
configuration.
// end::component-template-def[]
Component templates are only used during index creation. Changes to component templates do not
affect existing indices. Settings and mappings specified in <<indices-create-index, create index>>
API requests and <<indices-templates,index templates>> override any settings or mappings specified
in a component template.
===== Comments in component templates
You can use C-style /* */ block comments in component templates.
You can include comments anywhere in the request body,
except before the opening curly bracket.
[[getting-component-templates]]
===== Getting component templates
=== Get component template API
++++
<titleabbrev>Get component template</titleabbrev>
++++
Returns information about one or more component templates.
[source,console]
--------------------------------------------------
GET /_component_template/template_1
--------------------------------------------------
[[get-component-template-api-request]]
==== {api-request-title}
`GET /_component-template/<component-template>`
[[get-component-template-api-path-params]]
==== {api-path-parms-title}
include::{docdir}/rest-api/common-parms.asciidoc[tag=index-template]
+
To return all component templates, omit this parameter or use a value of `*`.
[[get-component-template-api-query-params]]
==== {api-query-parms-title}
include::{docdir}/rest-api/common-parms.asciidoc[tag=flat-settings]
include::{docdir}/rest-api/common-parms.asciidoc[tag=local]
include::{docdir}/rest-api/common-parms.asciidoc[tag=master-timeout]
[[get-component-template-api-example]]
==== {api-examples-title}
[[get-component-template-api-wildcard-ex]]
===== Get component templates using a wildcard expression
[source,console]
--------------------------------------------------
GET /_component_template/temp*
--------------------------------------------------
[[get-component-template-api-all-ex]]
===== Get all component templates
[source,console]
--------------------------------------------------
GET /_component_template
--------------------------------------------------
[[put-component-template-api-path-params]]
==== {api-path-parms-title}
`<component-template>`::
(Required, string)
Name of the component template to create.
[[put-component-template-api-query-params]]
==== {api-query-parms-title}
`create`::
(Optional, boolean)
If `true`, this request cannot replace or update existing component templates.
Defaults to `false`.
include::{docdir}/rest-api/common-parms.asciidoc[tag=master-timeout]
[[put-component-template-api-request-body]]
==== {api-request-body-title}
`template`::
(Required, object)
This is the template to be applied, may optionally include a `mappings`,
`settings`, or `aliases` configuration.
include::{docdir}/rest-api/common-parms.asciidoc[tag=aliases]
include::{docdir}/rest-api/common-parms.asciidoc[tag=mappings]
include::{docdir}/rest-api/common-parms.asciidoc[tag=settings]
`version`::
(Optional, integer)
Version number used to manage component templates externally.
This number is not automatically generated or incremented by {es}.
`_meta`::
(Optional, object)
Optional user metadata about the component template. May have any contents.
This map is not automatically generated by {es}.
[[put-component-template-api-example]]
==== {api-examples-title}
===== Component template with index aliases
You can include <<indices-aliases,index aliases>> in a component template.
[source,console]
--------------------------------------------------
PUT _component_template/template_1
{
"template": {
"settings" : {
"number_of_shards" : 1
},
"aliases" : {
"alias1" : {},
"alias2" : {
"filter" : {
"term" : {"user" : "kimchy" }
},
"routing" : "kimchy"
},
"{index}-alias" : {} <1>
}
}
}
--------------------------------------------------
<1> the `{index}` placeholder in the alias name will be replaced with the
actual index name that the template gets applied to, during index creation.
[[applying-component-templates]]
===== Applying component templates
Component templates on their own don't apply or interact with indices at all. In order for them to
be effective, they must be used as part of index template's `composed_of` field to take effect. See
the <<indices-templates,index templates>> documentation for more information.
[[component-templates-version]]
===== Component template versioning
You can use the `version` parameter to add an optional version number to a component template. External
systems can use these version numbers to simplify template management.
The `version` parameter is completely optional and not automatically generated by {es}.
To unset a `version`, replace the template without specifying one.
[source,console]
--------------------------------------------------
PUT /_component_template/template_1
{
"template": {
"settings" : {
"number_of_shards" : 1
}
},
"version": 123
}
--------------------------------------------------
To check the `version`, you can use the <<getting-component-templates,get component template>> API.
[[component-templates-metadata]]
===== Component template metadata
You can use the `_meta` parameter to add optional metadata to a component template. This is a
user-defined map that can contain any data. This data will be stored in the cluster state however,
so keeping it short is preferrable.
The `_meta` parameter is completely optional and not automatically generated by {es}.
To unset `_meta`, replace the template without specifying one.
[source,console]
--------------------------------------------------
PUT /_component_template/template_1
{
"template": {
"settings" : {
"number_of_shards" : 1
}
},
"_meta": {
"description": "set number of shards to one",
"serialization": {
"class": "MyComponentTemplate",
"id": 10
}
}
}
--------------------------------------------------
To check the `_meta`, you can use the <<getting-component-templates,get component template>> API.

View File

@ -49,4 +49,4 @@ include::{docdir}/rest-api/common-parms.asciidoc[tag=index-template]
[[delete-template-api-query-params]]
==== {api-query-parms-title}
include::{docdir}/rest-api/common-parms.asciidoc[tag=timeoutparms]
include::{docdir}/rest-api/common-parms.asciidoc[tag=timeoutparms]

View File

@ -1,4 +1,4 @@
[[indices-get-template]]
[[indices-get-template-v1]]
=== Get index template API
++++
<titleabbrev>Get index template</titleabbrev>
@ -33,13 +33,13 @@ GET /_template/template_1
--------------------------------------------------
[[get-template-api-request]]
[[get-template-v1-api-request]]
==== {api-request-title}
`GET /_template/<index-template>`
[[get-template-api-path-params]]
[[get-template-v1-api-path-params]]
==== {api-path-parms-title}
include::{docdir}/rest-api/common-parms.asciidoc[tag=index-template]
@ -48,7 +48,7 @@ To return all index templates, omit this parameter
or use a value of `_all` or `*`.
[[get-template-api-query-params]]
[[get-template-v1-api-query-params]]
==== {api-query-parms-title}
include::{docdir}/rest-api/common-parms.asciidoc[tag=flat-settings]
@ -60,11 +60,11 @@ include::{docdir}/rest-api/common-parms.asciidoc[tag=local]
include::{docdir}/rest-api/common-parms.asciidoc[tag=master-timeout]
[[get-template-api-example]]
[[get-template-v1-api-example]]
==== {api-examples-title}
[[get-template-api-multiple-ex]]
[[get-template-v1-api-multiple-ex]]
===== Get multiple index templates
[source,console]
@ -73,7 +73,7 @@ GET /_template/template_1,template_2
--------------------------------------------------
[[get-template-api-wildcard-ex]]
[[get-template-v1-api-wildcard-ex]]
===== Get index templates using a wildcard expression
[source,console]
@ -82,7 +82,7 @@ GET /_template/temp*
--------------------------------------------------
[[get-template-api-all-ex]]
[[get-template-v1-api-all-ex]]
===== Get all index templates
[source,console]

View File

@ -0,0 +1,411 @@
[[indices-templates]]
=== Index template API
++++
<titleabbrev>Index template</titleabbrev>
++++
This documentation is about V2 (version 2) index templates. For V1 templates please see the
<<indices-templates-v1,V1 template documentation>>.
[NOTE]
====
In {es} 7.8 the `prefer_v2_templates` querystring parameter was introduced to the index, update,
bulk, and create index APIs to allow configuring whether to favor V2 or V1 index templates. In 7.x
the default value for this parameter is `false` meaning that the <<indices-templates-v1,V1 index
templates>> take precedence. In 8.x the default value for the parameter is `true` meaning that V2
index templates take precedence (V1 templates may still match if no V2 template matches).
====
An index template is a way to tell {es} how to configure an index when it is created. Templates are
configured prior to index creation and then when an index is created either manually or through
indexing a document, the template settings are used as a basis for creating the index.
There are two types of templates, index templates and <<indices-component-template,component
templates>>. Component templates are reusable building blocks that configure mappings, settings, and
aliases. You use component templates to construct index templates, they aren't directly applied to a
set of indices. Index templates can contain a collection of component templates, as well as directly
specify settings, mappings, and aliases.
If a new index matches more than one index template, the index template with the highest priority is used.
If an index is created with explicit settings and also matches an index template,
the settings from the create index request take precedence over settings specified in the index template and its component templates.
[source,console]
--------------------------------------------------
PUT _index_template/template_1
{
"index_patterns": ["te*", "bar*"],
"template": {
"settings": {
"number_of_shards": 1
},
"mappings": {
"_source": {
"enabled": false
},
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date",
"format": "EEE MMM dd HH:mm:ss Z yyyy"
}
}
},
"aliases": {
"mydata": { }
}
},
"priority": 10,
"composed_of": ["component_template1", "other_component_template"],
"version": 3,
"_meta": {
"description": "my custom"
}
}
--------------------------------------------------
// TESTSETUP
//////////////////////////
[source,console]
--------------------------------------------------
DELETE _index_template/template_*
DELETE _component_template/*
--------------------------------------------------
// TEARDOWN
//////////////////////////
[[put-index-template-api-request]]
==== {api-request-title}
`PUT /_index_template/<index-template>`
[[put-index-template-api-desc]]
==== {api-description-title}
Creates or updates an index template.
// tag::index-template-def[]
Index templates define <<index-modules-settings,settings>> and <<mapping,mappings>> that you can
automatically apply when creating new indices. {es} applies templates to new indices based on an
index pattern that matches the index name.
// end::index-template-def[]
Index templates are only applied during index creation. Changes to index templates do not affect
existing indices. Settings and mappings specified in <<indices-create-index, create index>> API
requests override any settings or mappings specified in an index template.
===== Comments in index templates
You can use C-style /* */ block comments in index templates. You can include comments anywhere in
the request body, except before the opening curly bracket.
[[getting]]
===== Getting templates
[[indices-get-template]]
=== Get index template API
++++
<titleabbrev>Get index template</titleabbrev>
++++
Returns information about one or more index templates.
[source,console]
--------------------------------------------------
GET /_index_template/template_1
--------------------------------------------------
[[get-template-api-request]]
==== {api-request-title}
`GET /_index_template/<index-template>`
[[get-template-api-path-params]]
==== {api-path-parms-title}
include::{docdir}/rest-api/common-parms.asciidoc[tag=index-template]
+
To return all index templates, omit this parameter or use a value of `*`.
[[get-template-api-query-params]]
==== {api-query-parms-title}
include::{docdir}/rest-api/common-parms.asciidoc[tag=flat-settings]
include::{docdir}/rest-api/common-parms.asciidoc[tag=local]
include::{docdir}/rest-api/common-parms.asciidoc[tag=master-timeout]
[[get-template-api-example]]
==== {api-examples-title}
[[get-template-api-wildcard-ex]]
===== Get index templates using a wildcard expression
[source,console]
--------------------------------------------------
GET /_index_template/temp*
--------------------------------------------------
[[get-template-api-all-ex]]
===== Get all index templates
[source,console]
--------------------------------------------------
GET /_index_template
--------------------------------------------------
[[put-index-template-api-path-params]]
==== {api-path-parms-title}
`<index-template>`::
(Required, string)
Name of the index template to create.
[[put-index-template-api-query-params]]
==== {api-query-parms-title}
`create`::
(Optional, boolean)
If `true`, this request cannot replace or update existing index templates. Defaults to `false`.
include::{docdir}/rest-api/common-parms.asciidoc[tag=master-timeout]
[[put-index-template-api-request-body]]
==== {api-request-body-title}
`index_patterns`::
(Required, array of strings)
Array of wildcard expressions
used to match the names of indices during creation.
include::{docdir}/rest-api/common-parms.asciidoc[tag=aliases]
include::{docdir}/rest-api/common-parms.asciidoc[tag=mappings]
include::{docdir}/rest-api/common-parms.asciidoc[tag=settings]
`template`::
(Optional, object)
This is the template to be applied, may optionally include a `mappings`,
`settings`, or `aliases` configuration.
`composed_of`::
(Optional, array of strings)
An ordered list of component template names. Component templates will be merged in the order
specified, meaning that the last component template specified has the highest precedence. See the
<<indices-component-template,component template>> reference for information.
`priority`::
(Optional, integer)
Priority to determine index template precedence when a new index is created. The index template with
the highest priority is chosen.
This number is not automatically generated by {es}.
`version`::
(Optional, integer)
Version number used to manage index templates externally.
This number is not automatically generated by {es}.
`_meta`::
(Optional, object)
Optional user metadata about the index template. May have any contents.
This map is not automatically generated by {es}.
[[put-index-template-api-example]]
==== {api-examples-title}
==== Composing multiple component templates
When multiple component templates are specified in the `composed_of` field for an index template,
they are merged in the order specified, meaning that later component templates override earlier
component templates.
For two component templates:
[source,console]
--------------------------------------------------
PUT /_component_template/template_with_2_shards
{
"template": {
"settings": {
"index.number_of_shards": 2
}
}
}
--------------------------------------------------
[source,console]
--------------------------------------------------
PUT /_component_template/template_with_3_shards
{
"template": {
"settings": {
"index.number_of_shards": 3
}
}
}
--------------------------------------------------
The order they are specified changes the number of shards for an index:
[source,console]
--------------------------------------------------
PUT /_index_template/template_1
{
"index_patterns": ["t*"],
"composed_of": ["template_with_2_shards", "template_with_3_shards"]
}
--------------------------------------------------
In this case, an index matching `t*` will have three primary shards. If the order of composed
templates were reversed, the index would have two primary shards.
===== Index template with index aliases
You can include <<indices-aliases,index aliases>> in an index template.
[source,console]
--------------------------------------------------
PUT _index_template/template_1
{
"index_patterns" : ["te*"],
"template": {
"settings" : {
"number_of_shards" : 1
},
"aliases" : {
"alias1" : {},
"alias2" : {
"filter" : {
"term" : {"user" : "kimchy" }
},
"routing" : "kimchy"
},
"{index}-alias" : {} <1>
}
}
}
--------------------------------------------------
<1> the `{index}` placeholder in the alias name will be replaced with the
actual index name that the template gets applied to, during index creation.
[[multiple-templates]]
===== Indices matching multiple templates
When an index is created that matches multiple index templates, only the index template with the
highest priority is applied. For example:
[source,console]
--------------------------------------------------
PUT /_index_template/template_1
{
"index_patterns" : ["t*"],
"priority" : 0,
"template": {
"settings" : {
"number_of_shards" : 1,
"number_of_replicas": 0
},
"mappings" : {
"_source" : { "enabled" : false }
}
}
}
PUT /_index_template/template_2
{
"index_patterns" : ["te*"],
"priority" : 1,
"template": {
"settings" : {
"number_of_shards" : 2
},
"mappings" : {
"_source" : { "enabled" : true }
}
}
}
--------------------------------------------------
For indices that start with `te*`, `_source` will enabled, and the index will have two primary
shards and one replica, because only `template_2` will be applied.
NOTE: Multiple templates with overlapping index patterns at the same priority are not allowed, and
an error will be thrown when attempting to create a template matching an existing index template at
identical priorities.
[[versioning-templates]]
===== Template versioning
You can use the `version` parameter to add an optional version number to an index template. External
systems can use these version numbers to simplify template management.
The `version` parameter is completely optional and not automatically generated by {es}.
To unset a `version`, replace the template without specifying one.
[source,console]
--------------------------------------------------
PUT /_index_template/template_1
{
"index_patterns" : ["foo", "bar"],
"priority" : 0,
"template": {
"settings" : {
"number_of_shards" : 1
}
},
"version": 123
}
--------------------------------------------------
To check the `version`, you can use the <<indices-get-template, get index template>> API.
[[template-metadata]]
===== Template metadata
You can use the `_meta` parameter to add optional metadata to an index template. This is a
user-defined map that can contain any data. This data will be stored in the cluster state however,
so keeping it short is preferrable.
The `_meta` parameter is completely optional and not automatically generated by {es}.
To unset `_meta`, replace the template without specifying one.
[source,console]
--------------------------------------------------
PUT /_index_template/template_1
{
"index_patterns": ["foo", "bar"],
"template": {
"settings" : {
"number_of_shards" : 3
}
},
"_meta": {
"description": "set number of shards to three",
"serialization": {
"class": "MyIndexTemplate",
"id": 17
}
}
}
--------------------------------------------------
To check the `_meta`, you can use the <<indices-get-template, get index template>> API.

View File

@ -1,9 +1,21 @@
[[indices-templates]]
[[indices-templates-v1]]
=== Put index template API
++++
<titleabbrev>Put index template</titleabbrev>
++++
This documentation is about V1 (version 1) index templates, which are deprecated and will be
replaced by V2 templates. For information about V2 templates, see <<indices-templates>>.
[NOTE]
====
In {es} 7.8 the `prefer_v2_templates` querystring parameter was introduced to the index, update,
bulk, and create index APIs to allow configuring whether to favor V2 or V1 index templates. In 7.x
the default value for this parameter is `false` meaning that the V1 index templates take precedence.
In 8.x the default value for the parameter is `true` meaning that <<indices-templates,V2 index
templates>> take precedence (V1 templates may still match if no V2 template matches).
====
Creates or updates an index template.
[source,console]
@ -42,13 +54,13 @@ DELETE _template/template_*
//////////////////////////
[[put-index-template-api-request]]
[[put-index-template-v1-api-request]]
==== {api-request-title}
`PUT /_template/<index-template>`
[[put-index-template-api-desc]]
[[put-index-template-v1-api-desc]]
==== {api-description-title}
Use the PUT index template API
@ -71,13 +83,13 @@ You can use C-style /* */ block comments in index templates.
You can include comments anywhere in the request body,
except before the opening curly bracket.
[[getting]]
[[getting-v1]]
===== Getting templates
See <<indices-get-template>>.
See <<indices-get-template-v1>>.
[[put-index-template-api-path-params]]
[[put-index-template-v1-api-path-params]]
==== {api-path-parms-title}
`<index-template>`::
@ -85,7 +97,7 @@ See <<indices-get-template>>.
Name of the index template to create.
[[put-index-template-api-query-params]]
[[put-index-template-v1-api-query-params]]
==== {api-query-parms-title}
`create`::
@ -105,7 +117,7 @@ overriding templates with lower values.
include::{docdir}/rest-api/common-parms.asciidoc[tag=master-timeout]
[[put-index-template-api-request-body]]
[[put-index-template-v1-api-request-body]]
==== {api-request-body-title}
`index_patterns`::
@ -125,7 +137,7 @@ Version number used to manage index templates externally.
This number is not automatically generated by {es}.
[[put-index-template-api-example]]
[[put-index-template-v1-api-example]]
==== {api-examples-title}
===== Index template with index aliases
@ -157,7 +169,7 @@ PUT _template/template_1
actual index name that the template gets applied to, during index creation.
[[multiple-templates]]
[[multiple-templates-v1]]
===== Indices matching multiple templates
Multiple index templates can potentially match an index, in this case,
@ -203,7 +215,7 @@ NOTE: Multiple matching templates with the same order value will
result in a non-deterministic merging order.
[[versioning-templates]]
[[versioning-templates-v1]]
===== Template versioning
You can use the `version` parameter