mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-02-06 21:18:31 +00:00
36a3b84fc9
* Default include_type_name to false for get and put mappings. * Default include_type_name to false for get field mappings. * Add a constant for the default include_type_name value. * Default include_type_name to false for get and put index templates. * Default include_type_name to false for create index. * Update create index calls in REST documentation to use include_type_name=true. * Some minor clean-ups around the get index API. * In REST tests, use include_type_name=true by default for index creation. * Make sure to use 'expression == false'. * Clarify the different IndexTemplateMetaData toXContent methods. * Fix FullClusterRestartIT#testSnapshotRestore. * Fix the ml_anomalies_default_mappings test. * Fix GetFieldMappingsResponseTests and GetIndexTemplateResponseTests. We make sure to specify include_type_name=true during xContent parsing, so we continue to test the legacy typed responses. XContent generation for the typeless responses is currently only covered by REST tests, but we will be adding unit test coverage for these as we implement each typeless API in the Java HLRC. This commit also refactors GetMappingsResponse to follow the same appraoch as the other mappings-related responses, where we read include_type_name out of the xContent params, instead of creating a second toXContent method. This gives better consistency in the response parsing code. * Fix more REST tests. * Improve some wording in the create index documentation. * Add a note about types removal in the create index docs. * Fix SmokeTestMonitoringWithSecurityIT#testHTTPExporterWithSSL. * Make sure to mention include_type_name in the REST docs for affected APIs. * Make sure to use 'expression == false' in FullClusterRestartIT. * Mention include_type_name in the REST templates docs.
141 lines
4.2 KiB
Plaintext
141 lines
4.2 KiB
Plaintext
[[dynamic-field-mapping]]
|
|
=== Dynamic field mapping
|
|
|
|
By default, when a previously unseen field is found in a document,
|
|
Elasticsearch will add the new field to the type mapping. This behaviour can
|
|
be disabled, both at the document and at the <<object,`object`>> level, by
|
|
setting the <<dynamic,`dynamic`>> parameter to `false` (to ignore new fields) or to `strict` (to throw
|
|
an exception if an unknown field is encountered).
|
|
|
|
Assuming `dynamic` field mapping is enabled, some simple rules are used to
|
|
determine which datatype the field should have:
|
|
|
|
[horizontal]
|
|
*JSON datatype*:: *Elasticsearch datatype*
|
|
|
|
`null`:: No field is added.
|
|
`true` or `false`:: <<boolean,`boolean`>> field
|
|
floating{nbsp}point{nbsp}number:: <<number,`float`>> field
|
|
integer:: <<number,`long`>> field
|
|
object:: <<object,`object`>> field
|
|
array:: Depends on the first non-`null` value in the array.
|
|
string:: Either a <<date,`date`>> field
|
|
(if the value passes <<date-detection,date detection>>),
|
|
a <<number,`double`>> or <<number,`long`>> field
|
|
(if the value passes <<numeric-detection,numeric detection>>)
|
|
or a <<text,`text`>> field, with a <<keyword,`keyword`>> sub-field.
|
|
|
|
These are the only <<mapping-types,field datatypes>> that are dynamically
|
|
detected. All other datatypes must be mapped explicitly.
|
|
|
|
Besides the options listed below, dynamic field mapping rules can be further
|
|
customised with <<dynamic-templates,`dynamic_templates`>>.
|
|
|
|
[[date-detection]]
|
|
==== Date detection
|
|
|
|
If `date_detection` is enabled (default), then new string fields are checked
|
|
to see whether their contents match any of the date patterns specified in
|
|
`dynamic_date_formats`. If a match is found, a new <<date,`date`>> field is
|
|
added with the corresponding format.
|
|
|
|
The default value for `dynamic_date_formats` is:
|
|
|
|
[ <<strict-date-time,`"strict_date_optional_time"`>>,`"yyyy/MM/dd HH:mm:ss Z||yyyy/MM/dd Z"`]
|
|
|
|
For example:
|
|
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT my_index/_doc/1
|
|
{
|
|
"create_date": "2015/09/02"
|
|
}
|
|
|
|
GET my_index/_mapping <1>
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
<1> The `create_date` field has been added as a <<date,`date`>>
|
|
field with the <<mapping-date-format,`format`>>: +
|
|
`"yyyy/MM/dd HH:mm:ss Z||yyyy/MM/dd Z"`.
|
|
|
|
===== Disabling date detection
|
|
|
|
Dynamic date detection can be disabled by setting `date_detection` to `false`:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT my_index?include_type_name=true
|
|
{
|
|
"mappings": {
|
|
"_doc": {
|
|
"date_detection": false
|
|
}
|
|
}
|
|
}
|
|
|
|
PUT my_index/_doc/1 <1>
|
|
{
|
|
"create": "2015/09/02"
|
|
}
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
|
|
<1> The `create_date` field has been added as a <<text,`text`>> field.
|
|
|
|
===== Customising detected date formats
|
|
|
|
Alternatively, the `dynamic_date_formats` can be customised to support your
|
|
own <<mapping-date-format,date formats>>:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT my_index?include_type_name=true
|
|
{
|
|
"mappings": {
|
|
"_doc": {
|
|
"dynamic_date_formats": ["MM/dd/yyyy"]
|
|
}
|
|
}
|
|
}
|
|
|
|
PUT my_index/_doc/1
|
|
{
|
|
"create_date": "09/25/2015"
|
|
}
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
|
|
|
|
[[numeric-detection]]
|
|
==== Numeric detection
|
|
|
|
While JSON has support for native floating point and integer datatypes, some
|
|
applications or languages may sometimes render numbers as strings. Usually the
|
|
correct solution is to map these fields explicitly, but numeric detection
|
|
(which is disabled by default) can be enabled to do this automatically:
|
|
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT my_index?include_type_name=true
|
|
{
|
|
"mappings": {
|
|
"_doc": {
|
|
"numeric_detection": true
|
|
}
|
|
}
|
|
}
|
|
|
|
PUT my_index/_doc/1
|
|
{
|
|
"my_float": "1.0", <1>
|
|
"my_integer": "1" <2>
|
|
}
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
<1> The `my_float` field is added as a <<number,`float`>> field.
|
|
<2> The `my_integer` field is added as a <<number,`long`>> field.
|
|
|