mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-02-26 14:54:56 +00:00
d684119618
Together with types removal, any mention of "fields with the same name in the same index" doesn't make sense anymore. (cherry picked from commit c5190106cbd4c007945156249cce462956933326)
106 lines
3.1 KiB
Plaintext
106 lines
3.1 KiB
Plaintext
[[ignore-malformed]]
|
|
=== `ignore_malformed`
|
|
|
|
Sometimes you don't have much control over the data that you receive. One
|
|
user may send a `login` field that is a <<date,`date`>>, and another sends a
|
|
`login` field that is an email address.
|
|
|
|
Trying to index the wrong datatype into a field throws an exception by
|
|
default, and rejects the whole document. The `ignore_malformed` parameter, if
|
|
set to `true`, allows the exception to be ignored. The malformed field is not
|
|
indexed, but other fields in the document are processed normally.
|
|
|
|
For example:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT my_index
|
|
{
|
|
"mappings": {
|
|
"properties": {
|
|
"number_one": {
|
|
"type": "integer",
|
|
"ignore_malformed": true
|
|
},
|
|
"number_two": {
|
|
"type": "integer"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
PUT my_index/_doc/1
|
|
{
|
|
"text": "Some text value",
|
|
"number_one": "foo" <1>
|
|
}
|
|
|
|
PUT my_index/_doc/2
|
|
{
|
|
"text": "Some text value",
|
|
"number_two": "foo" <2>
|
|
}
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
// TEST[catch:bad_request]
|
|
<1> This document will have the `text` field indexed, but not the `number_one` field.
|
|
<2> This document will be rejected because `number_two` does not allow malformed values.
|
|
|
|
TIP: The `ignore_malformed` setting value can be updated on
|
|
existing fields using the <<indices-put-mapping,PUT mapping API>>.
|
|
|
|
|
|
[[ignore-malformed-setting]]
|
|
==== Index-level default
|
|
|
|
The `index.mapping.ignore_malformed` setting can be set on the index level to
|
|
allow to ignore malformed content globally across all mapping types.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT my_index
|
|
{
|
|
"settings": {
|
|
"index.mapping.ignore_malformed": true <1>
|
|
},
|
|
"mappings": {
|
|
"properties": {
|
|
"number_one": { <1>
|
|
"type": "byte"
|
|
},
|
|
"number_two": {
|
|
"type": "integer",
|
|
"ignore_malformed": false <2>
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
|
|
<1> The `number_one` field inherits the index-level setting.
|
|
<2> The `number_two` field overrides the index-level setting to turn off `ignore_malformed`.
|
|
|
|
==== Dealing with malformed fields
|
|
|
|
Malformed fields are silently ignored at indexing time when `ignore_malformed`
|
|
is turned on. Whenever possible it is recommended to keep the number of
|
|
documents that have a malformed field contained, or queries on this field will
|
|
become meaningless. Elasticsearch makes it easy to check how many documents
|
|
have malformed fields by using `exist` or `term` queries on the special
|
|
<<mapping-ignored-field,`_ignored`>> field.
|
|
|
|
[[json-object-limits]]
|
|
==== Limits for JSON Objects
|
|
You can't use `ignore_malformed` with the following datatypes:
|
|
|
|
* <<nested, Nested datatype>>
|
|
* <<object, Object datatype>>
|
|
* <<range, Range datatypes>>
|
|
|
|
You also can't use `ignore_malformed` to ignore JSON objects submitted to fields
|
|
of the wrong datatype. A JSON object is any data surrounded by curly brackets
|
|
`"{}"` and includes data mapped to the nested, object, and range datatypes.
|
|
|
|
If you submit a JSON object to an unsupported field, {es} will return an error
|
|
and reject the entire document regardless of the `ignore_malformed` setting. |