honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

ingest: doc: move Dot Expander Processor doc to correct position (#31743) 

No changes to the content.
This commit is contained in:
Jake Landis 2018-08-03 07:21:05 -07:00 committed by GitHub
parent bc274b2ff2
commit 3d4c84f7ca
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 119 additions and 119 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

238
docs/reference/ingest/ingest-node.asciidoc
View File

@ -1049,6 +1049,125 @@ understands this to mean `2016-04-01` as is explained in the <<date-math-index-n
| `index_name_format` | no | yyyy-MM-dd | The format to be used when printing the parsed date into the index name. An valid Joda pattern is expected here. | `index_name_format` | no | yyyy-MM-dd | The format to be used when printing the parsed date into the index name. An valid Joda pattern is expected here.
|====== |======
[[dot-expand-processor]]
=== Dot Expander Processor
Expands a field with dots into an object field. This processor allows fields
with dots in the name to be accessible by other processors in the pipeline.
Otherwise these <<accessing-data-in-pipelines,fields>> can't be accessed by any processor.
[[dot-expender-options]]
.Dot Expand Options
[options="header"]
|======
| Name | Required | Default | Description
| `field` | yes | - | The field to expand into an object field
| `path` | no | - | The field that contains the field to expand. Only required if the field to expand is part another object field, because the `field` option can only understand leaf fields.
|======
[source,js]
--------------------------------------------------
{
"dot_expander": {
"field": "foo.bar"
}
}
--------------------------------------------------
// NOTCONSOLE
For example the dot expand processor would turn this document:
[source,js]
--------------------------------------------------
{
"foo.bar" : "value"
}
--------------------------------------------------
// NOTCONSOLE
into:
[source,js]
--------------------------------------------------
{
"foo" : {
"bar" : "value"
}
}
--------------------------------------------------
// NOTCONSOLE
If there is already a `bar` field nested under `foo` then
this processor merges the `foo.bar` field into it. If the field is
a scalar value then it will turn that field into an array field.
For example, the following document:
[source,js]
--------------------------------------------------
{
"foo.bar" : "value2",
"foo" : {
"bar" : "value1"
}
}
--------------------------------------------------
// NOTCONSOLE
is transformed by the `dot_expander` processor into:
[source,js]
--------------------------------------------------
{
"foo" : {
"bar" : ["value1", "value2"]
}
}
--------------------------------------------------
// NOTCONSOLE
If any field outside of the leaf field conflicts with a pre-existing field of the same name,
then that field needs to be renamed first.
Consider the following document:
[source,js]
--------------------------------------------------
{
"foo": "value1",
"foo.bar": "value2"
}
--------------------------------------------------
// NOTCONSOLE
Then the `foo` needs to be renamed first before the `dot_expander`
processor is applied. So in order for the `foo.bar` field to properly
be expanded into the `bar` field under the `foo` field the following
pipeline should be used:
[source,js]
--------------------------------------------------
{
"processors" : [
{
"rename" : {
"field" : "foo",
"target_field" : "foo.bar""
}
},
{
"dot_expander": {
"field": "foo.bar"
}
}
]
}
--------------------------------------------------
// NOTCONSOLE
The reason for this is that Ingest doesn't know how to automatically cast
a scalar field to an object field.
[[fail-processor]] [[fail-processor]]
=== Fail Processor === Fail Processor
Raises an exception. This is useful for when Raises an exception. This is useful for when
@ -2058,125 +2177,6 @@ Converts a string to its uppercase equivalent.
-------------------------------------------------- --------------------------------------------------
// NOTCONSOLE // NOTCONSOLE
[[dot-expand-processor]]
=== Dot Expander Processor
Expands a field with dots into an object field. This processor allows fields
with dots in the name to be accessible by other processors in the pipeline.
Otherwise these <<accessing-data-in-pipelines,fields>> can't be accessed by any processor.
[[dot-expender-options]]
.Dot Expand Options
[options="header"]
|======
| Name | Required | Default | Description
| `field` | yes | - | The field to expand into an object field
| `path` | no | - | The field that contains the field to expand. Only required if the field to expand is part another object field, because the `field` option can only understand leaf fields.
|======
[source,js]
--------------------------------------------------
{
"dot_expander": {
"field": "foo.bar"
}
}
--------------------------------------------------
// NOTCONSOLE
For example the dot expand processor would turn this document:
[source,js]
--------------------------------------------------
{
"foo.bar" : "value"
}
--------------------------------------------------
// NOTCONSOLE
into:
[source,js]
--------------------------------------------------
{
"foo" : {
"bar" : "value"
}
}
--------------------------------------------------
// NOTCONSOLE
If there is already a `bar` field nested under `foo` then
this processor merges the `foo.bar` field into it. If the field is
a scalar value then it will turn that field into an array field.
For example, the following document:
[source,js]
--------------------------------------------------
{
"foo.bar" : "value2",
"foo" : {
"bar" : "value1"
}
}
--------------------------------------------------
// NOTCONSOLE
is transformed by the `dot_expander` processor into:
[source,js]
--------------------------------------------------
{
"foo" : {
"bar" : ["value1", "value2"]
}
}
--------------------------------------------------
// NOTCONSOLE
If any field outside of the leaf field conflicts with a pre-existing field of the same name,
then that field needs to be renamed first.
Consider the following document:
[source,js]
--------------------------------------------------
{
"foo": "value1",
"foo.bar": "value2"
}
--------------------------------------------------
// NOTCONSOLE
Then the `foo` needs to be renamed first before the `dot_expander`
processor is applied. So in order for the `foo.bar` field to properly
be expanded into the `bar` field under the `foo` field the following
pipeline should be used:
[source,js]
--------------------------------------------------
{
"processors" : [
{
"rename" : {
"field" : "foo",
"target_field" : "foo.bar""
}
},
{
"dot_expander": {
"field": "foo.bar"
}
}
]
}
--------------------------------------------------
// NOTCONSOLE
The reason for this is that Ingest doesn't know how to automatically cast
a scalar field to an object field.
[[urldecode-processor]] [[urldecode-processor]]
=== URL Decode Processor === URL Decode Processor
URL-decodes a string URL-decodes a string