123 lines
3.0 KiB
Plaintext
123 lines
3.0 KiB
Plaintext
[[dot-expand-processor]]
|
|
=== Dot expander processor
|
|
++++
|
|
<titleabbrev>Dot expander</titleabbrev>
|
|
++++
|
|
|
|
Expands a field with dots into an object field. This processor allows fields
|
|
with dots in the name to be accessible by other processors in the pipeline.
|
|
Otherwise these <<accessing-data-in-pipelines,fields>> can't be accessed by any processor.
|
|
|
|
[[dot-expander-options]]
|
|
.Dot Expand Options
|
|
[options="header"]
|
|
|======
|
|
| Name | Required | Default | Description
|
|
| `field` | yes | - | The field to expand into an object field
|
|
| `path` | no | - | The field that contains the field to expand. Only required if the field to expand is part another object field, because the `field` option can only understand leaf fields.
|
|
include::common-options.asciidoc[]
|
|
|======
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"dot_expander": {
|
|
"field": "foo.bar"
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
For example the dot expand processor would turn this document:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foo.bar" : "value"
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
into:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foo" : {
|
|
"bar" : "value"
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
If there is already a `bar` field nested under `foo` then
|
|
this processor merges the `foo.bar` field into it. If the field is
|
|
a scalar value then it will turn that field into an array field.
|
|
|
|
For example, the following document:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foo.bar" : "value2",
|
|
"foo" : {
|
|
"bar" : "value1"
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
is transformed by the `dot_expander` processor into:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foo" : {
|
|
"bar" : ["value1", "value2"]
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
If any field outside of the leaf field conflicts with a pre-existing field of the same name,
|
|
then that field needs to be renamed first.
|
|
|
|
Consider the following document:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foo": "value1",
|
|
"foo.bar": "value2"
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
Then the `foo` needs to be renamed first before the `dot_expander`
|
|
processor is applied. So in order for the `foo.bar` field to properly
|
|
be expanded into the `bar` field under the `foo` field the following
|
|
pipeline should be used:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"processors" : [
|
|
{
|
|
"rename" : {
|
|
"field" : "foo",
|
|
"target_field" : "foo.bar""
|
|
}
|
|
},
|
|
{
|
|
"dot_expander": {
|
|
"field": "foo.bar"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
The reason for this is that Ingest doesn't know how to automatically cast
|
|
a scalar field to an object field.
|