OpenSearch/docs/reference/ingest/processors/dot-expand.asciidoc

[[dot-expand-processor]]
=== Dot Expander Processor

Expands a field with dots into an object field. This processor allows fields
with dots in the name to be accessible by other processors in the pipeline.
Otherwise these <<accessing-data-in-pipelines,fields>> can't be accessed by any processor.

[[dot-expander-options]]
.Dot Expand Options
[options="header"]
|======
| Name     | Required  | Default  | Description
| `field`  | yes       | -        | The field to expand into an object field
| `path`   | no        | -        | The field that contains the field to expand. Only required if the field to expand is part another object field, because the `field` option can only understand leaf fields.
include::common-options.asciidoc[]
|======

[source,js]
--------------------------------------------------
{
  "dot_expander": {
    "field": "foo.bar"
  }
}
--------------------------------------------------
// NOTCONSOLE

For example the dot expand processor would turn this document:

[source,js]
--------------------------------------------------
{
  "foo.bar" : "value"
}
--------------------------------------------------
// NOTCONSOLE

into:

[source,js]
--------------------------------------------------
{
  "foo" : {
    "bar" : "value"
  }
}
--------------------------------------------------
// NOTCONSOLE

If there is already a `bar` field nested under `foo` then
this processor merges the `foo.bar` field into it. If the field is
a scalar value then it will turn that field into an array field.

For example, the following document:

[source,js]
--------------------------------------------------
{
  "foo.bar" : "value2",
  "foo" : {
    "bar" : "value1"
  }
}
--------------------------------------------------
// NOTCONSOLE

is transformed by the `dot_expander` processor into:

[source,js]
--------------------------------------------------
{
  "foo" : {
    "bar" : ["value1", "value2"]
  }
}
--------------------------------------------------
// NOTCONSOLE

If any field outside of the leaf field conflicts with a pre-existing field of the same name,
then that field needs to be renamed first.

Consider the following document:

[source,js]
--------------------------------------------------
{
  "foo": "value1",
  "foo.bar": "value2"
}
--------------------------------------------------
// NOTCONSOLE

Then the `foo` needs to be renamed first before the `dot_expander`
processor is applied. So in order for the `foo.bar` field to properly
be expanded into the `bar` field under the `foo` field the following
pipeline should be used:

[source,js]
--------------------------------------------------
{
  "processors" : [
    {
      "rename" : {
        "field" : "foo",
        "target_field" : "foo.bar""
      }
    },
    {
      "dot_expander": {
        "field": "foo.bar"
      }
    }
  ]
}
--------------------------------------------------
// NOTCONSOLE

The reason for this is that Ingest doesn't know how to automatically cast
a scalar field to an object field.