161 lines
4.6 KiB
Plaintext
161 lines
4.6 KiB
Plaintext
[[foreach-processor]]
|
|
=== Foreach Processor
|
|
|
|
Processes elements in an array of unknown length.
|
|
|
|
All processors can operate on elements inside an array, but if all elements of an array need to
|
|
be processed in the same way, defining a processor for each element becomes cumbersome and tricky
|
|
because it is likely that the number of elements in an array is unknown. For this reason the `foreach`
|
|
processor exists. By specifying the field holding array elements and a processor that
|
|
defines what should happen to each element, array fields can easily be preprocessed.
|
|
|
|
A processor inside the foreach processor works in the array element context and puts that in the ingest metadata
|
|
under the `_ingest._value` key. If the array element is a json object it holds all immediate fields of that json object.
|
|
and if the nested object is a value is `_ingest._value` just holds that value. Note that if a processor prior to the
|
|
`foreach` processor used `_ingest._value` key then the specified value will not be available to the processor inside
|
|
the `foreach` processor. The `foreach` processor does restore the original value, so that value is available to processors
|
|
after the `foreach` processor.
|
|
|
|
Note that any other field from the document are accessible and modifiable like with all other processors. This processor
|
|
just puts the current array element being read into `_ingest._value` ingest metadata attribute, so that it may be
|
|
pre-processed.
|
|
|
|
If the `foreach` processor fails to process an element inside the array, and no `on_failure` processor has been specified,
|
|
then it aborts the execution and leaves the array unmodified.
|
|
|
|
[[foreach-options]]
|
|
.Foreach Options
|
|
[options="header"]
|
|
|======
|
|
| Name | Required | Default | Description
|
|
| `field` | yes | - | The array field
|
|
| `processor` | yes | - | The processor to execute against each field
|
|
| `ignore_missing` | no | false | If `true` and `field` does not exist or is `null`, the processor quietly exits without modifying the document
|
|
include::common-options.asciidoc[]
|
|
|======
|
|
|
|
Assume the following document:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"values" : ["foo", "bar", "baz"]
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
When this `foreach` processor operates on this sample document:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foreach" : {
|
|
"field" : "values",
|
|
"processor" : {
|
|
"uppercase" : {
|
|
"field" : "_ingest._value"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
Then the document will look like this after preprocessing:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"values" : ["FOO", "BAR", "BAZ"]
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
Let's take a look at another example:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"persons" : [
|
|
{
|
|
"id" : "1",
|
|
"name" : "John Doe"
|
|
},
|
|
{
|
|
"id" : "2",
|
|
"name" : "Jane Doe"
|
|
}
|
|
]
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
In this case, the `id` field needs to be removed,
|
|
so the following `foreach` processor is used:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foreach" : {
|
|
"field" : "persons",
|
|
"processor" : {
|
|
"remove" : {
|
|
"field" : "_ingest._value.id"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
After preprocessing the result is:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"persons" : [
|
|
{
|
|
"name" : "John Doe"
|
|
},
|
|
{
|
|
"name" : "Jane Doe"
|
|
}
|
|
]
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
The wrapped processor can have a `on_failure` definition.
|
|
For example, the `id` field may not exist on all person objects.
|
|
Instead of failing the index request, you can use an `on_failure`
|
|
block to send the document to the 'failure_index' index for later inspection:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foreach" : {
|
|
"field" : "persons",
|
|
"processor" : {
|
|
"remove" : {
|
|
"field" : "_value.id",
|
|
"on_failure" : [
|
|
{
|
|
"set" : {
|
|
"field", "_index",
|
|
"value", "failure_index"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
In this example, if the `remove` processor does fail, then
|
|
the array elements that have been processed thus far will
|
|
be updated.
|
|
|
|
Another advanced example can be found in the {plugins}/ingest-attachment-with-arrays.html[attachment processor documentation].
|