* Add list to map processor. Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Tweak one last file Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Fix typo Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _data-prepper/pipelines/configuration/processors/list-to-map.md Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/list-to-map.md Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/list-to-map.md Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/list-to-map.md Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com> * Update mutate-event.md * Update _data-prepper/pipelines/configuration/processors/list-to-map.md Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/list-to-map.md Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/list-to-map.md Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/list-to-map.md * Add Chris' feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * A couple more wording tweaks Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _data-prepper/pipelines/configuration/processors/list-to-map.md * Update _data-prepper/pipelines/configuration/processors/list-to-map.md Co-authored-by: Nathan Bower <nbower@amazon.com> * Update _data-prepper/pipelines/configuration/processors/list-to-map.md Co-authored-by: Nathan Bower <nbower@amazon.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> --------- Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com> Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com>
2.0 KiB
layout, title, parent, grand_parent, nav_order
layout | title | parent | grand_parent | nav_order |
---|---|---|---|---|
default | Add entries processor | Processors | Pipelines | 40 |
add_entries
The add_entries
processor adds entries to an event.
Configuration
You can configure the add_entries
processor with the following options.
Option | Required | Description |
---|---|---|
entries |
Yes | A list of entries to add to an event. |
key |
Yes | The key of the new entry to be added. Some examples of keys include my_key , myKey , and object/sub_Key . |
value |
Yes | The value of the new entry to be added. You can use the following data types: strings, Booleans, numbers, null, nested objects, and arrays. |
overwrite_if_key_exists |
No | When set to true , the existing value is overwritten if key already exists in the event. The default value is false . |
Usage
To get started, create the following pipeline.yaml
file:
pipeline:
source:
file:
path: "/full/path/to/logs_json.log"
record_type: "event"
format: "json"
processor:
- add_entries:
entries:
- key: "newMessage"
value: 3
overwrite_if_key_exists: true
sink:
- stdout:
{% include copy.html %}
Next, create a log file named logs_json.log
and replace the path
in the file source of your pipeline.yaml
file with that filepath. For more information, see Configuring Data Prepper.
For example, before you run the add_entries
processor, if the logs_json.log
file contains the following event record:
{"message": "hello"}
Then when you run the add_entries
processor using the previous configuration, it adds a new entry {"newMessage": 3}
to the existing event {"message": "hello"}
so that the new event contains two entries in the final output:
{"message": "hello", "newMessage": 3}
If
newMessage
already exists, its existing value is overwritten with a value of3
.