Naarcha-AWS 4251fb04bd
Add DLQ and aggregate processor edits. Baseline processor names (#3880)
* Add DLQ and aggregate processor edits. Baseline processor names

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Remove processor from titles

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Update dlq.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Add additional feedback

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* add cross links

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix typos

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add doc review

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Remove link

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update dlq.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Fix processor title

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-27 14:01:01 -05:00

1.4 KiB

layout title parent grand_parent nav_order
default delete_entries Processors Pipelines 51

delete_entries

The delete_entries processor deletes entries, such as key-value pairs, from an event. You can define the keys you want to delete in the with-keys field following delete_entries in the YAML configuration file. Those keys and their values are deleted.

Configuration

You can configure the delete_entries processor with the following options.

Option Required Description
with_keys Yes An array of keys for the entries to be deleted.

Usage

To get started, create the following pipeline.yaml file:

pipeline:
  source:
    ...
  ....  
  processor:
    - delete_entries:
        with_keys: ["message"]
  sink:

{% include copy.html %}

Next, create a log file named logs_json.log and replace the path in the file source of your pipeline.yaml file with that filepath. For more information, see Configuring Data Prepper.

For example, before you run the delete_entries processor, if the logs_json.log file contains the following event record:

{"message": "hello", "message2": "goodbye"}

When you run the delete_entries processor, it parses the message into the following output:

{"message2": "goodbye"}

If message does not exist in the event, then no action occurs.