opensearch-docs-cn/_data-prepper/pipelines/configuration/processors/key-value.md

45 lines
8.4 KiB
Markdown
Raw Normal View History

Restructure Data Prepper plugins documentation (#2073) * Removed content from Data Prepper reference and broke out into separate pages. Signed-off-by: carolxob <carolxob@amazon.com> * Checking in file to make sure it's the right version. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update. Signed-off-by: carolxob <carolxob@amazon.com> * Updated files. Signed-off-by: carolxob <carolxob@amazon.com> * Adding Sinks file. Signed-off-by: carolxob <carolxob@amazon.com> * Added file to PR. Signed-off-by: carolxob <carolxob@amazon.com> * Corrected TOC hierarchy. Signed-off-by: carolxob <carolxob@amazon.com> * Added images and reorganized files. Signed-off-by: carolxob <carolxob@amazon.com> * Reconfigured some content based on David's feedback. Signed-off-by: carolxob <carolxob@amazon.com> * Modified reference page. Signed-off-by: carolxob <carolxob@amazon.com> * Fixed minor heading issue. Signed-off-by: carolxob <carolxob@amazon.com> * Minor edits. Signed-off-by: carolxob <carolxob@amazon.com> * Major edits, files created, moved, and content broken out from main config page. Signed-off-by: carolxob <carolxob@amazon.com> * Adding key value and processors pages to the PR. Signed-off-by: carolxob <carolxob@amazon.com> * Basic ToC reorg. Signed-off-by: carolxob <carolxob@amazon.com> * ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update.: Signed-off-by: carolxob <carolxob@amazon.com> * Minor edits to ToC again. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor TOC update Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC edits. Signed-off-by: carolxob <carolxob@amazon.com> * Changed filename. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates Signed-off-by: carolxob <carolxob@amazon.com> * Making small Toc changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Added comment blocks for Config and Metrics sections. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates to add Sinks and Sources under config guide. Signed-off-by: carolxob <carolxob@amazon.com> Signed-off-by: carolxob <carolxob@amazon.com>
2022-12-27 17:58:48 -05:00
---
layout: default
Add DLQ and aggregate processor edits. Baseline processor names (#3880) * Add DLQ and aggregate processor edits. Baseline processor names Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Remove processor from titles Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update dlq.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Add additional feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * add cross links Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Fix typos Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add doc review Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Remove link Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update dlq.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Fix processor title Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-27 15:01:01 -04:00
title: key_value
Restructure Data Prepper plugins documentation (#2073) * Removed content from Data Prepper reference and broke out into separate pages. Signed-off-by: carolxob <carolxob@amazon.com> * Checking in file to make sure it's the right version. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update. Signed-off-by: carolxob <carolxob@amazon.com> * Updated files. Signed-off-by: carolxob <carolxob@amazon.com> * Adding Sinks file. Signed-off-by: carolxob <carolxob@amazon.com> * Added file to PR. Signed-off-by: carolxob <carolxob@amazon.com> * Corrected TOC hierarchy. Signed-off-by: carolxob <carolxob@amazon.com> * Added images and reorganized files. Signed-off-by: carolxob <carolxob@amazon.com> * Reconfigured some content based on David's feedback. Signed-off-by: carolxob <carolxob@amazon.com> * Modified reference page. Signed-off-by: carolxob <carolxob@amazon.com> * Fixed minor heading issue. Signed-off-by: carolxob <carolxob@amazon.com> * Minor edits. Signed-off-by: carolxob <carolxob@amazon.com> * Major edits, files created, moved, and content broken out from main config page. Signed-off-by: carolxob <carolxob@amazon.com> * Adding key value and processors pages to the PR. Signed-off-by: carolxob <carolxob@amazon.com> * Basic ToC reorg. Signed-off-by: carolxob <carolxob@amazon.com> * ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update.: Signed-off-by: carolxob <carolxob@amazon.com> * Minor edits to ToC again. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor TOC update Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC edits. Signed-off-by: carolxob <carolxob@amazon.com> * Changed filename. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates Signed-off-by: carolxob <carolxob@amazon.com> * Making small Toc changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Added comment blocks for Config and Metrics sections. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates to add Sinks and Sources under config guide. Signed-off-by: carolxob <carolxob@amazon.com> Signed-off-by: carolxob <carolxob@amazon.com>
2022-12-27 17:58:48 -05:00
parent: Processors
Data Prepper ToC Update (#2514) * Creating PR with first file. Signed-off-by: carolxob <carolxob@amazon.com> * Adding newly created files to PR. Signed-off-by: carolxob <carolxob@amazon.com> * Reorganized files and added appropriate metadata to map ToC correctly. Signed-off-by: carolxob <carolxob@amazon.com> * Moved Authoring pipelines page. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates to Sources section for Data Prepper. Signed-off-by: carolxob <carolxob@amazon.com> * Updated Buffers section under Data Prepper. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update to otelmetricssource. Signed-off-by: carolxob <carolxob@amazon.com> * Restructured ToC in Processors section for Data Prepper. Signed-off-by: carolxob <carolxob@amazon.com> * Minor filename change. Signed-off-by: carolxob <carolxob@amazon.com> * Adjustments to metadata in ToC. Signed-off-by: carolxob <carolxob@amazon.com> * Minor edit. Signed-off-by: carolxob <carolxob@amazon.com> * Fixed nav order in metadata. Signed-off-by: carolxob <carolxob@amazon.com> * Minor edit. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update top metadata for ToC. Signed-off-by: carolxob <carolxob@amazon.com> * Adjustmenets to Toc order. Signed-off-by: carolxob <carolxob@amazon.com> * Minor adjustments to ToC metadata. Signed-off-by: carolxob <carolxob@amazon.com> * Minor adjustments to Sinks section. Signed-off-by: carolxob <carolxob@amazon.com> * Adjustements to high level ToC. Signed-off-by: carolxob <carolxob@amazon.com> * Minor adjustement to Pipelines.md Signed-off-by: carolxob <carolxob@amazon.com> * Minor update. Signed-off-by: carolxob <carolxob@amazon.com> * Slight reorganization. Removed two placeholder pages for now. Signed-off-by: carolxob <carolxob@amazon.com> * Removed a page and replaced with pipelines content. Signed-off-by: carolxob <carolxob@amazon.com> * Minor changes/additions to content for placeholder pages. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update to page link. Signed-off-by: carolxob <carolxob@amazon.com> * Minor adjustments to ToC metadata. Signed-off-by: carolxob <carolxob@amazon.com> * Minor edits. Signed-off-by: carolxob <carolxob@amazon.com> * Removed /clients from redirects to correct nav order. Signed-off-by: carolxob <carolxob@amazon.com> * Minor edits. Signed-off-by: carolxob <carolxob@amazon.com> * Minor adjustments to ToC metadata. Signed-off-by: carolxob <carolxob@amazon.com> * Minor adjustments. Signed-off-by: carolxob <carolxob@amazon.com> * Minor adjustment ot metadata. Signed-off-by: carolxob <carolxob@amazon.com> * TOC link fixes Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Changed page name. Signed-off-by: carolxob <carolxob@amazon.com> * Corrected references to Peer Forwarder. Signed-off-by: carolxob <carolxob@amazon.com> * Renamed Data Prepper folder. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates to phrasing and capitalization. Signed-off-by: carolxob <carolxob@amazon.com> * Minor phrasing update. Signed-off-by: carolxob <carolxob@amazon.com> * Minor phrasing update. Signed-off-by: carolxob <carolxob@amazon.com> * Minor change. Signed-off-by: carolxob <carolxob@amazon.com> * Minor change to change S3 Source to S3Source. Signed-off-by: carolxob <carolxob@amazon.com> * Updated references to peer forwarder and changed capitalization. Signed-off-by: carolxob <carolxob@amazon.com> * Updated capitalization for peer forwarder. Signed-off-by: carolxob <carolxob@amazon.com> * Made edits based on doc review feedback. Signed-off-by: carolxob <carolxob@amazon.com> * Update to one word. Signed-off-by: carolxob <carolxob@amazon.com> --------- Signed-off-by: carolxob <carolxob@amazon.com> Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Co-authored-by: Naarcha-AWS <naarcha@amazon.com>
2023-02-03 17:06:10 -05:00
grand_parent: Pipelines
nav_order: 56
Restructure Data Prepper plugins documentation (#2073) * Removed content from Data Prepper reference and broke out into separate pages. Signed-off-by: carolxob <carolxob@amazon.com> * Checking in file to make sure it's the right version. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update. Signed-off-by: carolxob <carolxob@amazon.com> * Updated files. Signed-off-by: carolxob <carolxob@amazon.com> * Adding Sinks file. Signed-off-by: carolxob <carolxob@amazon.com> * Added file to PR. Signed-off-by: carolxob <carolxob@amazon.com> * Corrected TOC hierarchy. Signed-off-by: carolxob <carolxob@amazon.com> * Added images and reorganized files. Signed-off-by: carolxob <carolxob@amazon.com> * Reconfigured some content based on David's feedback. Signed-off-by: carolxob <carolxob@amazon.com> * Modified reference page. Signed-off-by: carolxob <carolxob@amazon.com> * Fixed minor heading issue. Signed-off-by: carolxob <carolxob@amazon.com> * Minor edits. Signed-off-by: carolxob <carolxob@amazon.com> * Major edits, files created, moved, and content broken out from main config page. Signed-off-by: carolxob <carolxob@amazon.com> * Adding key value and processors pages to the PR. Signed-off-by: carolxob <carolxob@amazon.com> * Basic ToC reorg. Signed-off-by: carolxob <carolxob@amazon.com> * ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update.: Signed-off-by: carolxob <carolxob@amazon.com> * Minor edits to ToC again. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor TOC update Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC edits. Signed-off-by: carolxob <carolxob@amazon.com> * Changed filename. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates Signed-off-by: carolxob <carolxob@amazon.com> * Making small Toc changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Added comment blocks for Config and Metrics sections. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates to add Sinks and Sources under config guide. Signed-off-by: carolxob <carolxob@amazon.com> Signed-off-by: carolxob <carolxob@amazon.com>
2022-12-27 17:58:48 -05:00
---
# key_value
Add examples to Key value page (#3105) * Created PR to add examples to Key value doc. Signed-off-by: carolxob <carolxob@amazon.com> * Minor adjustements. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update to table formatting. Signed-off-by: carolxob <carolxob@amazon.com> * Minor change. Signed-off-by: carolxob <carolxob@amazon.com> * Corrected table formatting. Signed-off-by: carolxob <carolxob@amazon.com> * Added examples field table data. Signed-off-by: carolxob <carolxob@amazon.com> * Minor phrasing change. Signed-off-by: carolxob <carolxob@amazon.com> * Updated regex definition. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates to intro. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates to intro. Signed-off-by: carolxob <carolxob@amazon.com> * Added default value column.? Signed-off-by: carolxob <carolxob@amazon.com> * Edits based on doc review feedback. Signed-off-by: carolxob <carolxob@amazon.com> * Minor changes. Signed-off-by: carolxob <carolxob@amazon.com> * Incorporated doc review feedback. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates. Signed-off-by: carolxob <carolxob@amazon.com> * Major modifications to table formatting. Signed-off-by: carolxob <carolxob@amazon.com> * Updates to phrasing for examples for clarity. Signed-off-by: carolxob <carolxob@amazon.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Minor update to content. Signed-off-by: carolxob <carolxob@amazon.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com> --------- Signed-off-by: carolxob <carolxob@amazon.com> Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-25 11:07:10 -04:00
You can use the `key_value` processor to parse the specified field into key-value pairs. You can customize the `key_value` processor to parse field information with the following options. The type for each of the following options is `string`.
| Option | Description | Example |
| :--- | :--- | :--- |
| source | The message field to be parsed. Optional. Default value is `message`. | If `source` is `"message1"`, `{"message1": {"key1=value1"}, "message2": {"key2=value2"}}` parses into `{"message1": {"key1=value1"}, "message2": {"key2=value2"}, "parsed_message": {"key1": "value1"}}`. |
| destination | The destination field for the parsed source. The parsed source overwrites the preexisting data for that key. Optional. If `destination` is set to `null`, the parsed fields will be written to the root of the event. Default value is `parsed_message`. | If `destination` is `"parsed_data"`, `{"message": {"key1=value1"}}` parses into `{"message": {"key1=value1"}, "parsed_data": {"key1": "value1"}}`. |
Add examples to Key value page (#3105) * Created PR to add examples to Key value doc. Signed-off-by: carolxob <carolxob@amazon.com> * Minor adjustements. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update to table formatting. Signed-off-by: carolxob <carolxob@amazon.com> * Minor change. Signed-off-by: carolxob <carolxob@amazon.com> * Corrected table formatting. Signed-off-by: carolxob <carolxob@amazon.com> * Added examples field table data. Signed-off-by: carolxob <carolxob@amazon.com> * Minor phrasing change. Signed-off-by: carolxob <carolxob@amazon.com> * Updated regex definition. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates to intro. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates to intro. Signed-off-by: carolxob <carolxob@amazon.com> * Added default value column.? Signed-off-by: carolxob <carolxob@amazon.com> * Edits based on doc review feedback. Signed-off-by: carolxob <carolxob@amazon.com> * Minor changes. Signed-off-by: carolxob <carolxob@amazon.com> * Incorporated doc review feedback. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates. Signed-off-by: carolxob <carolxob@amazon.com> * Major modifications to table formatting. Signed-off-by: carolxob <carolxob@amazon.com> * Updates to phrasing for examples for clarity. Signed-off-by: carolxob <carolxob@amazon.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Minor update to content. Signed-off-by: carolxob <carolxob@amazon.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com> * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com> --------- Signed-off-by: carolxob <carolxob@amazon.com> Signed-off-by: Caroline <113052567+carolxob@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-04-25 11:07:10 -04:00
| field_delimiter_regex | A regular expression specifying the delimiter that separates key-value pairs. Special regular expression characters such as `[` and `]` must be escaped with `\\`. Cannot be defined at the same time as `field_split_characters`. Optional. If this option is not defined, `field_split_characters` is used. | If `field_delimiter_regex` is `"&\\{2\\}"`, `{"key1=value1&&key2=value2"}` parses into `{"key1": "value1", "key2": "value2"}`. |
| field_split_characters | A string of characters specifying the delimeter that separates key-value pairs. Special regular expression characters such as `[` and `]` must be escaped with `\\`. Cannot be defined at the same time as `field_delimiter_regex`. Optional. Default value is `&`. | If `field_split_characters` is `"&&"`, `{"key1=value1&&key2=value2"}` parses into `{"key1": "value1", "key2": "value2"}`. |
| key_value_delimiter_regex | A regular expression specifying the delimiter that separates the key and value within a key-value pair. Special regular expression characters such as `[` and `]` must be escaped with `\\`. This option cannot be defined at the same time as `value_split_characters`. Optional. If this option is not defined, `value_split_characters` is used. | If `key_value_delimiter_regex` is `"=\\{2\\}"`, `{"key1==value1"}` parses into `{"key1": "value1"}`. |
| value_split_characters | A string of characters specifying the delimiter that separates the key and value within a key-value pair. Special regular expression characters such as `[` and `]` must be escaped with `\\`. Cannot be defined at the same time as `key_value_delimiter_regex`. Optional. Default value is `=`. | If `value_split_characters` is `"=="`, `{"key1==value1"}` parses into `{"key1": "value1"}`. |
| non_match_value | When a key-value pair cannot be successfully split, the key-value pair is placed in the `key` field, and the specified value is placed in the `value` field. Optional. Default value is `null`. | `key1value1&key2=value2` parses into `{"key1value1": null, "key2": "value2"}`. |
| prefix | A prefix to append before all keys. Optional. Default value is an empty string. | If `prefix` is `"custom"`, `{"key1=value1"}` parses into `{"customkey1": "value1"}`.|
| delete_key_regex | A regular expression specifying the characters to delete from the key. Special regular expression characters such as `[` and `]` must be escaped with `\\`. Cannot be an empty string. Optional. No default value. | If `delete_key_regex` is `"\s"`, `{"key1 =value1"}` parses into `{"key1": "value1"}`. |
| delete_value_regex | A regular expression specifying the characters to delete from the value. Special regular expression characters such as `[` and `]` must be escaped with `\\`. Cannot be an empty string. Optional. No default value. | If `delete_value_regex` is `"\s"`, `{"key1=value1 "}` parses into `{"key1": "value1"}`. |
| include_keys | An array specifying the keys that should be added for parsing. By default, all keys will be added. | If `include_keys` is `["key2"]`,`key1=value1&key2=value2` will parse into `{"key2": "value2"}`. |
| exclude_keys | An array specifying the parsed keys that should not be added to the event. By default, no keys will be excluded. | If `exclude_keys` is `["key2"]`, `key1=value1&key2=value2` will parse into `{"key1": "value1"}`. |
| default_values | A map specifying the default keys and their values that should be added to the event in case these keys do not exist in the source field being parsed. If the default key already exists in the message, the value is not changed. The `include_keys` filter will be applied to the message before `default_values`. | If `default_values` is `{"defaultkey": "defaultvalue"}`, `key1=value1` will parse into `{"key1": "value1", "defaultkey": "defaultvalue"}`. <br /> If `default_values` is `{"key1": "abc"}`, `key1=value1` will parse into `{"key1": "value1"}`. <br /> If `include_keys` is `["key1"]` and `default_values` is `{"key2": "value2"}`, `key1=value1&key2=abc` will parse into `{"key1": "value1", "key2": "value2"}`. |
| transform_key | When to lowercase, uppercase, or capitalize keys. | If `transform_key` is `lowercase`, `{"Key1=value1"}` will parse into `{"key1": "value1"}`. <br /> If `transform_key` is `uppercase`, `{"key1=value1"}` will parse into `{"KEY1": "value1"}`. <br /> If `transform_key` is `capitalize`, `{"key1=value1"}` will parse into `{"Key1": "value1"}`. |
| whitespace | Specifies whether to be lenient or strict with the acceptance of unnecessary white space surrounding the configured value-split sequence. Default is `lenient`. | If `whitespace` is `"lenient"`, `{"key1 = value1"}` will parse into `{"key1 ": " value1"}`. If `whitespace` is `"strict"`, `{"key1 = value1"}` will parse into `{"key1": "value1"}`. |
| skip_duplicate_values | A Boolean option for removing duplicate key-value pairs. When set to `true`, only one unique key-value pair will be preserved. Default is `false`. | If `skip_duplicate_values` is `false`, `{"key1=value1&key1=value1"}` will parse into `{"key1": ["value1", "value1"]}`. If `skip_duplicate_values` is `true`, `{"key1=value1&key1=value1"}` will parse into `{"key1": "value1"}`. |
| remove_brackets | Specifies whether to treat square brackets, angle brackets, and parentheses as value "wrappers" that should be removed from the value. Default is `false`. | If `remove_brackets` is `true`, `{"key1=(value1)"}` will parse into `{"key1": value1}`. If `remove_brackets` is `false`, `{"key1=(value1)"}` will parse into `{"key1": "(value1)"}`. |
| recursive | Specifies whether to recursively obtain additional key-value pairs from values. The extra key-value pairs will be stored as sub-keys of the root key. Default is `false`. The levels of recursive parsing must be defined by different brackets for each level: `[]`, `()`, and `<>`, in this order. Any other configurations specified will only be applied to the outmost keys. <br />When `recursive` is `true`: <br /> `remove_brackets` cannot also be `true`;<br />`skip_duplicate_values` will always be `true`; <br />`whitespace` will always be `"strict"`. | If `recursive` is true, `{"item1=[item1-subitem1=item1-subitem1-value&item1-subitem2=(item1-subitem2-subitem2A=item1-subitem2-subitem2A-value&item1-subitem2-subitem2B=item1-subitem2-subitem2B-value)]&item2=item2-value"}` will parse into `{"item1": {"item1-subitem1": "item1-subitem1-value", "item1-subitem2" {"item1-subitem2-subitem2A": "item1-subitem2-subitem2A-value", "item1-subitem2-subitem2B": "item1-subitem2-subitem2B-value"}}}`. |
| overwrite_if_destination_exists | Specifies whether to overwrite existing fields if there are key conflicts when writing parsed fields to the event. Default is `true`. | If `overwrite_if_destination_exists` is `true` and destination is `null`, `{"key1": "old_value", "message": "key1=new_value"}` will parse into `{"key1": "new_value", "message": "key1=new_value"}`. |
| tags_on_failure | When a `kv` operation causes a runtime exception within the processor, the operation is safely stopped without crashing the processor, and the event is tagged with the provided tags. | If `tags_on_failure` is set to `["keyvalueprocessor_failure"]`, `{"tags": ["keyvalueprocessor_failure"]}` will be added to the event's metadata in the event of a runtime exception. |
Restructure Data Prepper plugins documentation (#2073) * Removed content from Data Prepper reference and broke out into separate pages. Signed-off-by: carolxob <carolxob@amazon.com> * Checking in file to make sure it's the right version. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update. Signed-off-by: carolxob <carolxob@amazon.com> * Updated files. Signed-off-by: carolxob <carolxob@amazon.com> * Adding Sinks file. Signed-off-by: carolxob <carolxob@amazon.com> * Added file to PR. Signed-off-by: carolxob <carolxob@amazon.com> * Corrected TOC hierarchy. Signed-off-by: carolxob <carolxob@amazon.com> * Added images and reorganized files. Signed-off-by: carolxob <carolxob@amazon.com> * Reconfigured some content based on David's feedback. Signed-off-by: carolxob <carolxob@amazon.com> * Modified reference page. Signed-off-by: carolxob <carolxob@amazon.com> * Fixed minor heading issue. Signed-off-by: carolxob <carolxob@amazon.com> * Minor edits. Signed-off-by: carolxob <carolxob@amazon.com> * Major edits, files created, moved, and content broken out from main config page. Signed-off-by: carolxob <carolxob@amazon.com> * Adding key value and processors pages to the PR. Signed-off-by: carolxob <carolxob@amazon.com> * Basic ToC reorg. Signed-off-by: carolxob <carolxob@amazon.com> * ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor update.: Signed-off-by: carolxob <carolxob@amazon.com> * Minor edits to ToC again. Signed-off-by: carolxob <carolxob@amazon.com> * Minor updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor TOC update Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC edits. Signed-off-by: carolxob <carolxob@amazon.com> * Changed filename. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates Signed-off-by: carolxob <carolxob@amazon.com> * Making small Toc changes. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates. Signed-off-by: carolxob <carolxob@amazon.com> * Added comment blocks for Config and Metrics sections. Signed-off-by: carolxob <carolxob@amazon.com> * Minor ToC updates to add Sinks and Sources under config guide. Signed-off-by: carolxob <carolxob@amazon.com> Signed-off-by: carolxob <carolxob@amazon.com>
2022-12-27 17:58:48 -05:00
<!--- ## Configuration
Content will be added to this section.
## Metrics
Content will be added to this section. --->