From 0249991f7693be810a8d152eaa1054c115a87dea Mon Sep 17 00:00:00 2001 From: Caroline <113052567+carolxob@users.noreply.github.com> Date: Fri, 3 Feb 2023 15:06:10 -0700 Subject: [PATCH] Data Prepper ToC Update (#2514) * Creating PR with first file. Signed-off-by: carolxob * Adding newly created files to PR. Signed-off-by: carolxob * Reorganized files and added appropriate metadata to map ToC correctly. Signed-off-by: carolxob * Moved Authoring pipelines page. Signed-off-by: carolxob * Minor ToC updates. Signed-off-by: carolxob * Minor ToC updates to Sources section for Data Prepper. Signed-off-by: carolxob * Updated Buffers section under Data Prepper. Signed-off-by: carolxob * Minor update to otelmetricssource. Signed-off-by: carolxob * Restructured ToC in Processors section for Data Prepper. Signed-off-by: carolxob * Minor filename change. Signed-off-by: carolxob * Adjustments to metadata in ToC. Signed-off-by: carolxob * Minor edit. Signed-off-by: carolxob * Fixed nav order in metadata. Signed-off-by: carolxob * Minor edit. Signed-off-by: carolxob * Minor update top metadata for ToC. Signed-off-by: carolxob * Adjustmenets to Toc order. Signed-off-by: carolxob * Minor adjustments to ToC metadata. Signed-off-by: carolxob * Minor adjustments to Sinks section. Signed-off-by: carolxob * Adjustements to high level ToC. Signed-off-by: carolxob * Minor adjustement to Pipelines.md Signed-off-by: carolxob * Minor update. Signed-off-by: carolxob * Slight reorganization. Removed two placeholder pages for now. Signed-off-by: carolxob * Removed a page and replaced with pipelines content. Signed-off-by: carolxob * Minor changes/additions to content for placeholder pages. Signed-off-by: carolxob * Minor update to page link. Signed-off-by: carolxob * Minor adjustments to ToC metadata. Signed-off-by: carolxob * Minor edits. Signed-off-by: carolxob * Removed /clients from redirects to correct nav order. Signed-off-by: carolxob * Minor edits. Signed-off-by: carolxob * Minor adjustments to ToC metadata. Signed-off-by: carolxob * Minor adjustments. Signed-off-by: carolxob * Minor adjustment ot metadata. Signed-off-by: carolxob * TOC link fixes Signed-off-by: Naarcha-AWS * Changed page name. Signed-off-by: carolxob * Corrected references to Peer Forwarder. Signed-off-by: carolxob * Renamed Data Prepper folder. Signed-off-by: carolxob * Minor updates to phrasing and capitalization. Signed-off-by: carolxob * Minor phrasing update. Signed-off-by: carolxob * Minor phrasing update. Signed-off-by: carolxob * Minor change. Signed-off-by: carolxob * Minor change to change S3 Source to S3Source. Signed-off-by: carolxob * Updated references to peer forwarder and changed capitalization. Signed-off-by: carolxob * Updated capitalization for peer forwarder. Signed-off-by: carolxob * Made edits based on doc review feedback. Signed-off-by: carolxob * Update to one word. Signed-off-by: carolxob --------- Signed-off-by: carolxob Signed-off-by: Naarcha-AWS Co-authored-by: Naarcha-AWS --- .../common-use-cases/common-use-cases.md | 12 ++++++++++ .../{ => common-use-cases}/log-analytics.md | 1 + _data-prepper/getting-started.md | 6 ++--- _data-prepper/index.md | 4 +--- .../configuring-data-prepper.md} | 18 +++++++-------- .../configuring-log4j.md} | 3 ++- .../{ => managing-data-prepper}/core-apis.md | 3 ++- .../managing-data-prepper.md | 10 +++++++++ .../{ => managing-data-prepper}/monitoring.md | 3 ++- _data-prepper/migrate-open-distro.md | 2 +- .../migrating-from-logstash-data-prepper.md | 8 +++---- .../configuration/buffers/bounded-blocking.md | 2 +- .../configuration/buffers/buffers.md | 4 ++-- .../configuration/processors/add-entries.md | 2 +- .../configuration/processors/aggregate.md | 2 +- .../configuration/processors/copy-values.md | 2 +- .../configuration/processors/csv.md | 2 +- .../configuration/processors/date.md | 2 +- .../processors/delete-entries.md | 2 +- .../configuration/processors/drop-events.md | 4 ++-- .../configuration/processors/grok.md | 2 +- .../configuration/processors/json.md | 2 +- .../configuration/processors/key-value.md | 2 +- .../processors/lowercase-string.md | 2 +- .../processors/otel-trace-raw.md | 2 +- .../configuration/processors/processors.md | 8 ++----- .../configuration/processors/rename-keys.md | 2 +- .../configuration/processors/routes.md | 2 +- .../processors/service-map-stateful.md | 2 +- .../configuration/processors/sinks.md | 2 +- .../configuration/processors/split-string.md | 2 +- .../processors/string-converter.md | 2 +- .../processors/substitute-string.md | 2 +- .../configuration/processors/trim-string.md | 2 +- .../processors/uppercase-string.md | 2 +- .../configuration/sinks/file.md | 2 +- .../configuration/sinks/opensearch.md | 2 +- .../configuration/sinks/pipeline.md | 2 +- .../configuration/sinks/sinks.md | 4 ++-- .../configuration/sinks/stdout.md | 2 +- .../configuration/sources/http-source.md | 2 +- .../sources/otel-metrics-source.md | 2 +- .../configuration/sources/otel-trace.md | 2 +- .../configuration/sources/s3.md | 8 +++---- .../configuration/sources/sources.md | 4 ++-- .../{ => pipelines}/expression-syntax.md | 3 ++- .../pipelines-configuration-options.md | 4 ++-- _data-prepper/{ => pipelines}/pipelines.md | 22 +++++++++++-------- _observability-plugin/log-analytics.md | 2 +- 49 files changed, 105 insertions(+), 84 deletions(-) create mode 100644 _data-prepper/common-use-cases/common-use-cases.md rename _data-prepper/{ => common-use-cases}/log-analytics.md (99%) rename _data-prepper/{configuration/data-prepper-configuration.md => managing-data-prepper/configuring-data-prepper.md} (91%) rename _data-prepper/{logs.md => managing-data-prepper/configuring-log4j.md} (96%) rename _data-prepper/{ => managing-data-prepper}/core-apis.md (98%) create mode 100644 _data-prepper/managing-data-prepper/managing-data-prepper.md rename _data-prepper/{ => managing-data-prepper}/monitoring.md (98%) rename _data-prepper/{ => pipelines}/configuration/buffers/bounded-blocking.md (93%) rename _data-prepper/{ => pipelines}/configuration/buffers/buffers.md (82%) rename _data-prepper/{ => pipelines}/configuration/processors/add-entries.md (96%) rename _data-prepper/{ => pipelines}/configuration/processors/aggregate.md (97%) rename _data-prepper/{ => pipelines}/configuration/processors/copy-values.md (95%) rename _data-prepper/{ => pipelines}/configuration/processors/csv.md (97%) rename _data-prepper/{ => pipelines}/configuration/processors/date.md (98%) rename _data-prepper/{ => pipelines}/configuration/processors/delete-entries.md (93%) rename _data-prepper/{ => pipelines}/configuration/processors/drop-events.md (77%) rename _data-prepper/{ => pipelines}/configuration/processors/grok.md (97%) rename _data-prepper/{ => pipelines}/configuration/processors/json.md (96%) rename _data-prepper/{ => pipelines}/configuration/processors/key-value.md (98%) rename _data-prepper/{ => pipelines}/configuration/processors/lowercase-string.md (93%) rename _data-prepper/{ => pipelines}/configuration/processors/otel-trace-raw.md (96%) rename _data-prepper/{ => pipelines}/configuration/processors/processors.md (99%) rename _data-prepper/{ => pipelines}/configuration/processors/rename-keys.md (95%) rename _data-prepper/{ => pipelines}/configuration/processors/routes.md (93%) rename _data-prepper/{ => pipelines}/configuration/processors/service-map-stateful.md (93%) rename _data-prepper/{ => pipelines}/configuration/processors/sinks.md (87%) rename _data-prepper/{ => pipelines}/configuration/processors/split-string.md (96%) rename _data-prepper/{ => pipelines}/configuration/processors/string-converter.md (92%) rename _data-prepper/{ => pipelines}/configuration/processors/substitute-string.md (96%) rename _data-prepper/{ => pipelines}/configuration/processors/trim-string.md (93%) rename _data-prepper/{ => pipelines}/configuration/processors/uppercase-string.md (93%) rename _data-prepper/{ => pipelines}/configuration/sinks/file.md (91%) rename _data-prepper/{ => pipelines}/configuration/sinks/opensearch.md (99%) rename _data-prepper/{ => pipelines}/configuration/sinks/pipeline.md (90%) rename _data-prepper/{ => pipelines}/configuration/sinks/sinks.md (87%) rename _data-prepper/{ => pipelines}/configuration/sinks/stdout.md (89%) rename _data-prepper/{ => pipelines}/configuration/sources/http-source.md (98%) rename _data-prepper/{ => pipelines}/configuration/sources/otel-metrics-source.md (98%) rename _data-prepper/{ => pipelines}/configuration/sources/otel-trace.md (98%) rename _data-prepper/{ => pipelines}/configuration/sources/s3.md (89%) rename _data-prepper/{ => pipelines}/configuration/sources/sources.md (75%) rename _data-prepper/{ => pipelines}/expression-syntax.md (99%) rename _data-prepper/{configuration => pipelines}/pipelines-configuration-options.md (92%) rename _data-prepper/{ => pipelines}/pipelines.md (89%) diff --git a/_data-prepper/common-use-cases/common-use-cases.md b/_data-prepper/common-use-cases/common-use-cases.md new file mode 100644 index 00000000..342a8fc8 --- /dev/null +++ b/_data-prepper/common-use-cases/common-use-cases.md @@ -0,0 +1,12 @@ +--- +layout: default +title: Common use cases +has_children: true +nav_order: 15 +redirect_from: + - /data-prepper/common-use-cases/ +--- + +# Common use cases + +You can use Data Prepper for several different purposes, including trace analytics, log analytics, Amazon S3 log analytics, and metrics ingestion. \ No newline at end of file diff --git a/_data-prepper/log-analytics.md b/_data-prepper/common-use-cases/log-analytics.md similarity index 99% rename from _data-prepper/log-analytics.md rename to _data-prepper/common-use-cases/log-analytics.md index 4a10cd77..54243b6f 100644 --- a/_data-prepper/log-analytics.md +++ b/_data-prepper/common-use-cases/log-analytics.md @@ -1,6 +1,7 @@ --- layout: default title: Log analytics +parent: Common use cases nav_order: 15 --- diff --git a/_data-prepper/getting-started.md b/_data-prepper/getting-started.md index 0d8b1796..4564d010 100644 --- a/_data-prepper/getting-started.md +++ b/_data-prepper/getting-started.md @@ -3,8 +3,6 @@ layout: default title: Getting started nav_order: 5 redirect_from: - - /clients/data-prepper/getting-started/ - - /data-prepper/get-started/ - /clients/data-prepper/get-started/ --- @@ -44,7 +42,7 @@ You will configure two files: Depending on your use case, we have a few different guides to configuring Data Prepper. * [Trace Analytics](https://github.com/opensearch-project/data-prepper/blob/main/docs/trace_analytics.md) -* [Log Ingestion](https://github.com/opensearch-project/data-prepper/blob/main/docs/log_analytics.md): Learn how to set up Data Prepper for log observability. +* [Log Analytics]({{site.url}}{{site.baseurl}}/data-prepper/common-use-cases/log-analytics/): Learn how to set up Data Prepper for log observability. * [Simple Pipeline](https://github.com/opensearch-project/data-prepper/blob/main/docs/simple_pipelines.md): Learn the basics of Data Prepper pipelines with some simple configurations. ## 3. Defining a pipeline @@ -71,7 +69,7 @@ docker run --name data-prepper \ opensearchproject/data-prepper:latest ``` -This sample pipeline configuration above demonstrates a simple pipeline with a source (`random`) sending data to a sink (`stdout`). For more examples and details about more advanced pipeline configurations, see [Pipelines]({{site.url}}{{site.baseurl}}/clients/data-prepper/pipelines). +The preceding example pipeline configuration above demonstrates a simple pipeline with a source (`random`) sending data to a sink (`stdout`). For further detailed examples of more advanced pipeline configurations, see [Pipelines]({{site.url}}{{site.baseurl}}/clients/data-prepper/pipelines/). After starting Data Prepper, you should see log output and some UUIDs after a few seconds: diff --git a/_data-prepper/index.md b/_data-prepper/index.md index 20662a11..49888741 100644 --- a/_data-prepper/index.md +++ b/_data-prepper/index.md @@ -4,9 +4,7 @@ title: Data Prepper nav_order: 1 has_children: false has_toc: false -redirect_from: - - /clients/tools/data-prepper/ - - /clients/data-prepper/ +redirect_from: - /clients/data-prepper/index/ --- diff --git a/_data-prepper/configuration/data-prepper-configuration.md b/_data-prepper/managing-data-prepper/configuring-data-prepper.md similarity index 91% rename from _data-prepper/configuration/data-prepper-configuration.md rename to _data-prepper/managing-data-prepper/configuring-data-prepper.md index dc262eae..e0eb6ff6 100644 --- a/_data-prepper/configuration/data-prepper-configuration.md +++ b/_data-prepper/managing-data-prepper/configuring-data-prepper.md @@ -1,10 +1,8 @@ --- layout: default title: Configuring Data Prepper -has_children: true -nav_order: 100 -redirect_from: - - /clients/data-prepper/data-prepper-reference/ +parent: Managing Data Prepper +nav_order: 10 --- # Configuring Data Prepper @@ -31,15 +29,15 @@ peer_forwarder | No | Object | Peer forwarder configurations. See [Peer forwarde The following section details various configuration options for peer forwarder. -#### General options for peer forwarder +#### General options for peer forwarding Option | Required | Type | Description :--- | :--- | :--- | :--- -port | No | Integer | The port number peer forwarder server is running on. Valid options are between 0 and 65535. Defaults is 4994. -request_timeout | No | Integer | Request timeout in milliseconds for peer forwarder HTTP server. Default is 10000. -server_thread_count | No | Integer | Number of threads used by peer forwarder server. Default is 200. -client_thread_count | No | Integer | Number of threads used by peer forwarder client. Default is 200. -max_connection_count | No | Integer | Maximum number of open connections for peer forwarder server. Default is 500. +port | No | Integer | The peer forwarding server port. Valid options are between 0 and 65535. Defaults is 4994. +request_timeout | No | Integer | Request timeout for the peer forwarder HTTP server in milliseconds. Default is 10000. +server_thread_count | No | Integer | Number of threads used by the peer forwarder server. Default is 200. +client_thread_count | No | Integer | Number of threads used by the peer forwarder client. Default is 200. +max_connection_count | No | Integer | Maximum number of open connections for the peer forwarder server. Default is 500. max_pending_requests | No | Integer | Maximum number of allowed tasks in ScheduledThreadPool work queue. Default is 1024. discovery_mode | No | String | Peer discovery mode to use. Valid options are `local_node`, `static`, `dns`, or `aws_cloud_map`. Defaults to `local_node`, which processes events locally. static_endpoints | Conditionally | List | A list containing endpoints of all Data Prepper instances. Required if `discovery_mode` is set to static. diff --git a/_data-prepper/logs.md b/_data-prepper/managing-data-prepper/configuring-log4j.md similarity index 96% rename from _data-prepper/logs.md rename to _data-prepper/managing-data-prepper/configuring-log4j.md index 61035425..a4b74a5c 100644 --- a/_data-prepper/logs.md +++ b/_data-prepper/managing-data-prepper/configuring-log4j.md @@ -1,7 +1,8 @@ --- layout: default title: Configuring Log4j -nav_order: 25 +parent: Managing Data Prepper +nav_order: 20 --- # Configuring Log4j diff --git a/_data-prepper/core-apis.md b/_data-prepper/managing-data-prepper/core-apis.md similarity index 98% rename from _data-prepper/core-apis.md rename to _data-prepper/managing-data-prepper/core-apis.md index cc540792..b3d0966c 100644 --- a/_data-prepper/core-apis.md +++ b/_data-prepper/managing-data-prepper/core-apis.md @@ -1,7 +1,8 @@ --- layout: default title: Core APIs -nav_order: 20 +parent: Managing Data Prepper +nav_order: 15 --- # Core APIs diff --git a/_data-prepper/managing-data-prepper/managing-data-prepper.md b/_data-prepper/managing-data-prepper/managing-data-prepper.md new file mode 100644 index 00000000..ea2d1f11 --- /dev/null +++ b/_data-prepper/managing-data-prepper/managing-data-prepper.md @@ -0,0 +1,10 @@ +--- +layout: default +title: Managing Data Prepper +has_children: true +nav_order: 20 +--- + +# Managing Data Prepper + +You can perform administrator functions for Data Prepper, including system configuration, interacting with core APIs, Log4j configuration, and monitoring. You can set up peer forwarding to coordinate multiple Data Prepper nodes when using stateful aggregation. \ No newline at end of file diff --git a/_data-prepper/monitoring.md b/_data-prepper/managing-data-prepper/monitoring.md similarity index 98% rename from _data-prepper/monitoring.md rename to _data-prepper/managing-data-prepper/monitoring.md index ea1c2a26..4304d5f3 100644 --- a/_data-prepper/monitoring.md +++ b/_data-prepper/managing-data-prepper/monitoring.md @@ -1,7 +1,8 @@ --- layout: default title: Monitoring -nav_order: 33 +parent: Administrating Data Prepper +nav_order: 25 --- # Monitoring Data Prepper with metrics diff --git a/_data-prepper/migrate-open-distro.md b/_data-prepper/migrate-open-distro.md index f0e76421..e7fdacbd 100644 --- a/_data-prepper/migrate-open-distro.md +++ b/_data-prepper/migrate-open-distro.md @@ -1,7 +1,7 @@ --- layout: default title: Migrating from Open Distro -nav_order: 35 +nav_order: 30 --- # Migrating from Open Distro diff --git a/_data-prepper/migrating-from-logstash-data-prepper.md b/_data-prepper/migrating-from-logstash-data-prepper.md index 6f1e68f3..088ef24c 100644 --- a/_data-prepper/migrating-from-logstash-data-prepper.md +++ b/_data-prepper/migrating-from-logstash-data-prepper.md @@ -1,8 +1,8 @@ --- layout: default title: Migrating from Logstash -nav_order: 30 -redirect_from: +nav_order: 25 +redirect_from: - /data-prepper/configure-logstash-data-prepper/ --- @@ -10,7 +10,7 @@ redirect_from: You can run Data Prepper with a Logstash configuration. -As mentioned in the [Getting started]({{site.url}}{{site.baseurl}}/data-prepper/get-started/) guide, you'll need to configure Data Prepper with a pipeline using a `pipelines.yaml` file. +As mentioned in [Getting started with Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/), you'll need to configure Data Prepper with a pipeline using a `pipelines.yaml` file. Alternatively, if you have a Logstash configuration `logstash.conf` to configure Data Prepper instead of `pipelines.yaml`. @@ -28,7 +28,7 @@ As of the Data Prepper 1.2 release, the following plugins from the Logstash conf ## Running Data Prepper with a Logstash configuration -1. To install Data Prepper's Docker image, see the Installing Data Prepper in [Get Started]({{site.url}}{{site.baseurl}}/data-prepper/getting-started#1-installing-data-prepper). +1. To install Data Prepper's Docker image, see Installing Data Prepper in [Getting Started]({{site.url}}{{site.baseurl}}/data-prepper/getting-started#1-installing-data-prepper). 2. Run the Docker image installed in Step 1 by supplying your `logstash.conf` configuration. diff --git a/_data-prepper/configuration/buffers/bounded-blocking.md b/_data-prepper/pipelines/configuration/buffers/bounded-blocking.md similarity index 93% rename from _data-prepper/configuration/buffers/bounded-blocking.md rename to _data-prepper/pipelines/configuration/buffers/bounded-blocking.md index 5f26b2f2..44521514 100644 --- a/_data-prepper/configuration/buffers/bounded-blocking.md +++ b/_data-prepper/pipelines/configuration/buffers/bounded-blocking.md @@ -2,7 +2,7 @@ layout: default title: Bounded blocking parent: Buffers -grand_parent: Configuring Data Prepper +grand_parent: Pipelines nav_order: 50 --- diff --git a/_data-prepper/configuration/buffers/buffers.md b/_data-prepper/pipelines/configuration/buffers/buffers.md similarity index 82% rename from _data-prepper/configuration/buffers/buffers.md rename to _data-prepper/pipelines/configuration/buffers/buffers.md index ed4a38c4..5d56638c 100644 --- a/_data-prepper/configuration/buffers/buffers.md +++ b/_data-prepper/pipelines/configuration/buffers/buffers.md @@ -1,9 +1,9 @@ --- layout: default title: Buffers -parent: Configuring Data Prepper +parent: Pipelines has_children: true -nav_order: 50 +nav_order: 20 --- # Buffers diff --git a/_data-prepper/configuration/processors/add-entries.md b/_data-prepper/pipelines/configuration/processors/add-entries.md similarity index 96% rename from _data-prepper/configuration/processors/add-entries.md rename to _data-prepper/pipelines/configuration/processors/add-entries.md index 27322c9e..e7015668 100644 --- a/_data-prepper/configuration/processors/add-entries.md +++ b/_data-prepper/pipelines/configuration/processors/add-entries.md @@ -2,7 +2,7 @@ layout: default title: add_entries parent: Processors -grand_parent: Configuring Data Prepper +grand_parent: Pipelines nav_order: 45 --- diff --git a/_data-prepper/configuration/processors/aggregate.md b/_data-prepper/pipelines/configuration/processors/aggregate.md similarity index 97% rename from _data-prepper/configuration/processors/aggregate.md rename to _data-prepper/pipelines/configuration/processors/aggregate.md index 97d8e4fd..b4bc7959 100644 --- a/_data-prepper/configuration/processors/aggregate.md +++ b/_data-prepper/pipelines/configuration/processors/aggregate.md @@ -2,7 +2,7 @@ layout: default title: aggregate parent: Processors -grand_parent: Configuring Data Prepper +grand_parent: Pipelines nav_order: 45 --- diff --git a/_data-prepper/configuration/processors/copy-values.md b/_data-prepper/pipelines/configuration/processors/copy-values.md similarity index 95% rename from _data-prepper/configuration/processors/copy-values.md rename to _data-prepper/pipelines/configuration/processors/copy-values.md index 30482514..8a636024 100644 --- a/_data-prepper/configuration/processors/copy-values.md +++ b/_data-prepper/pipelines/configuration/processors/copy-values.md @@ -2,7 +2,7 @@ layout: default title: copy_values parent: Processors -grand_parent: Configuring Data Prepper +grand_parent: Pipelines nav_order: 45 --- diff --git a/_data-prepper/configuration/processors/csv.md b/_data-prepper/pipelines/configuration/processors/csv.md similarity index 97% rename from _data-prepper/configuration/processors/csv.md rename to _data-prepper/pipelines/configuration/processors/csv.md index 1753108a..7245b517 100644 --- a/_data-prepper/configuration/processors/csv.md +++ b/_data-prepper/pipelines/configuration/processors/csv.md @@ -2,7 +2,7 @@ layout: default title: csv parent: Processors -grand_parent: Configuring Data Prepper +grand_parent: Pipelines nav_order: 45 --- diff --git a/_data-prepper/configuration/processors/date.md b/_data-prepper/pipelines/configuration/processors/date.md similarity index 98% rename from _data-prepper/configuration/processors/date.md rename to _data-prepper/pipelines/configuration/processors/date.md index 3ca383f0..90f60e5f 100644 --- a/_data-prepper/configuration/processors/date.md +++ b/_data-prepper/pipelines/configuration/processors/date.md @@ -2,7 +2,7 @@ layout: default title: date parent: Processors -grand_parent: Configuring Data Prepper +grand_parent: Pipelines nav_order: 45 --- diff --git a/_data-prepper/configuration/processors/delete-entries.md b/_data-prepper/pipelines/configuration/processors/delete-entries.md similarity index 93% rename from _data-prepper/configuration/processors/delete-entries.md rename to _data-prepper/pipelines/configuration/processors/delete-entries.md index 661d86d5..37f25c40 100644 --- a/_data-prepper/configuration/processors/delete-entries.md +++ b/_data-prepper/pipelines/configuration/processors/delete-entries.md @@ -2,7 +2,7 @@ layout: default title: delete_entries parent: Processors -grand_parent: Configuring Data Prepper +grand_parent: Pipelines nav_order: 45 --- diff --git a/_data-prepper/configuration/processors/drop-events.md b/_data-prepper/pipelines/configuration/processors/drop-events.md similarity index 77% rename from _data-prepper/configuration/processors/drop-events.md rename to _data-prepper/pipelines/configuration/processors/drop-events.md index 8ee03ca6..50d4b780 100644 --- a/_data-prepper/configuration/processors/drop-events.md +++ b/_data-prepper/pipelines/configuration/processors/drop-events.md @@ -2,7 +2,7 @@ layout: default title: drop_events parent: Processors -grand_parent: Configuring Data Prepper +grand_parent: Pipelines nav_order: 45 --- @@ -14,7 +14,7 @@ Drops all the events that are passed into this processor. Option | Required | Type | Description :--- | :--- | :--- | :--- -drop_when | Yes | String | Accepts a Data Prepper Expression string following the [Data Prepper Expression Syntax](https://github.com/opensearch-project/data-prepper/blob/main/docs/expression_syntax.md). Configuring `drop_events` with `drop_when: true` drops all the events received. +drop_when | Yes | String | Accepts a Data Prepper Expression string following the [Data Prepper Expression Syntax]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/expression-syntax/). Configuring `drop_events` with `drop_when: true` drops all the events received. handle_failed_events | No | Enum | Specifies how exceptions are handled when an exception occurs while evaluating an event. Default value is `drop`, which drops the event so it doesn't get sent to OpenSearch. Available options are `drop`, `drop_silently`, `skip`, `skip_silently`. For more information, see [handle_failed_events](https://github.com/opensearch-project/data-prepper/tree/main/data-prepper-plugins/drop-events-processor#handle_failed_events).