iSharkFly-Docs/opensearch-docs-cn
1
0
Fork 0
mirror of https://github.com/iSharkFly-Docs/opensearch-docs-cn synced 2025-02-07 18:58:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
opensearch-docs-cn/_data-prepper/pipelines/configuration/processors/aggregate.md
Caroline ef83f6d7d5
Added metrics sections to Aggregate processor subpages. (#2730) 
* Added metrics section to Aggregate processor page.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added Metrics section to individual Processors pages.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added metrics section for JSON processor.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added metrics sections. Changed Default is to Default value is.

Signed-off-by: carolxob <carolxob@amazon.com>

* Corrected references from AWS S3 to Amazon S3.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to Metrics sections and phrasing.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updated Action link.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to buffer_size and batch_size default values.

Signed-off-by: carolxob <carolxob@amazon.com>

* Edits to Metrics sections for each processor.

Signed-off-by: carolxob <carolxob@amazon.com>

* Update made based ondoc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to intro text for processor pages. Minor adjustements to other text for clarity.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Adjustements to phrasing, fixed typos.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor updates to word choice and corrected a typo.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edit.

Signed-off-by: carolxob <carolxob@amazon.com>

* Made updates based ondoc review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Updates to http-source.

Signed-off-by: carolxob <carolxob@amazon.com>

* Added common processors table to affected docs.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update to one file.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update based on tech review feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major editorial feedback incorporated through key-value.md.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated major editorial feedback thup to service-map-stateful.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated major editorial feedback for Processors section.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major editorial updates, specifically to inclusion of text introducing option configuration tables.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major editorial feedback through otel-trace.md incorporated.

Signed-off-by: carolxob <carolxob@amazon.com>

* Major editorial edits incorporated.

Signed-off-by: carolxob <carolxob@amazon.com>

* Technical feedback and editorial feedback incorporated.

Signed-off-by: carolxob <carolxob@amazon.com>

* Incorporated missing editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor adjustements to OpenSearch sink.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor changes to capitalization.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor edits.

Signed-off-by: carolxob <carolxob@amazon.com>

* Made one instance of processor name consistent with other references.

Signed-off-by: carolxob <carolxob@amazon.com>

* Minor update based on editorial feedback.

Signed-off-by: carolxob <carolxob@amazon.com>

---------

Signed-off-by: carolxob <carolxob@amazon.com>
2023-02-27 11:40:59 -05:00

3.8 KiB
Raw Blame History

layout title parent grand_parent nav_order
default aggregate Processors Pipelines 45

aggregate

Overview

The aggregate processor groups events based on the keys provided and performs an action on each group. The following table describes the options you can use to configure the aggregate processor.

Option Required Type Description
identification_keys Yes List An unordered list by which to group events. Events with the same values as these keys are put into the same group. If an event does not contain one of the identification_keys, then the value of that key is considered to be equal to null. At least one identification_key is required (for example, ["sourceIp", "destinationIp", "port"]).
action Yes AggregateAction The action to be performed for each group. One of the available aggregate actions must be provided or you can create custom aggregate actions. remove_duplicates and put_all are the available actions. For more information, see Creating New Aggregate Actions.
group_duration No String The amount of time that a group should exist before it is concluded automatically. Supports ISO_8601 notation strings ("PT20.345S", "PT15M", etc.) as well as simple notation for seconds ("60s") and milliseconds ("1500ms"). Default value is 180s.

Metrics

The following table describes common Abstract processor metrics.

Metric name Type Description
recordsIn Counter Metric representing the ingress of records to a pipeline component.
recordsOut Counter Metric representing the egress of records from a pipeline component.
timeElapsed Timer Metric representing the time elapsed during execution of a pipeline component.

The aggregate processor includes the following custom metrics.

Counter

  • actionHandleEventsOut: The number of events that have been returned from the handleEvent call to the configured action.
  • actionHandleEventsDropped: The number of events that have not been returned from the handleEvent call to the configured action.
  • actionHandleEventsProcessingErrors: The number of calls made to handleEvent for the configured action that resulted in an error.
  • actionConcludeGroupEventsOut: The number of events that have been returned from the concludeGroup call to the configured action.
  • actionConcludeGroupEventsDropped: The number of events that have not been returned from the condludeGroup call to the configured action.
  • actionConcludeGroupEventsProcessingErrors: The number of calls made to concludeGroup for the configured action that resulted in an error.

Gauge

  • currentAggregateGroups: The current number of groups. This gauge decreases when a group concludes and increases when an event initiates the creation of a new group.