opensearch-docs-cn/_data-prepper/pipelines/configuration/processors/aggregate.md

layout	title	parent	grand_parent	nav_order
default	Aggregate processor	Processors	Pipelines	41

aggregate

The aggregate processor groups events based on the keys provided and performs an action on each group.

Configuration

The following table describes the options you can use to configure the aggregate processor.

Option	Required	Type	Description
identification_keys	Yes	List	An unordered list by which to group events. Events with the same values as these keys are put into the same group. If an event does not contain one of the identification_keys, then the value of that key is considered to be equal to null. At least one identification_key is required (for example, ["sourceIp", "destinationIp", "port"]).
action	Yes	AggregateAction	The action to be performed for each group. One of the available aggregate actions must be provided or you can create custom aggregate actions. remove_duplicates and put_all are the available actions. For more information, see Creating New Aggregate Actions.
group_duration	No	String	The amount of time that a group should exist before it is concluded automatically. Supports ISO_8601 notation strings ("PT20.345S", "PT15M", etc.) as well as simple notation for seconds ("60s") and milliseconds ("1500ms"). Default value is 180s.

Metrics

The following table describes common Abstract processor metrics.

Metric name	Type	Description
recordsIn	Counter	Metric representing the ingress of records to a pipeline component.
recordsOut	Counter	Metric representing the egress of records from a pipeline component.
timeElapsed	Timer	Metric representing the time elapsed during execution of a pipeline component.

The aggregate processor includes the following custom metrics.

Counter

• actionHandleEventsOut: The number of events that have been returned from the handleEvent call to the configured action.
• actionHandleEventsDropped: The number of events that have not been returned from the handleEvent call to the configured action.
• actionHandleEventsProcessingErrors: The number of calls made to handleEvent for the configured action that resulted in an error.
• actionConcludeGroupEventsOut: The number of events that have been returned from the concludeGroup call to the configured action.
• actionConcludeGroupEventsDropped: The number of events that have not been returned from the condludeGroup call to the configured action.
• actionConcludeGroupEventsProcessingErrors: The number of calls made to concludeGroup for the configured action that resulted in an error.

Gauge

• currentAggregateGroups: The current number of groups. This gauge decreases when a group concludes and increases when an event initiates the creation of a new group.