opensearch-docs-cn/_monitoring-your-cluster/pa/reference.md

layout	title	parent	nav_order	redirect_from
default	Metrics Reference	Performance Analyzer	3	/monitoring-plugins/pa/reference/

Metrics reference

Performance Analyzer provides a number of metrics to help you evaluate performance. The following tables describe the available metrics, grouped by the dimensions that are most relevant for that metric. All metrics support the avg, sum, min, and max aggregations, although for certain metrics, the measured value is the same regardless of aggregation type.

For information about each of the dimensions, see dimensions reference later in this topic.

This list is extensive. We recommend using Ctrl/Cmd + F to find what you're looking for. {: .tip }

Relevant dimensions: ShardID, IndexName, Operation, ShardRole

Metric	Description
CPU_Utilization	CPU usage ratio. CPU time (in milliseconds) used by the associated thread(s) in the past five seconds, divided by 5000 milliseconds.
Paging_MajfltRate	The number of major faults per second in the past five seconds. A major fault requires the process to load a memory page from disk.
Paging_MinfltRate	The number of minor faults per second in the past five seconds. A minor fault does not requires the process to load a memory page from disk.
Paging_RSS	The number of pages the process has in real memory---the pages that count towards text, data, or stack space. This number does not include pages that have not been demand-loaded in or swapped out.
Sched_Runtime	Time (seconds) spent executing on the CPU per context switch.
Sched_Waittime	Time (seconds) spent waiting on a run queue per context switch.
Sched_CtxRate	Number of times run on the CPU per second in the past five seconds.
Heap_AllocRate	An approximation, in bytes, of the heap memory allocated per second in the last 5 seconds.
IO_ReadThroughput	Number of bytes read per second in the last five seconds.
IO_WriteThroughput	Number of bytes written per second in the last five seconds.
IO_TotThroughput	Number of bytes read or written per second in the last five seconds.
IO_ReadSyscallRate	Read system calls per second in the last five seconds.
IO_WriteSyscallRate	Write system calls per second in the last five seconds.
IO_TotalSyscallRate	Read and write system calls per second in the last five seconds.
Thread_Blocked_Time	The average amount of time, in seconds, that the associated thread has been blocked from entering or reentering a monitor.
Thread_Blocked_Event	The total number of times that the associated thread has been blocked from entering or reentering a monitor (that is, the number of times a thread has been in the `blocked` state).
Thread_Waited_Time	The average amount of time, in seconds, that the associated thread has waited to enter or reenter a monitor (that is, the amount of time a thread has been in the `WAITING` or `TIMED_WAITING` state)".
Thread_Waited_Event	The total number of times that the associated thread has waited to enter or reenter a monitor (that is, the number of times a thread has been in the WAITING or TIMED_WAITING state).
ShardEvents	The total number of events executed on a shard in the past five seconds.
ShardBulkDocs	The total number of documents indexed in the past five seconds.

Relevant dimensions: ShardID, IndexName

Metric	Description
Indexing_ThrottleTime	Time (milliseconds) that the index has been under merge throttling control in the past five seconds.
Cache_Query_Hit	The number of successful lookups in the query cache in the past five seconds.
Cache_Query_Miss	The number of lookups in the query cache that failed to retrieve a `DocIdSet` in the past five seconds. `DocIdSet` is a set of document IDs in Lucene.
Cache_Query_Size	Query cache memory size in bytes.
Cache_FieldData_Eviction	The number of times OpenSearch has evicted data from the fielddata heap space (occurs when the heap space is full) in the past five seconds.
Cache_FieldData_Size	Fielddata memory size in bytes.
Cache_Request_Hit	The number of successful lookups in the shard request cache in the past five seconds.
Cache_Request_Miss	The number of lookups in the request cache that failed to retrieve the results of search requests in the past five seconds.
Cache_Request_Eviction	The number of times OpenSearch evicts data from shard request cache (occurs when the request cache is full) in the past five seconds.
Cache_Request_Size	Shard request cache memory size in bytes.

Relevant dimensions: ShardID, IndexName, IndexingStage

Metric	Description
Indexing_Pressure_Current_Limits	The total heap size, in bytes, that is available for use by an index shard in a particular indexing stage (Coordinating, Primary, or Replica).
Indexing_Pressure_Current_Bytes	The total heap size, in bytes, occupied by an index shard in a particular indexing stage (Coordinating, Primary, or Replica).
Indexing_Pressure_Last_Successful_Timestamp	The timestamp of a successful request for an index shard in a particular indexing stage (Coordinating, Primary, or Replica).
Indexing_Pressure_Rejection_Count	The total number of rejections performed by OpenSearch for an index shard in a particular indexing stage (Coordinating, Primary, or Replica).
Indexing_Pressure_Average_Window_Throughput	The average throughput of the last n requests (The value of n is determined by the `shard_indexing_pressure.secondary_parameter.throughput.request_size_window` setting) for an index shard in a particular indexing stage (Coordinating, Primary, or Replica).

Relevant dimensions: Operation, Exception, Indices, HTTPRespCode, ShardID, IndexName, ShardRole

Metric	Description
Latency	Latency (milliseconds) of a request.

Relevant dimension: MemType

Metric	Description
GC_Collection_Event	The number of garbage collections that have occurred in the past five seconds.
GC_Collection_Time	The approximate accumulated time (milliseconds) of all garbage collections that have occurred in the past five seconds.
Heap_Committed	The amount of memory (bytes) that is committed for the JVM to use.
Heap_Init	The amount of memory (bytes) that the JVM initially requests from the operating system for memory management.
Heap_Max	The maximum amount of memory (bytes) that can be used for memory management.
Heap_Used	The amount of used memory in bytes.

Relevant dimension: DiskName

Metric	Description
Disk_Utilization	Disk utilization rate: percentage of disk time spent reading and writing by the OpenSearch process in the past five seconds.
Disk_WaitTime	Average duration (milliseconds) of read and write operations in the past five seconds.
Disk_ServiceRate	Service rate: MB read or written per second in the past five seconds. This metric assumes that each disk sector stores 512 bytes.

Relevant dimension: DestAddr

Metric	Description
Net_TCP_NumFlows	The number of samples collected. Performance Analyzer collects 1 sample every 5 seconds.
Net_TCP_TxQ	The average number of TCP packets in the send buffer.
Net_TCP_RxQ	The average number of TCP packets in the receive buffer.
Net_TCP_Lost	The average number of unrecovered recurring timeouts. This number is reset when the recovery finishes or `SND.UNA` is advanced. `SND.UNA` is the sequence number of the first byte of data that has been sent but not yet acknowledged.
Net_TCP_SendCWND	The average size, in bytes, of the sending congestion window.
Net_TCP_SSThresh	The average size, in bytes, of the slow start size threshold.

Relevant dimension: Direction

Metric	Description
Net_PacketRate4	The total number of IPv4 datagrams transmitted/received from/by interfaces per second, including those transmitted or received in error.
Net_PacketDropRate4	The total number of IPv4 datagrams transmitted or received in error per second.
Net_PacketRate6	The total number of IPv6 datagrams transmitted or received from or by interfaces per second, including those transmitted or received in error.
Net_PacketDropRate6	The total number of IPv6 datagrams transmitted or received in error per second.
Net_Throughput	The number of bits transmitted or received per second by all network interfaces.

Relevant dimension: ThreadPoolType

Metric	Description
ThreadPool_QueueSize	The size of the task queue.
ThreadPool_RejectedReqs	The number of rejected executions.
ThreadPool_TotalThreads	The current number of threads in the pool.
ThreadPool_ActiveThreads	The approximate number of threads that are actively executing tasks.
ThreadPool_QueueLatency	The latency of the task queue.
ThreadPool_QueueCapacity	The current capacity of the task queue.

Relevant dimension: ClusterManager_PendingTaskType

Metric	Description
ClusterManager_PendingQueueSize	The current number of pending tasks in the cluster state update thread. Each node has a cluster state update thread that submits cluster state update tasks, such as create index, update mapping, allocate shard, and fail shard.

Relevant dimensions: Operation, Exception, Indices, HTTPRespCode

Metric	Description
HTTP_RequestDocs	The number of items in the request (only for the `_bulk` request type).
HTTP_TotalRequests	The number of requests completed in the last 5 seconds.

Relevant dimension: CBType

Metric	Description
CB_EstimatedSize	The current number of estimated bytes.
CB_TrippedEvents	The number of times that the circuit breaker has tripped.
CB_ConfiguredSize	The limit, in bytes, of the amount of memory operations can use.

Relevant dimensions: ClusterManagerTaskInsertOrder, ClusterManagerTaskPriority, ClusterManagerTaskType, ClusterManagerTaskMetadata

Metric	Description
ClusterManager_Task_Queue_Time	The amount of time, in milliseconds, that a cluster manager task spent in the queue.
ClusterManager_Task_Run_Time	The amount of time, in milliseconds, that a cluster manager task has been running.

Relevant dimension: CacheType

Metric	Description
Cache_MaxSize	The maximum size of the cache, in bytes.

Relevant dimension: ControllerName

Metric	Description
AdmissionControl_RejectionCount	The total number of rejections performed by a Controller of Admission Control.
AdmissionControl_CurrentValue	The current value for Controller of Admission Control.
AdmissionControl_ThresholdValue	The threshold value for Controller of Admission Control.

Relevant dimension: NodeID

Metric	Description
Data_RetryingPendingTasksCount	The number of throttled pending tasks on which the data node is actively performing retries. It is an absolute metric at that point in time.
ClusterManager_ThrottledPendingTasksCount	The sum of the total pending tasks that were throttled by the cluster manager node. This is a cumulative metric, so make sure to check the max aggregation.

Relevant dimensions: N/A

The following metrics are relevant to the cluster as a whole and do not require specific dimensions.

Metric	Description
Election_Term	A number that increases monotonically with every cluster manager election.
PublishClusterState_Latency	The amount of time taken by the quorum of nodes to publish the new cluster state. This metric is available for the current cluster manager.
PublishClusterState_Failure	The number of times the new cluster state failed to publish on the cluster manager node.
ClusterApplierService_Latency	The amount of time taken by each node for the apply cluster state sent by the cluster manager.
ClusterApplierService_Failure	The number of times that the apply cluster state action failed on each node.

Relevant dimensions: IndexName, NodeName, ShardType, ShardID

Metric	Description
Shard_State	The state of each shard, for example, `STARTED`, `UNASSIGNED`, or `RELOCATING`.

Relevant dimensions: NodeID, searchbp_mode

Metric	Description
SearchBP_Shard_Stats_CancellationCount	The number of tasks marked for cancellation at the shard task level.
SearchBP_Shard_Stats_LimitReachedCount	The number of times that the cancellable task total exceeded the set cancellation threshold at the shard task level.
SearchBP_Shard_Stats_Resource_Heap_Usage_CancellationCount	The number of tasks marked for cancellation because of excessive heap usage since the node last restarted at the shard task level.
SearchBP_Shard_Stats_Resource_Heap_Usage_CurrentMax	The maximum heap usage for tasks currently running at the shard task level.
SearchBP_Shard_Stats_Resource_Heap_Usage_RollingAvg	The rolling average heap usage for the _n_ most recent tasks at the shard task level. The default value for _n_ is `100`.
SearchBP_Shard_Stats_Resource_CPU_Usage_CancellationCount	The number of tasks marked for cancellation because of excessive CPU usage since the node last restarted at the shard task level.
SearchBP_Shard_Stats_Resource_CPU_Usage_CurrentMax	The maximum CPU time for all tasks currently running on the node at the shard task level.
SearchBP_Shard_Stats_Resource_CPU_Usage_CurrentAvg	The average CPU time for all tasks currently running on the node at the shard task level.
SearchBP_Shard_Stats_Resource_ElaspedTime_Usage_CancellationCount	The number of tasks marked for cancellation because of excessive time elapsed since the node last restarted at the shard task level.
SearchBP_Shard_Stats_Resource_ElaspedTime_Usage_CurrentMax	The maximum time elapsed for all tasks currently running on the node at the shard task level.
SearchBP_Shard_Stats_Resource_ElaspedTime_Usage_CurrentAvg	The average time elapsed for all tasks currently running on the node at the shard task level.
Searchbp_Task_Stats_CancellationCount	The number of tasks marked for cancellation at the search task level.
SearchBP_Task_Stats_LimitReachedCount	The number of times that the cancellable task total exceeded the set cancellation threshold at the search task level.
SearchBP_Task_Stats_Resource_Heap_Usage_CancellationCount	The number of tasks marked for cancellation because of excessive heap usage since the node last restarted at the search task level.
SearchBP_Task_Stats_Resource_Heap_Usage_CurrentMax	The maximum heap usage for tasks currently running at the search task level.
SearchBP_Task_Stats_Resource_Heap_Usage_RollingAvg	The rolling average heap usage for the _n_ most recent tasks at the search task level. The default value for _n_ is `10`.
SearchBP_Task_Stats_Resource_CPU_Usage_CancellationCount	The number of tasks marked for cancellation because of excessive CPU usage since the node last restarted at the search task level.
SearchBP_Task_Stats_Resource_CPU_Usage_CurrentMax	The maximum CPU time for all tasks currently running on the node at the search task level.
SearchBP_Task_Stats_Resource_CPU_Usage_CurrentAvg	The average CPU time for all tasks currently running on the node at the search task level.
SearchBP_Task_Stats_Resource_ElaspedTime_Usage_CancellationCount	The number of tasks marked for cancellation because of excessive time elapsed since the node last restarted at the search task level.
SearchBP_Task_Stats_Resource_ElaspedTime_Usage_CurrentMax	The maximum time elapsed for all tasks currently running on the node at the search task level.
SearchBP_Task_Stats_Resource_ElaspedTime_Usage_CurrentAvg	The average time elapsed for all tasks currently running on the node at the search task level.

Dimensions reference

Dimension	Return values
ShardID	The ID of the shard, for example, 1.
IndexName	The name of the index, for example, my-index.
Operation	The type of operation, for example, shardbulk.
ShardRole	The shard role, for example, primary or replica.
Exception	OpenSearch exceptions, for example, org.opensearch.index_not_found_exception.
Indices	The list of indexes in the request URL.
HTTPRespCode	The OpenSearch response code, for example, 200.
MemType	The memory type, for example, totYoungGC, totFullGC, Survivor, PermGen, OldGen, Eden, NonHeap, or Heap.
DiskName	The name of the disk, for example, sda1.
DestAddr	The destination address, for example, 010015AC.
Direction	The direction, for example, in or out.
ThreadPoolType	The OpenSearch thread pools, for example, index, search, or snapshot.
CBType	The circuit breaker type, for example, accounting, fielddata, in_flight_requests, parent, or request.
ClusterManagerTaskInsertOrder	The order in which the task was inserted, for example, 3691.
ClusterManagerTaskPriority	The priority of the task, for example, URGENT. OpenSearch executes higher-priority tasks before lower-priority ones, regardless of insert_order.
ClusterManagerTaskType	The task type, for example, shard-started, create-index, delete-index, refresh-mapping, put-mapping, CleanupSnapshotRestoreState, or Update snapshot state.
ClusterManagerTaskMetadata	The metadata for the task (if any).
CacheType	The cache type, for example, Field_Data_Cache, Shard_Request_Cache, or Node_Query_Cache.
NodeID	The ID of the node.
Searchbp_mode	The search backpressure mode, for example, monitor_only (default), enforced, or disabled.