Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HDFS-14526. RBF: Update the document of RBF related metrics. Contributed by Takanobu Asanuma.

This commit is contained in:
Ayush Saxena 2019-06-05 22:03:27 +05:30 committed by Brahma Reddy Battula
parent 1579136fa7
commit 812256baad
3 changed files with 37 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

34
hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
View File

@ -478,6 +478,40 @@ contains tags such as Hostname as additional information along with metrics.
| `FileIoErrorRateNumOps` | The number of file io error operations within an interval time of metric | | `FileIoErrorRateNumOps` | The number of file io error operations within an interval time of metric |
| `FileIoErrorRateAvgTime` | It measures the mean time in milliseconds from the start of an operation to hitting a failure | | `FileIoErrorRateAvgTime` | It measures the mean time in milliseconds from the start of an operation to hitting a failure |
RBFMetrics
----------------
RBFMetrics shows the metrics which are the aggregated values of sub-clusters' information in the Router-based federation.
| Name | Description |
|:---- |:---- |
| `NumFiles` | Current number of files and directories |
| `NumBlocks` | Current number of allocated blocks |
| `NumOfBlocksPendingReplication` | Current number of blocks pending to be replicated |
| `NumOfBlocksUnderReplicated` | Current number of blocks under replicated |
| `NumOfBlocksPendingDeletion` | Current number of blocks pending deletion |
| `ProvidedSpace` | The total remote storage capacity mounted in the federated cluster |
| `NumInMaintenanceLiveDataNodes` | Number of live Datanodes which are in maintenance state |
| `NumInMaintenanceDeadDataNodes` | Number of dead Datanodes which are in maintenance state |
| `NumEnteringMaintenanceDataNodes` | Number of Datanodes that are entering the maintenance state |
| `TotalCapacity` | Current raw capacity of DataNodes in bytes |
| `UsedCapacity` | Current used capacity across all DataNodes in bytes |
| `RemainingCapacity` | Current remaining capacity in bytes |
| `NumOfMissingBlocks` | Current number of missing blocks |
| `NumLiveNodes` | Number of datanodes which are currently live |
| `NumDeadNodes` | Number of datanodes which are currently dead |
| `NumStaleNodes` | Current number of DataNodes marked stale due to delayed heartbeat |
| `NumDecomLiveNodes` | Number of datanodes which have been decommissioned and are now live |
| `NumDecomDeadNodes` | Number of datanodes which have been decommissioned and are now dead |
| `NumDecommissioningNodes` | Number of datanodes in decommissioning state |
| `Namenodes` | Current information about all the namenodes |
| `Nameservices` | Current information for each registered nameservice |
| `MountTable` | The mount table for the federated filesystem |
| `Routers` | Current information about all routers |
| `NumNameservices` | Number of nameservices |
| `NumNamenodes` | Number of namenodes |
| `NumExpiredNamenodes` | Number of expired namenodes |
| `NodeUsage` | Max, Median, Min and Standard Deviation of DataNodes usage |
RouterRPCMetrics RouterRPCMetrics
---------------- ----------------
RouterRPCMetrics shows the statistics of the Router component in Router-based federation. RouterRPCMetrics shows the statistics of the Router component in Router-based federation.

2
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/RBFMetrics.java
View File

@ -77,6 +77,7 @@ import org.apache.hadoop.hdfs.server.federation.store.records.MembershipStats;
import org.apache.hadoop.hdfs.server.federation.store.records.MountTable; import org.apache.hadoop.hdfs.server.federation.store.records.MountTable;
import org.apache.hadoop.hdfs.server.federation.store.records.RouterState; import org.apache.hadoop.hdfs.server.federation.store.records.RouterState;
import org.apache.hadoop.hdfs.server.federation.store.records.StateStoreVersion; import org.apache.hadoop.hdfs.server.federation.store.records.StateStoreVersion;
import org.apache.hadoop.metrics2.annotation.Metrics;
import org.apache.hadoop.metrics2.util.MBeans; import org.apache.hadoop.metrics2.util.MBeans;
import org.apache.hadoop.security.UserGroupInformation; import org.apache.hadoop.security.UserGroupInformation;
import org.apache.hadoop.util.StringUtils; import org.apache.hadoop.util.StringUtils;
@ -91,6 +92,7 @@ import com.google.common.annotations.VisibleForTesting;
/** /**
* Implementation of the Router metrics collector. * Implementation of the Router metrics collector.
*/ */
@Metrics(name="RBFActivity", about="RBF metrics", context="dfs")
public class RBFMetrics implements RouterMBean, FederationMBean { public class RBFMetrics implements RouterMBean, FederationMBean {
private static final Logger LOG = private static final Logger LOG =

2
hadoop-hdfs-project/hadoop-hdfs-rbf/src/site/markdown/HDFSRouterFederation.md
View File

@ -473,4 +473,4 @@ Metrics
------- -------
The Router and State Store statistics are exposed in metrics/JMX. These info will be very useful for monitoring. The Router and State Store statistics are exposed in metrics/JMX. These info will be very useful for monitoring.
More metrics info can see [Router RPC Metrics](../../hadoop-project-dist/hadoop-common/Metrics.html#RouterRPCMetrics) and [State Store Metrics](../../hadoop-project-dist/hadoop-common/Metrics.html#StateStoreMetrics). More metrics info can see [RBF Metrics](../../hadoop-project-dist/hadoop-common/Metrics.html#RBFMetrics), [Router RPC Metrics](../../hadoop-project-dist/hadoop-common/Metrics.html#RouterRPCMetrics) and [State Store Metrics](../../hadoop-project-dist/hadoop-common/Metrics.html#StateStoreMetrics).