78 lines
4.4 KiB
Plaintext
78 lines
4.4 KiB
Plaintext
|
Overview
|
||
|
|
========
|
||
|
|
hbase-metrics and hbase-metrics-api are two modules that define and implement the "new" metric
|
||
|
|
system used internally within HBase. These two modules (and some other code in hbase-hadoop2-compat)
|
||
|
|
module are referred as "HBase metrics framework".
|
||
|
|
|
||
|
|
HBase-metrics-api Module
|
||
|
|
========================
|
||
|
|
HBase Metrics API (hbase-metrics-api) contains the interface
|
||
|
|
that HBase exposes internally and to third party code (including coprocessors). It is a thin
|
||
|
|
abstraction over the actual implementation for backwards compatibility guarantees. The source
|
||
|
|
/ binary and other compatibility guarantees are for "LimitedPrivate API" (see [1] for an
|
||
|
|
explanation).
|
||
|
|
|
||
|
|
The metrics API in this hbase-metrics-api module is inspired by the Dropwizard metrics 3.1 API
|
||
|
|
(See [2]). It is a subset of the API only containing metrics collection. However, the implementation
|
||
|
|
is HBase-specific and provided in hbase-metrics module. All of the classes in this module is
|
||
|
|
HBase-internal. See the latest documentation of Dropwizard metrics for examples of defining / using
|
||
|
|
metrics.
|
||
|
|
|
||
|
|
|
||
|
|
HBase-metrics Module
|
||
|
|
====================
|
||
|
|
hbase-metrics module contains implementation of the "HBase Metrics API", including MetricRegistry,
|
||
|
|
Counter, Histogram, etc. These are highly concurrent implementations of the Metric interfaces.
|
||
|
|
Metrics in HBase are grouped into different sets (like WAL, RPC, RegionServer, etc). Each group of
|
||
|
|
metrics should be tracked via a MetricRegistry specific to that group. Metrics can be dynamically
|
||
|
|
added or removed from the registry with a name. Each Registry is independent of the other
|
||
|
|
registries and will have it's own JMX context and MetricRecord (when used with Metrics2).
|
||
|
|
|
||
|
|
|
||
|
|
MetricRegistry's themselves are tracked via a global registry (of MetricRegistries) called
|
||
|
|
MetricRegistries. MetricRegistries.global() can be used to obtain the global instance.
|
||
|
|
MetricRegistry instances can also be dynamically registered and removed. However, unlike the
|
||
|
|
MetricRegistry, MetricRegistries does reference counting of the MetricRegistry instances. Only
|
||
|
|
Metrics in the MetricRegistry instances that are in the global MetricRegistry are exported to the
|
||
|
|
metric sinks or JMX.
|
||
|
|
|
||
|
|
|
||
|
|
Coprocessor Metrics
|
||
|
|
===================
|
||
|
|
HBase allows custom coprocessors to track and export metrics using the new framework.
|
||
|
|
Coprocessors and other third party code should only use the classes and interfaces from
|
||
|
|
hbase-metrics-api module and only the classes that are marked with InterfaceAudience.LimitedPrivate
|
||
|
|
annotation. There is no guarantee on the compatibility requirements for other classes.
|
||
|
|
|
||
|
|
Coprocessors can obtain the MetricRegistry to register their custom metrics via corresponding
|
||
|
|
CoprocessorEnvironment context. See ExampleRegionObserverWithMetrics and
|
||
|
|
ExampleMasterObserverWithMetrics classes in hbase-examples module for usage.
|
||
|
|
|
||
|
|
|
||
|
|
Developer Notes
|
||
|
|
===============
|
||
|
|
Historically, HBase has been using Hadoop's Metrics2 framework [3] for collecting and reporting the
|
||
|
|
metrics internally. However, due to the difficultly of dealing with the Metrics2 framework, HBase is
|
||
|
|
moving away from Hadoop's metrics implementation to its custom implementation. The move will happen
|
||
|
|
incrementally, and during the time, both Hadoop Metrics2-based metrics and hbase-metrics module
|
||
|
|
based classes will be in the source code. All new implementations for metrics SHOULD use the new
|
||
|
|
API and framework.
|
||
|
|
|
||
|
|
Examples of the new framework can be found in MetricsCoprocessor and MetricsRegionServerSourceImpl
|
||
|
|
classes. See HBASE-9774 [4] for more context.
|
||
|
|
|
||
|
|
hbase-metrics module right now only deals with metrics tracking and collection. It does not do JMX
|
||
|
|
reporting or reporting to console, ganglia, opentsdb, etc. We use Hadoop's Metrics2 for reporting
|
||
|
|
metrics to different sinks or exporting via JMX. However, this is contained within the
|
||
|
|
hbase-hadoop2-compat module completely, so that rest of the code does not know anything about the
|
||
|
|
Metrics2 dependency. HBaseMetrics2HadoopMetricsAdapter is the adapter that can collect metrics
|
||
|
|
in a MetricRegistry using the metric2 MetricsCollector / MetricRecordBuilder interfaces.
|
||
|
|
GlobalMetricRegistriesSource is the global Metrics2 Source that collects all of the metrics in all
|
||
|
|
of the metric registries in the MetricRegistries.global() instance.
|
||
|
|
|
||
|
|
|
||
|
|
References
|
||
|
|
1. https://hbase.apache.org/book.html#hbase.versioning
|
||
|
|
2. http://metrics.dropwizard.io/
|
||
|
|
3. https://hadoop.apache.org/docs/r2.7.2/api/org/apache/hadoop/metrics2/package-summary.html
|
||
|
|
4. https://issues.apache.org/jira/browse/HBASE-9774
|