Wire up the `ConfigurationObserver` chain for
`RegionNormalizerManager`. The following configuration keys support
hot-reloading:
* hbase.normalizer.throughput.max_bytes_per_sec
* hbase.normalizer.split.enabled
* hbase.normalizer.merge.enabled
* hbase.normalizer.min.region.count
* hbase.normalizer.merge.min_region_age.days
* hbase.normalizer.merge.min_region_size.mb
Note that support for `hbase.normalizer.period` is not provided
here. Support would need to be implemented generally for the `Chore`
subsystem.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Aman Poonia <aman.poonia.29@gmail.com>
The core change here is to the loop in
`SimpleRegionNormalizer#computeMergeNormalizationPlans`. It's a nested
loop that walks the table's region chain once, looking for contiguous
sequences of regions that meet the criteria for merge. The outer loop
tracks the starting point of the next sequence, the inner loop looks
for the end of that sequence. A single sequence becomes an instance of
`MergeNormalizationPlan`.
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
Remove the RegionStates.include method as its name is ambiguous.
Add more comments to describe the logic on why we filter region like
this.
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
Modified compared to main branch to deprecate obviated MasterObserver interface methods instead of remove them.
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit c5ca191921cad9100fb47aa57348571b3ddc06ad)
Conflicts:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AdminOverAsyncAdmin.java
hbase-server/src/test/java/org/apache/hadoop/hbase/rsgroup/VerifyingRSGroupAdmin.java
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestWithDisabledAuthorization.java
* HBASE-25065 - WAL archival to be done by a separate thread
* Fix checkstyle comments
* Fix compile issue
* Fix checkstyle and make the failing test more reliable
* Remove unused import
Implement a rate limiter for the normalizer. Implemented in terms of
MB/sec of affacted region size (the same metrics used to make
normalization decisions). Uses Guava `RateLimiter` to perform the
resource accounting. `RateLimiter` works by blocking (uninterruptible
😖) the calling thread. Thus, the whole construction of the normalizer
subsystem needed refactoring. See the provided `package-info.java` for
an overview of this new structure.
Introduces a new configuration,
`hbase.normalizer.throughput.max_bytes_per_sec`, for specifying a
limit on the throughput of actions executed by the normalizer. Note
that while this configuration value is in bytes, the minimum honored
valued `1_000_000`. Supports values configured using the
human-readable suffixes honored by `Configuration.getLongBytes`
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: Michael Stack <stack@apache.org>
Make it so WALPlayer can replay recovered.edits files.
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALInputFormat.java
Allow for WAL files that do NOT have a startime in their name.
Use the 'generic' WAL-filename parser instead of the one that
used be local here. Implement support for 'startTime' filter.
Previous was just not implemented.
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
Checkstyle.
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java
Use the new general WAL name timestamp parser.
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
Utility for parsing timestamp from WAL filename.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRecoveredEdits.java
Export attributes about the local recovered.edits test file
so other tests can play with it.
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Change the test to wait for evidence that the active master has seen
that the backup master killed by the test has gone away. This is done
before proceeding to validate that the dead backup is correctly
omitted from the ClusterStatus report.
Also, minor fixup to several assertions, using `assertEquals` instead
of `assertTrue(...equals(...))` and correcting expected vs. actual
ordering of assertion arguments.
Signed-off-by: Michael Stack <stack@apache.org>
Give the comparator a more descriptive name, a better location,
and make it work even when passed hbase:meta WAL files.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Pass WALFactory to Replication instead of WALProvider. WALFactory has all
WALProviders in it, not just the user-space WALProvider. Do this so
ReplicationService has access to all WALProviders in the Server (To be
exploited by the follow-on patch in HBASE-25055)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Editing logging around region replicas: shortening and adding context.
Checkstyle fixes in edited files while I was in there.
Bug fix in AssignRegionHandler -- was using M_RS_CLOSE_META to open
a Region instead of a M_RS_OPEN_META.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Closes#2423
Untangle RegionInfo, RegionInfoBuilder, and MutableRegionInfo static
initializations some. Move MutableRegionInfo from inner-class of
RegionInfoBuilder to be (package private) standalone. Undo static
initializing references from RI to RIB.
Co-authored-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
