8476 Commits

Author SHA1 Message Date
Jan Hentschel
2cd5f914af
HBASE-25197 Remove SingletonCoprocessorService interface (#2560)
Remove the SingletonCoprocessorService interface targeted for removal in
3.0.0.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-21 19:04:17 +02:00
Jan Hentschel
1b58af1841
HBASE-25198 Remove deprecated RpcSchedulerFactory#create (#2561)
Remove the deprecated RpcSchedulerFactory#create(Configuration,
PriorityFunction) method from the interface and in all implementing
classes.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-21 21:40:45 +08:00
Ankit Singhal
7d00e43dce
HBASE-25166 MobFileCompactionChore is closing the master's shared cluster connection (#2514) 2020-10-19 22:22:33 -07:00
shahrs87
e6c78f4037
HBASE-25179 : Fix Assert format in HFilePerformanceEvaluation class
Closes #2551

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-16 12:08:10 +05:30
Nick Dimiduk
70d2f37ee3
HBASE-24628 Region normalizer now respects a rate limit (HMaster chore shutdown NPE ADDENDUM) (#2540)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-15 22:02:53 +08:00
Duo Zhang
38ac7e26ae
HBASE-25186 TestMasterRegionOnTwoFileSystems is failing after HBASE-25065 (#2544)
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
2020-10-15 10:22:14 +08:00
XinSun
78b7244091
HBASE-25117 ReplicationSourceShipper thread can not be finished (#2521)
Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-14 18:08:54 +01:00
Duo Zhang
92c3bcd9fb
HBASE-25164 Make ModifyTableProcedure support changing meta replica count (#2513)
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-13 09:43:56 +08:00
stack
c28220522d HBASE-25168 Unify WAL name timestamp parsers
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-10-12 07:52:26 -07:00
niuyulin
8eea052359
HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE (#2450)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-12 11:02:37 +08:00
Nick Dimiduk
c8c860c906
HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (ADDENDUM) (#2529)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-11 21:52:11 +08:00
bsglz
5a474449c1
HBASE-24840 Avoid shows closing region task when create table (#2226) 2020-10-11 19:12:56 +08:00
ramkrish86
accd9750aa
HBASE-25065 WAL archival to be done by a separate thread (#2501)
* HBASE-25065 WAL archival can be batched/throttled and also done by a separate thread

* Fix checkstyle issues

* Address review comments

* checkstyle comments

* Addressing final review comments
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-11 10:46:06 +05:30
Michael Stack
665a8767a0
HBASE-22976 [HBCK2] Add RecoveredEditsPlayer (#2504)
Make it so WALPlayer can replay recovered.edits files.

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALInputFormat.java
 Allow for WAL files that do NOT have a startime in their name.
 Use the 'generic' WAL-filename parser instead of the one that
 used be local here. Implement support for 'startTime' filter.
 Previous was just not implemented.

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
 Checkstyle.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java
 Use the new general WAL name timestamp parser.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
 Utility for parsing timestamp from WAL filename.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRecoveredEdits.java
 Export attributes about the local recovered.edits test file
 so other tests can play with it.

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-10-09 08:46:05 -07:00
Nick Dimiduk
78ae1f176d HBASE-24628 Region normalizer now respects a rate limit
Implement a rate limiter for the normalizer. Implemented in terms of
MB/sec of affacted region size (the same metrics used to make
normalization decisions). Uses Guava `RateLimiter` to perform the
resource accounting. `RateLimiter` works by blocking (uninterruptible
😖) the calling thread. Thus, the whole construction of the normalizer
subsystem needed refactoring. See the provided `package-info.java` for
an overview of this new structure.

Introduces a new configuration,
`hbase.normalizer.throughput.max_bytes_per_sec`, for specifying a
limit on the throughput of actions executed by the normalizer. Note
that while this configuration value is in bytes, the minimum honored
valued `1_000_000`. Supports values configured using the
human-readable suffixes honored by `Configuration.getLongBytes`

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-08 15:45:08 -07:00
Nick Dimiduk
b82d8a5517
HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)
Change the test to wait for evidence that the active master has seen
that the backup master killed by the test has gone away. This is done
before proceeding to validate that the dead backup is correctly
omitted from the ClusterStatus report.

Also, minor fixup to several assertions, using `assertEquals` instead
of `assertTrue(...equals(...))` and correcting expected vs. actual
ordering of assertion arguments.

Signed-off-by: Michael Stack <stack@apache.org>
2020-10-08 14:23:38 -07:00
Mohammad Arshad
bdcafa895c
HBASE-24025: Improve performance of move_servers_rsgroup by using async region move API (#1549) 2020-10-08 10:30:16 -07:00
Michael Stack
8a995ae81b
HBASE-25165 Change 'State time' in UI so sorts (#2508)
Display startcode in iso8601.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-10-08 09:10:30 -07:00
Duo Zhang
0d63318f10
HBASE-25124 Support changing region replica count without disabling table (#2497)
Signed-off-by: stack <stack@apache.org>
2020-10-08 21:35:29 +08:00
Toshihiro Suzuki
aff8bbf0cb
HBASE-25160 Refactor AccessController and VisibilityController (#2506)
Signed-off-by: stack <stack@apache.org>
2020-10-08 17:04:48 +09:00
Duo Zhang
e88509841f Revert "HBASE-24813 ReplicationSource should clear buffer usage on Replicatio… (#2191)"
This reverts commit 687e53b7e49c1a149e55829bbeca6aa4edfe69e7.
2020-10-06 21:09:00 +08:00
Andor Molnár
14b523ec98
HBASE-25146 Add extra logging at info level to HFileCorruptionChecker in order to report progress (#2503)
Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Signed-off-by: Mate Szalay-Beko <szalay.beko.mate@gmail.com>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2020-10-06 09:38:18 +02:00
Duo Zhang
9ba90e1679
HBASE-25121 Refactor MetaTableAccessor.addRegionsToMeta and its usage places (#2476)
Signed-off-by: stack <stack@apache.org>
2020-10-05 21:29:55 +08:00
Viraj Jasani
23ce91819a
HBASE-25147 : Serialize regionNames in ReopenTableRegionsProcedure
Closes #2494

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-05 15:49:26 +05:30
niuyulin
3226c1795a
HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
Closes #2473

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-04 16:07:28 +05:30
Duo Zhang
b0170d0e24
HBASE-25080 Should not use AssignmentManager to test whether a table is enabled or not (#2436)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-03 20:48:38 +08:00
Michael Stack
3b91a15183
HBASE-25091 Move LogComparator from ReplicationSource to AbstractFSWALProvider#.WALsStartTimeComparator (#2449)
Give the comparator a more descriptive name, a better location,
and make it work even when passed hbase:meta WAL files.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-01 10:04:58 -07:00
ramkrish86
a0b3d6276f
HBASE-25135 Convert the internal seperator while emitting the memstore read metrics to # (#2486)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-10-01 18:07:28 +05:30
Wellington Ramos Chevreuil
9fc29c4cbf
HBASE-24877 addendum: additional checks to avoid one extra possible race control in the initialize loop (#2400)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-09-29 10:00:57 +01:00
Duo Zhang
b268b1f621
HBASE-25103 Remove ZNodePaths.metaReplicaZNodes (#2464)
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-09-29 15:48:57 +08:00
bsglz
5d926627ae
HBASE-24967 The table.jsp cost long time to load if the table include… (#2326)
* HBASE-24967 The table.jsp cost long time to load if the table include closed regions

* fix it  by another way

* fix review issue

* fix checkstyle warnings

* fix checkstyle warning
2020-09-29 15:36:03 +08:00
XinSun
68b56beab7
HBASE-25100 conf and conn are assigned twice in HBaseReplicationEndpoint and HBaseInterClusterReplicationEndpoint (#2463)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-29 08:27:37 +08:00
Viraj Jasani
80ffac29c1
HBASE-25070 : With new generic API getLogEntries, cleaning up unused RPC APIs
Closes #2426

Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-28 12:54:35 +05:30
Toshihiro Suzuki
86557edf2c
HBASE-25096 WAL size in RegionServer UI is wrong (#2456)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-28 14:47:18 +09:00
Mohammad Arshad
b57bef5b7d
HBASE-25077: hbck.jsp page loading fails, logs NPE in master log. (#2433)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2020-09-27 14:05:38 +08:00
XinSun
380585ef20
HBASE-25098 ReplicationStatisticsChore runs in wrong time unit (#2460)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-26 19:49:02 +08:00
meiyi
4f8aa75dca
HBASE-25097 Wrong RIT page number in Master UI (#2458)
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
2020-09-25 18:36:41 +08:00
Wellington Ramos Chevreuil
687e53b7e4
HBASE-24813 ReplicationSource should clear buffer usage on Replicatio… (#2191)
Signed-off-by: Josh Elser <elserj@apache.org>
2020-09-24 10:44:36 +01:00
Guanghao Zhang
8828643bb2
HBASE-25086 Refactor Replication: move the default ReplicationSinkService implementation out (#2444)
Signed-off-by: meiyi <myimeiyi@gmail.com>
2020-09-24 17:25:34 +08:00
Bharath Vissapragada
56c7505f8f
HBASE-25082: Per table WAL metrics: appendCount and appendSize (#2440)
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Ankit Jain <jain.ankit@salesforce.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-09-23 21:06:57 -07:00
Andrew Purtell
8bfa2cb2ee
HBASE-25079 Upgrade Bootstrap to 3.3.7 (#2442)
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
2020-09-23 14:56:50 -07:00
stack
e7797208d6 Revert "HBASE-25068 Pass WALFactory to Replication so it knows of all WALProviders, not just default/user-space"
This reverts commit 17ebf917ba354e4632b726323b2b32af3aa6c8de.
2020-09-22 20:48:31 -07:00
Guanghao Zhang
7e910a573f
HBASE-25074 Refactor ReplicationSinkManager: reduce code and make it easy to understand (#2430)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-09-23 08:30:43 +08:00
stack
17ebf917ba HBASE-25068 Pass WALFactory to Replication so it knows of all WALProviders, not just default/user-space
Pass WALFactory to Replication instead of WALProvider. WALFactory has all
WALProviders in it, not just the user-space WALProvider. Do this so
ReplicationService has access to all WALProviders in the Server (To be
exploited by the follow-on patch in HBASE-25055)
2020-09-22 10:18:26 -07:00
Michael Stack
70a947dc6b
HBASE-25067 Edit of log messages around async WAL Replication; checkstyle fixes; and a bugfix (#2435)
Editing logging around region replicas: shortening and adding context.
Checkstyle fixes in edited files while I was in there.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-09-22 10:10:31 -07:00
niuyulin
8a6299bb38
HBASE-25076 fix typo in MasterRegion java doc (#2432)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>

Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-09-21 19:00:49 +02:00
niuyulin
36e8b0beec
HBASE-25075 Fix typo in ReplicationProtobufUtil (#2431)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>

Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-09-21 18:44:47 +02:00
stack
3e13b3f059
HBASE-24896 'Stuck' in static initialization creating RegionInfo instance
Closes #2422

Untangle RegionInfo, RegionInfoBuilder, and MutableRegionInfo static
initializations some. Move MutableRegionInfo from inner-class of
RegionInfoBuilder to be (package private) standalone. Undo static
initializing references from RI to RIB.

Co-authored-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-20 14:18:32 +05:30
Mohammad Arshad
fd7260b506
HBASE-25069: Display region name instead of encoded region name in HBCK report page. (#2428)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-20 09:00:28 +08:00
Joseph295
58618e35ad
HBASE-24991 Replace MovedRegionsCleaner with guava cache (#2357)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-19 14:53:13 +08:00