18116 Commits

Author SHA1 Message Date
WenFeiYi
4bbc772ffc
HBASE-25173 Remove owner related methods in TableDescriptor/TableDescriptorBuilder (#2541)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-27 22:01:57 +08:00
Duo Zhang
164cc5a3dc
HBASE-25215 TestClientOperationTimeout.testScanTimeout is flaky (#2583)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-26 10:03:43 +08:00
Reid Chan
e5d4e2fc81
HBASE-25189 [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level (#2549)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-25 17:46:14 +08:00
Toshihiro Suzuki
881c92b892
HBASE-25206 Data loss can happen if a cloned table loses original split region(delete table) (#2569)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-24 22:49:14 +08:00
Sandeep Pal
5e3ffb1db6
HBASE-25193: Add support for row prefix and type in the WAL Pretty Printer
Closes #2556

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-23 12:30:35 +05:30
sanjeetnishad95
d8de24c311
HBASE-25128 RSGroupInfo's toString() and hashCode() does not take into account configuration map. (#2484)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-23 08:42:00 +08:00
Duo Zhang
74df1e1aba
HBASE-25207 Revisit the implementation and usage of RegionStates.include (#2571)
Remove the RegionStates.include method as its name is ambiguous.
Add more comments to describe the logic on why we filter region like
this.

Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2020-10-22 21:50:43 +08:00
Jan Hentschel
2cd5f914af
HBASE-25197 Remove SingletonCoprocessorService interface (#2560)
Remove the SingletonCoprocessorService interface targeted for removal in
3.0.0.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-21 19:04:17 +02:00
Jan Hentschel
1b58af1841
HBASE-25198 Remove deprecated RpcSchedulerFactory#create (#2561)
Remove the deprecated RpcSchedulerFactory#create(Configuration,
PriorityFunction) method from the interface and in all implementing
classes.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-21 21:40:45 +08:00
Jan Hentschel
6c8f5f30d5
HBASE-25196 Add deprecation documentation to HConstants (#2559)
Add the documentation when
HConstants#REPLICATION_DROP_ON_DELETED_TABLE_KEY was deprecated and when
it is expected to be removed.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-21 09:18:35 +02:00
Duo Zhang
200343c59c
HBASE-25203 Change the reference url to flaky list in our jenkins jobs (#2566)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-20 17:00:06 +08:00
Guanghao Zhang
28f2c176ec
HBASE-25204 Nightly job failed as the name of jdk and maven changed (#2567)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-20 16:58:43 +08:00
Ankit Singhal
7d00e43dce
HBASE-25166 MobFileCompactionChore is closing the master's shared cluster connection (#2514) 2020-10-19 22:22:33 -07:00
Duo Zhang
90052ccf1b
HBASE-25194 Do not publish workspace in flaky find job (#2564)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-10-20 11:51:34 +08:00
shahrs87
e6c78f4037
HBASE-25179 : Fix Assert format in HFilePerformanceEvaluation class
Closes #2551

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-16 12:08:10 +05:30
Nick Dimiduk
70d2f37ee3
HBASE-24628 Region normalizer now respects a rate limit (HMaster chore shutdown NPE ADDENDUM) (#2540)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-15 22:02:53 +08:00
Duo Zhang
38ac7e26ae
HBASE-25186 TestMasterRegionOnTwoFileSystems is failing after HBASE-25065 (#2544)
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
2020-10-15 10:22:14 +08:00
XinSun
78b7244091
HBASE-25117 ReplicationSourceShipper thread can not be finished (#2521)
Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-14 18:08:54 +01:00
Sean Busbey
fd0ecadbb9
HBASE-14067 bundle ruby files for hbase shell into a jar.
* removes some cruft from the hbase-shell pom that appears to be from coping the hbase-server pom long ago
* puts the ruby scripts into the hbase-shell jar following the guide from jruby for packaging
* removes hard coding the location of the implementation scripts from our runtime
* removes hard coding the load path for the implementation from the test code (leaves hard coding the test code location)
* provides a work around for a name conflict between our shell and the ruby stdlib shell.

closes #2515

Signed-off-by: Michael Stack <stack@apache.org>
2020-10-13 11:12:07 -05:00
Nick Dimiduk
bc20203965
"HBASE-25144 Add Hadoop-3.3.0 to personality hadoopcheck" try again (#2536)
Due to HBASE-23834, HBASE-19256, HADOOP-16152, HBase 2.2.x and 2.3.x
cannot run on Hadoop 3.3.0, or any Hadoop version that has upgraded to
Jetty 9.4.

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-13 10:18:47 +08:00
Duo Zhang
b186c51796
HBASE-25169 Update documentation about meta region replica (#2528)
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-13 09:44:18 +08:00
Duo Zhang
92c3bcd9fb
HBASE-25164 Make ModifyTableProcedure support changing meta replica count (#2513)
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-13 09:43:56 +08:00
stack
c28220522d HBASE-25168 Unify WAL name timestamp parsers
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-10-12 07:52:26 -07:00
niuyulin
8eea052359
HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE (#2450)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-12 11:02:37 +08:00
niuyulin
3d1aaa6632
HBASE-25175 Remove the constructors of HBaseConfiguration (#2530)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>

Signed-off-by: Jan Hentschel <janh@apache.org>
2020-10-11 17:11:32 +02:00
Nick Dimiduk
c8c860c906
HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (ADDENDUM) (#2529)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-11 21:52:11 +08:00
bsglz
5a474449c1
HBASE-24840 Avoid shows closing region task when create table (#2226) 2020-10-11 19:12:56 +08:00
ramkrish86
accd9750aa
HBASE-25065 WAL archival to be done by a separate thread (#2501)
* HBASE-25065 WAL archival can be batched/throttled and also done by a separate thread

* Fix checkstyle issues

* Address review comments

* checkstyle comments

* Addressing final review comments
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-11 10:46:06 +05:30
Qi Yu
c367e91aff
HBASE-25162 Make flaky tests run more aggressively (#2525)
Change the trigger interval from 12h to 4h

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-10 16:09:39 +08:00
XinSun
b815207096
HBASE-25171 Remove ZNodePaths.namespaceZNode (#2526)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-10 16:09:12 +08:00
Duo Zhang
f34ba06468
HBASE-25163 Increase the timeout value for nightly jobs (#2512)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-10 08:22:12 +08:00
Michael Stack
665a8767a0
HBASE-22976 [HBCK2] Add RecoveredEditsPlayer (#2504)
Make it so WALPlayer can replay recovered.edits files.

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALInputFormat.java
 Allow for WAL files that do NOT have a startime in their name.
 Use the 'generic' WAL-filename parser instead of the one that
 used be local here. Implement support for 'startTime' filter.
 Previous was just not implemented.

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
 Checkstyle.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java
 Use the new general WAL name timestamp parser.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
 Utility for parsing timestamp from WAL filename.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRecoveredEdits.java
 Export attributes about the local recovered.edits test file
 so other tests can play with it.

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-10-09 08:46:05 -07:00
Nick Dimiduk
78ae1f176d HBASE-24628 Region normalizer now respects a rate limit
Implement a rate limiter for the normalizer. Implemented in terms of
MB/sec of affacted region size (the same metrics used to make
normalization decisions). Uses Guava `RateLimiter` to perform the
resource accounting. `RateLimiter` works by blocking (uninterruptible
😖) the calling thread. Thus, the whole construction of the normalizer
subsystem needed refactoring. See the provided `package-info.java` for
an overview of this new structure.

Introduces a new configuration,
`hbase.normalizer.throughput.max_bytes_per_sec`, for specifying a
limit on the throughput of actions executed by the normalizer. Note
that while this configuration value is in bytes, the minimum honored
valued `1_000_000`. Supports values configured using the
human-readable suffixes honored by `Configuration.getLongBytes`

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-08 15:45:08 -07:00
Nick Dimiduk
b82d8a5517
HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)
Change the test to wait for evidence that the active master has seen
that the backup master killed by the test has gone away. This is done
before proceeding to validate that the dead backup is correctly
omitted from the ClusterStatus report.

Also, minor fixup to several assertions, using `assertEquals` instead
of `assertTrue(...equals(...))` and correcting expected vs. actual
ordering of assertion arguments.

Signed-off-by: Michael Stack <stack@apache.org>
2020-10-08 14:23:38 -07:00
Mohammad Arshad
bdcafa895c
HBASE-24025: Improve performance of move_servers_rsgroup by using async region move API (#1549) 2020-10-08 10:30:16 -07:00
Michael Stack
8a995ae81b
HBASE-25165 Change 'State time' in UI so sorts (#2508)
Display startcode in iso8601.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-10-08 09:10:30 -07:00
Duo Zhang
0d63318f10
HBASE-25124 Support changing region replica count without disabling table (#2497)
Signed-off-by: stack <stack@apache.org>
2020-10-08 21:35:29 +08:00
Toshihiro Suzuki
aff8bbf0cb
HBASE-25160 Refactor AccessController and VisibilityController (#2506)
Signed-off-by: stack <stack@apache.org>
2020-10-08 17:04:48 +09:00
Semen Komissarov
2fc79e22b5
HBASE-23959 Fix javadoc for JDK11 (#2500)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: stack <stack@apache.org>
2020-10-07 21:42:35 -07:00
Pankaj
3537f9139e
HBASE-23742 Document that with split-to-hfile data over the MOB threshold will be treated as normal data (#2505)
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
2020-10-07 20:54:09 -07:00
Nick Dimiduk
a8c49a6463
HBASE-25144 Add Hadoop-3.3.0 to personality hadoopcheck (#2492)
Now that Hadoop 3.3.0 is released, let's figure out where it goes in
our testing matrix. Start by adding it to precommit checks.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Matt Foley <mattf@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-10-06 12:00:56 -07:00
Duo Zhang
e88509841f Revert "HBASE-24813 ReplicationSource should clear buffer usage on Replicatio… (#2191)"
This reverts commit 687e53b7e49c1a149e55829bbeca6aa4edfe69e7.
2020-10-06 21:09:00 +08:00
Andor Molnár
14b523ec98
HBASE-25146 Add extra logging at info level to HFileCorruptionChecker in order to report progress (#2503)
Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Signed-off-by: Mate Szalay-Beko <szalay.beko.mate@gmail.com>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2020-10-06 09:38:18 +02:00
申胜利
9f238bd79e
HBASE-24054 To be safe, jetty's version number should be blocked. 2020-10-05 17:36:13 -07:00
Duo Zhang
81f2cc5089
HBASE-25154 Set java.io.tmpdir to project build directory to avoid writing std*deferred files to /tmp (#2502)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-10-06 08:35:16 +08:00
Nick Dimiduk
16251dbb53
HBASE-25143 Remove branch-1.3 from precommit and docs (#2491)
Following the announcement [0] to EOL branch-1.3, update the precommit
script to not consider this branch any longer, and refresh mentions of
this branch in the doc.

[0]: https://lists.apache.org/thread.html/r9552e9085aaac2a43f8b26b866d34825a84a9be7f19118ac560d14de%40%3Cuser.hbase.apache.org%3E

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-10-05 11:51:59 -07:00
Duo Zhang
9ba90e1679
HBASE-25121 Refactor MetaTableAccessor.addRegionsToMeta and its usage places (#2476)
Signed-off-by: stack <stack@apache.org>
2020-10-05 21:29:55 +08:00
Viraj Jasani
23ce91819a
HBASE-25147 : Serialize regionNames in ReopenTableRegionsProcedure
Closes #2494

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-05 15:49:26 +05:30
Joseph295
a8096b3ac3
HBASE-25048 [HBCK2] Bypassed parent procedures are not updated in store (#2410)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-10-04 21:39:27 -07:00
niuyulin
3226c1795a
HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
Closes #2473

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-04 16:07:28 +05:30