Commit Graph

18261 Commits

Author SHA1 Message Date
sanjeetnishad95 d8de24c311
HBASE-25128 RSGroupInfo's toString() and hashCode() does not take into account configuration map. (#2484)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-23 08:42:00 +08:00
Duo Zhang 74df1e1aba
HBASE-25207 Revisit the implementation and usage of RegionStates.include (#2571)
Remove the RegionStates.include method as its name is ambiguous.
Add more comments to describe the logic on why we filter region like
this.

Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2020-10-22 21:50:43 +08:00
Jan Hentschel 2cd5f914af
HBASE-25197 Remove SingletonCoprocessorService interface (#2560)
Remove the SingletonCoprocessorService interface targeted for removal in
3.0.0.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-21 19:04:17 +02:00
Jan Hentschel 1b58af1841
HBASE-25198 Remove deprecated RpcSchedulerFactory#create (#2561)
Remove the deprecated RpcSchedulerFactory#create(Configuration,
PriorityFunction) method from the interface and in all implementing
classes.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-21 21:40:45 +08:00
Jan Hentschel 6c8f5f30d5
HBASE-25196 Add deprecation documentation to HConstants (#2559)
Add the documentation when
HConstants#REPLICATION_DROP_ON_DELETED_TABLE_KEY was deprecated and when
it is expected to be removed.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-21 09:18:35 +02:00
Duo Zhang 200343c59c
HBASE-25203 Change the reference url to flaky list in our jenkins jobs (#2566)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-20 17:00:06 +08:00
Guanghao Zhang 28f2c176ec
HBASE-25204 Nightly job failed as the name of jdk and maven changed (#2567)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-20 16:58:43 +08:00
Ankit Singhal 7d00e43dce
HBASE-25166 MobFileCompactionChore is closing the master's shared cluster connection (#2514) 2020-10-19 22:22:33 -07:00
Duo Zhang 90052ccf1b
HBASE-25194 Do not publish workspace in flaky find job (#2564)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-10-20 11:51:34 +08:00
shahrs87 e6c78f4037
HBASE-25179 : Fix Assert format in HFilePerformanceEvaluation class
Closes #2551

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-16 12:08:10 +05:30
Nick Dimiduk 70d2f37ee3
HBASE-24628 Region normalizer now respects a rate limit (HMaster chore shutdown NPE ADDENDUM) (#2540)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-15 22:02:53 +08:00
Duo Zhang 38ac7e26ae
HBASE-25186 TestMasterRegionOnTwoFileSystems is failing after HBASE-25065 (#2544)
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
2020-10-15 10:22:14 +08:00
XinSun 78b7244091
HBASE-25117 ReplicationSourceShipper thread can not be finished (#2521)
Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-14 18:08:54 +01:00
Sean Busbey fd0ecadbb9
HBASE-14067 bundle ruby files for hbase shell into a jar.
* removes some cruft from the hbase-shell pom that appears to be from coping the hbase-server pom long ago
* puts the ruby scripts into the hbase-shell jar following the guide from jruby for packaging
* removes hard coding the location of the implementation scripts from our runtime
* removes hard coding the load path for the implementation from the test code (leaves hard coding the test code location)
* provides a work around for a name conflict between our shell and the ruby stdlib shell.

closes #2515

Signed-off-by: Michael Stack <stack@apache.org>
2020-10-13 11:12:07 -05:00
Nick Dimiduk bc20203965
"HBASE-25144 Add Hadoop-3.3.0 to personality hadoopcheck" try again (#2536)
Due to HBASE-23834, HBASE-19256, HADOOP-16152, HBase 2.2.x and 2.3.x
cannot run on Hadoop 3.3.0, or any Hadoop version that has upgraded to
Jetty 9.4.

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-13 10:18:47 +08:00
Duo Zhang b186c51796
HBASE-25169 Update documentation about meta region replica (#2528)
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-13 09:44:18 +08:00
Duo Zhang 92c3bcd9fb
HBASE-25164 Make ModifyTableProcedure support changing meta replica count (#2513)
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-13 09:43:56 +08:00
stack c28220522d HBASE-25168 Unify WAL name timestamp parsers
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-10-12 07:52:26 -07:00
niuyulin 8eea052359
HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE (#2450)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-12 11:02:37 +08:00
niuyulin 3d1aaa6632
HBASE-25175 Remove the constructors of HBaseConfiguration (#2530)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>

Signed-off-by: Jan Hentschel <janh@apache.org>
2020-10-11 17:11:32 +02:00
Nick Dimiduk c8c860c906
HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (ADDENDUM) (#2529)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-11 21:52:11 +08:00
bsglz 5a474449c1
HBASE-24840 Avoid shows closing region task when create table (#2226) 2020-10-11 19:12:56 +08:00
ramkrish86 accd9750aa
HBASE-25065 WAL archival to be done by a separate thread (#2501)
* HBASE-25065 WAL archival can be batched/throttled and also done by a separate thread

* Fix checkstyle issues

* Address review comments

* checkstyle comments

* Addressing final review comments
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-11 10:46:06 +05:30
Qi Yu c367e91aff
HBASE-25162 Make flaky tests run more aggressively (#2525)
Change the trigger interval from 12h to 4h

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-10 16:09:39 +08:00
XinSun b815207096
HBASE-25171 Remove ZNodePaths.namespaceZNode (#2526)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-10 16:09:12 +08:00
Duo Zhang f34ba06468
HBASE-25163 Increase the timeout value for nightly jobs (#2512)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-10 08:22:12 +08:00
Michael Stack 665a8767a0
HBASE-22976 [HBCK2] Add RecoveredEditsPlayer (#2504)
Make it so WALPlayer can replay recovered.edits files.

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALInputFormat.java
 Allow for WAL files that do NOT have a startime in their name.
 Use the 'generic' WAL-filename parser instead of the one that
 used be local here. Implement support for 'startTime' filter.
 Previous was just not implemented.

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
 Checkstyle.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java
 Use the new general WAL name timestamp parser.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
 Utility for parsing timestamp from WAL filename.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRecoveredEdits.java
 Export attributes about the local recovered.edits test file
 so other tests can play with it.

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-10-09 08:46:05 -07:00
Nick Dimiduk 78ae1f176d HBASE-24628 Region normalizer now respects a rate limit
Implement a rate limiter for the normalizer. Implemented in terms of
MB/sec of affacted region size (the same metrics used to make
normalization decisions). Uses Guava `RateLimiter` to perform the
resource accounting. `RateLimiter` works by blocking (uninterruptible
😖) the calling thread. Thus, the whole construction of the normalizer
subsystem needed refactoring. See the provided `package-info.java` for
an overview of this new structure.

Introduces a new configuration,
`hbase.normalizer.throughput.max_bytes_per_sec`, for specifying a
limit on the throughput of actions executed by the normalizer. Note
that while this configuration value is in bytes, the minimum honored
valued `1_000_000`. Supports values configured using the
human-readable suffixes honored by `Configuration.getLongBytes`

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-08 15:45:08 -07:00
Nick Dimiduk b82d8a5517
HBASE-25156 TestMasterFailover.testSimpleMasterFailover is flaky (#2507)
Change the test to wait for evidence that the active master has seen
that the backup master killed by the test has gone away. This is done
before proceeding to validate that the dead backup is correctly
omitted from the ClusterStatus report.

Also, minor fixup to several assertions, using `assertEquals` instead
of `assertTrue(...equals(...))` and correcting expected vs. actual
ordering of assertion arguments.

Signed-off-by: Michael Stack <stack@apache.org>
2020-10-08 14:23:38 -07:00
Mohammad Arshad bdcafa895c
HBASE-24025: Improve performance of move_servers_rsgroup by using async region move API (#1549) 2020-10-08 10:30:16 -07:00
Michael Stack 8a995ae81b
HBASE-25165 Change 'State time' in UI so sorts (#2508)
Display startcode in iso8601.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-10-08 09:10:30 -07:00
Duo Zhang 0d63318f10
HBASE-25124 Support changing region replica count without disabling table (#2497)
Signed-off-by: stack <stack@apache.org>
2020-10-08 21:35:29 +08:00
Toshihiro Suzuki aff8bbf0cb
HBASE-25160 Refactor AccessController and VisibilityController (#2506)
Signed-off-by: stack <stack@apache.org>
2020-10-08 17:04:48 +09:00
Semen Komissarov 2fc79e22b5
HBASE-23959 Fix javadoc for JDK11 (#2500)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: stack <stack@apache.org>
2020-10-07 21:42:35 -07:00
Pankaj 3537f9139e
HBASE-23742 Document that with split-to-hfile data over the MOB threshold will be treated as normal data (#2505)
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
2020-10-07 20:54:09 -07:00
Nick Dimiduk a8c49a6463
HBASE-25144 Add Hadoop-3.3.0 to personality hadoopcheck (#2492)
Now that Hadoop 3.3.0 is released, let's figure out where it goes in
our testing matrix. Start by adding it to precommit checks.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Matt Foley <mattf@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-10-06 12:00:56 -07:00
Duo Zhang e88509841f Revert "HBASE-24813 ReplicationSource should clear buffer usage on Replicatio… (#2191)"
This reverts commit 687e53b7e4.
2020-10-06 21:09:00 +08:00
Andor Molnár 14b523ec98
HBASE-25146 Add extra logging at info level to HFileCorruptionChecker in order to report progress (#2503)
Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Signed-off-by: Mate Szalay-Beko <szalay.beko.mate@gmail.com>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2020-10-06 09:38:18 +02:00
申胜利 9f238bd79e
HBASE-24054 To be safe, jetty's version number should be blocked. 2020-10-05 17:36:13 -07:00
Duo Zhang 81f2cc5089
HBASE-25154 Set java.io.tmpdir to project build directory to avoid writing std*deferred files to /tmp (#2502)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-10-06 08:35:16 +08:00
Nick Dimiduk 16251dbb53
HBASE-25143 Remove branch-1.3 from precommit and docs (#2491)
Following the announcement [0] to EOL branch-1.3, update the precommit
script to not consider this branch any longer, and refresh mentions of
this branch in the doc.

[0]: https://lists.apache.org/thread.html/r9552e9085aaac2a43f8b26b866d34825a84a9be7f19118ac560d14de%40%3Cuser.hbase.apache.org%3E

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-10-05 11:51:59 -07:00
Duo Zhang 9ba90e1679
HBASE-25121 Refactor MetaTableAccessor.addRegionsToMeta and its usage places (#2476)
Signed-off-by: stack <stack@apache.org>
2020-10-05 21:29:55 +08:00
Viraj Jasani 23ce91819a
HBASE-25147 : Serialize regionNames in ReopenTableRegionsProcedure
Closes #2494

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-05 15:49:26 +05:30
Joseph295 a8096b3ac3
HBASE-25048 [HBCK2] Bypassed parent procedures are not updated in store (#2410)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-10-04 21:39:27 -07:00
niuyulin 3226c1795a
HBASE-25115 HFilePrettyPrinter can't seek to the row which is the first row of a hfile
Closes #2473

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-04 16:07:28 +05:30
Duo Zhang b0170d0e24
HBASE-25080 Should not use AssignmentManager to test whether a table is enabled or not (#2436)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-03 20:48:38 +08:00
SteNicholas 5e5166dc40
HBASE-25120 Remove the deprecated annotation for MetaTableAccessor.getScanForTableName (#2493)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-02 21:27:01 +08:00
Michael Stack 3b91a15183
HBASE-25091 Move LogComparator from ReplicationSource to AbstractFSWALProvider#.WALsStartTimeComparator (#2449)
Give the comparator a more descriptive name, a better location,
and make it work even when passed hbase:meta WAL files.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-01 10:04:58 -07:00
Duo Zhang 11a336a74a Revert "HBASE-25134 Migrate HBase PreCommit jenkins job from Hadoop to hbase (#2488)"
This reverts commit 5351aca8a1.
2020-10-01 22:15:15 +08:00
ramkrish86 a0b3d6276f
HBASE-25135 Convert the internal seperator while emitting the memstore read metrics to # (#2486)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-10-01 18:07:28 +05:30