17398 Commits

Author SHA1 Message Date
Baiqiang Zhao
fdff8ef46d HBASE-25298 hbase.rsgroup.fallback.enable should support dynamic configuration (#2668)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-19 09:20:43 +08:00
XinSun
9769e3fe7b HBASE-25300 'Unknown table hbase:quota' happens when desc table in shell if quota disabled (#2673)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2020-11-19 09:20:34 +08:00
Sean Busbey
4d41de1df0 HBASE-25083 further HBase 1.y releases should have Hadoop 2.10 as a minimum version. (#2656)
(cherry picked from commit ca129e99ce095c9fe9e4e13f4e37ade63700950b)
2020-11-18 14:55:26 -06:00
XinSun
d9b175cbb6
HBASE-25289 [testing] Clean up resources after tests in rsgroup_shell_test.rb (#2672)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-18 14:29:02 +08:00
Mate Szalay-Beko
e8ecf2e016 HBASE-25261 Upgrade Bootstrap to 3.4.1
HBase UI is currently using in bootstrap 3.3.7. This version is vulnerable to 4
medium CVEs (CVE-2018-14040, CVE-2018-14041, CVE-2018-14042, and CVE-2019-8331).
Details on all the bootstrap versions and vulnerabilities is
here: https://snyk.io/vuln/npm:bootstrap

Upgrading to bootstrap 4 would be nice, but potentially more work to do. We
should at least upgrade to the latest bootstrap 3, which is 3.4.1 currently.

closes #2661

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-11-17 17:15:57 +01:00
stack
4b9a14d9f3 Revert "HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)"
This reverts commit aa324c285326e08547dded231b26243681e2bf9b.
2020-11-14 11:15:29 -08:00
clarax
aa324c2853 HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
2020-11-14 10:46:34 -08:00
stack
13abb47d40 HBASE-25284 Check-in "Enable memstore replication..." design 2020-11-14 09:13:35 -08:00
Duo Zhang
f50ede32d9 HBASE-25255 Master fails to initialize when creating rs group table (#2638)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-13 15:36:16 +08:00
Peter Somogyi
d2f9f698fb HBASE-25275 Upgrade asciidoctor (#2647)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-12 16:03:42 +01:00
WenFeiYi
ea1375d6be HBASE-25253 Deprecated master carrys regions related methods and configs (#2635)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-12 22:23:30 +08:00
Duo Zhang
b19df076a4
HBASE-25272 Support scan on a specific replica (#2645)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2020-11-12 21:25:53 +08:00
Mate Szalay-Beko
873bef1d7e HBASE-25267 Add SSL keystore type and truststore related configs for HBase RESTServer (#2642)
HBASE-25267 Make SSL keystore type configurable in HBase RESTServer

In this patch I want to introduce the hbase.rest.ssl.keystore.type parameter,
enabling us to customize the keystore type for the REST server. If the
parameter is not provided, then we should fall-back to the current behaviour
(which assumes keystore type JKS).

This is similar to how we already configure the InfoServer objects with the
ssl.server.keystore.type parameter to set up HTTPS for the various admin UIs.

Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-11-12 12:43:58 +01:00
niuyulin
c48f6ebd09 HBASE-25276 Need to throw the original exception in HRegion#openHRegion (#2648)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-12 17:41:39 +08:00
Norbert Kalmar
ec3f843b23
HBASE-20598 Upgrade to JRuby 9.2
- upgrade our default jruby to 9.2.13.0
- this major JRuby version update changes the Ruby compatibility from Ruby 2.3 to Ruby 2.5
- use a custom IRB prompt to convey similar information to before
- update the joni and jcoding dependencies to match this version of jruby-complete

closes #2308

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit f0c430aed226b7404bcc4d23b0ec164e5071276b)
2020-11-09 16:50:34 -06:00
Mate Szalay-Beko
9fe62c3ec6
HBASE-25181 Add options for disabling column family encryption and choosing hash algorithm for wrapped encryption keys.
Prior to this patch hbase always used the MD5 hash algorithm to store a hash for encryption keys.
This hash is needed to verify the secret key of the subject. (e.g. making
sure that the same secrey key is used during encrypted HFile read and write).
The MD5 algorithm is considered weak, and can not be used in some
(e.g. FIPS compliant) clusters.

In this patch we:
- add a config parameter to globally enable/disable column family encryption (def enabled)
- introduce a backward compatible way of specifying the hash algorithm.
  This enable us to use newer and more secure hash algorithms like SHA-384
  or SHA-512 (which are FIPS compliant).
- add a config parameter to fail if an hfile is encountered that uses a
  different hash algorithm than the one currently configured to ease validation after
  migrating key hash algorithms (def disabled)

Closes #2539

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
(cherry picked from commit 6a5c9285390c0ae66adfb07f9dafbad69f89fa32)
2020-11-09 14:15:22 -06:00
gvprathyusha6
21b32d9a6f
HBASE-24667 Rename configs that support atypical DNS set ups to put them in hbase.unsafe
Closes #2542

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-11-09 12:41:04 +05:30
Duo Zhang
7a07dfe059 HBASE-25254 Rewrite TestMultiLogThreshold to remove the LogDelegate in RSRpcServices (#2631)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-08 22:16:03 +08:00
Duo Zhang
38512e6f38 HBASE-25252 Move HMaster inner classes out (#2628)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-07 20:14:39 +08:00
Michael Stack
df0009d879
HBASE-25238 Upgrading HBase from 2.2.0 to 2.3.x fails because of “Message missing required fields: state” (#2625)
Make protobuf fields add post-2.0.0 release marked 'required' instead
be 'optional' so migrations from 2.0.x to 2.1+ or 2.2+ succeeds.
2020-11-05 08:36:55 -08:00
WenFeiYi
de74312d74
HBASE-25240 gson format of RpcServer.logResponse is abnormal
Closes #2623

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-05 19:59:34 +05:30
niuyulin
3d8152b635 HBASE-25053 WAL replay should ignore 0-length files (#2437)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-04 08:22:14 -08:00
niuyulin
770098128a HBASE-25210 RegionInfo.isOffline is now a duplication with RegionInfo.isSplit (#2580)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-11-04 08:11:00 -08:00
Duo Zhang
04293ad6dd HBASE-25216 The client zk syncer should deal with meta replica count change (#2614)
Signed-off-by: Yu Li <liyu@apache.org>
2020-11-04 18:25:08 +08:00
Andrew Purtell
1cc39ffd59
HBASE-25212 [branch-2] Optionally abort requests in progress after deciding a region should close (#2575)
If hbase.regionserver.close.wait.abort is set to true, interrupt RPC
handler threads holding the region close lock.

Until requests in progress can be aborted, wait on the region close lock for
a configurable interval (specified by hbase.regionserver.close.wait.time.ms,
default 60000 (1 minute)). If we have failed to acquire the close lock after
this interval elapses, if allowed (also specified by
hbase.regionserver.close.wait.abort), abort the regionserver.

We will attempt to interrupt any running handlers every
hbase.regionserver.close.wait.interval.ms (default 10000 (10 seconds)) until
either the close lock is acquired or we reach the maximum wait time.

Define a subset of region operations as interruptible. Track threads holding
the close lock transiting those operations. Set the thread interrupt status
of tracked threads when trying to close the region. Use the thread interrupt
status where safe to break out of request processing.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-03 15:21:32 -08:00
Sandeep Pal
729518af9d
HBASE-24859: Optimize in-memory representation of HBase map reduce table splits (#2610)
Patch fixes the single table input format case.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-11-03 07:40:25 -08:00
Nick Dimiduk
ca45b1b45c HBASE-25228 Delete dev-support/jenkins_precommit_jira_yetus.sh (#2611)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-11-02 13:18:57 -08:00
bsglz
3a45fc17e5
HBASE-24967 The table.jsp cost long time to load if the table include closed regions (#2613) 2020-11-01 21:11:15 +08:00
Nick Dimiduk
1c7d472537 HBASE-25167 Normalizer support for hot config reloading (#2523)
Wire up the `ConfigurationObserver` chain for
`RegionNormalizerManager`. The following configuration keys support
hot-reloading:
 * hbase.normalizer.throughput.max_bytes_per_sec
 * hbase.normalizer.split.enabled
 * hbase.normalizer.merge.enabled
 * hbase.normalizer.min.region.count
 * hbase.normalizer.merge.min_region_age.days
 * hbase.normalizer.merge.min_region_size.mb

Note that support for `hbase.normalizer.period` is not provided
here. Support would need to be implemented generally for the `Chore`
subsystem.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Aman Poonia <aman.poonia.29@gmail.com>
2020-10-30 15:32:36 -07:00
Nick Dimiduk
5e3fa7df65 HBASE-24200 Upgrade to Yetus 0.12.0
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-30 14:00:11 -07:00
Nick Dimiduk
b84e2f598b HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
The core change here is to the loop in
`SimpleRegionNormalizer#computeMergeNormalizationPlans`. It's a nested
loop that walks the table's region chain once, looking for contiguous
sequences of regions that meet the criteria for merge. The outer loop
tracks the starting point of the next sequence, the inner loop looks
for the end of that sequence. A single sequence becomes an instance of
`MergeNormalizationPlan`.

Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2020-10-30 10:43:43 -07:00
Pankaj
40608c33ef HBASE-24977 Meta table shouldn't be modified as read only (#2537)
Signed-off-by: stack <stack@apache.org>
2020-10-29 16:35:07 -07:00
GeorryHuang
69f282e21a
HBASE-25090 CompactionConfiguration logs unrealistic store file sizes (#2595)
Signed-off-by: stack <stack@apache.org>
2020-10-29 13:09:18 -07:00
Peter Somogyi
b155c62f38 HBASE-25224 Maximize sleep for checking meta and namespace regions availability (#2593)
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-28 18:09:16 +01:00
Minji Kim
970aaf38bd HBASE-25223 Use try-with-resources statement (#2592)
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-10-28 09:20:35 -07:00
niuyulin
bfe92ea53b HBASE-25201 YouAreDeadException should be moved to hbase-server module (#2581)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-28 15:02:30 +08:00
niuyulin
1bf68514ea HBASE-25176 MasterStoppedException should be moved to hbase-client module (#2538)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-28 15:02:20 +08:00
Reid Chan
70631e901f HBASE-25189 [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level (#2549)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-25 17:47:58 +08:00
Toshihiro Suzuki
f7e6143f49 HBASE-25206 Data loss can happen if a cloned table loses original split region(delete table) (#2569)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-24 23:04:33 +08:00
Sandeep Pal
3862dfc930
HBASE-25193: Add support for row prefix and type in the WAL Pretty Printer
Closes #2556

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-23 12:47:31 +05:30
sanjeetnishad95
dd0a14538b HBASE-25128 RSGroupInfo's toString() and hashCode() does not take into account configuration map. (#2484)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-23 08:43:31 +08:00
Duo Zhang
fdea847130 HBASE-25207 Revisit the implementation and usage of RegionStates.include (#2571)
Remove the RegionStates.include method as its name is ambiguous.
Add more comments to describe the logic on why we filter region like
this.

Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2020-10-22 22:01:17 +08:00
Duo Zhang
83b189fb46
HBASE-25186 Addendum remove extra sleep to fix TestMasterRegionWALCleaner (#2577)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-22 10:54:57 +08:00
Duo Zhang
9d61afae6e
HBASE-25186 Addendum fix compile error (#2572)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-21 16:47:56 +08:00
Guanghao Zhang
aa888f7e38 HBASE-25204 Nightly job failed as the name of jdk and maven changed (#2567)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-20 17:21:08 +08:00
Duo Zhang
02e77e0ba7 HBASE-25203 Change the reference url to flaky list in our jenkins jobs (#2566)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-20 17:02:35 +08:00
Duo Zhang
0c20f2b212 HBASE-25194 Do not publish workspace in flaky find job (#2564)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-10-20 12:31:30 +08:00
shahrs87
06bb12ed83
HBASE-25179 : Fix Assert format in HFilePerformanceEvaluation class
Closes #2551

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-16 12:09:10 +05:30
Nick Dimiduk
fc4957f456 HBASE-24628 Region normalizer now respects a rate limit (HMaster chore shutdown NPE ADDENDUM) (#2540)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-15 22:06:40 +08:00
Duo Zhang
2f566dc234 HBASE-25186 TestMasterRegionOnTwoFileSystems is failing after HBASE-25065 (#2544)
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
2020-10-15 10:45:09 +08:00