Commit Graph

17798 Commits

Author SHA1 Message Date
Baiqiang Zhao fdff8ef46d HBASE-25298 hbase.rsgroup.fallback.enable should support dynamic configuration (#2668)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-19 09:20:43 +08:00
XinSun 9769e3fe7b HBASE-25300 'Unknown table hbase:quota' happens when desc table in shell if quota disabled (#2673)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2020-11-19 09:20:34 +08:00
Sean Busbey 4d41de1df0 HBASE-25083 further HBase 1.y releases should have Hadoop 2.10 as a minimum version. (#2656)
(cherry picked from commit ca129e99ce)
2020-11-18 14:55:26 -06:00
XinSun d9b175cbb6
HBASE-25289 [testing] Clean up resources after tests in rsgroup_shell_test.rb (#2672)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-18 14:29:02 +08:00
Mate Szalay-Beko e8ecf2e016 HBASE-25261 Upgrade Bootstrap to 3.4.1
HBase UI is currently using in bootstrap 3.3.7. This version is vulnerable to 4
medium CVEs (CVE-2018-14040, CVE-2018-14041, CVE-2018-14042, and CVE-2019-8331).
Details on all the bootstrap versions and vulnerabilities is
here: https://snyk.io/vuln/npm:bootstrap

Upgrading to bootstrap 4 would be nice, but potentially more work to do. We
should at least upgrade to the latest bootstrap 3, which is 3.4.1 currently.

closes #2661

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-11-17 17:15:57 +01:00
stack 4b9a14d9f3 Revert "HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)"
This reverts commit aa324c2853.
2020-11-14 11:15:29 -08:00
clarax aa324c2853 HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
2020-11-14 10:46:34 -08:00
stack 13abb47d40 HBASE-25284 Check-in "Enable memstore replication..." design 2020-11-14 09:13:35 -08:00
Duo Zhang f50ede32d9 HBASE-25255 Master fails to initialize when creating rs group table (#2638)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-13 15:36:16 +08:00
Peter Somogyi d2f9f698fb HBASE-25275 Upgrade asciidoctor (#2647)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-12 16:03:42 +01:00
WenFeiYi ea1375d6be HBASE-25253 Deprecated master carrys regions related methods and configs (#2635)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-12 22:23:30 +08:00
Duo Zhang b19df076a4
HBASE-25272 Support scan on a specific replica (#2645)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2020-11-12 21:25:53 +08:00
Mate Szalay-Beko 873bef1d7e HBASE-25267 Add SSL keystore type and truststore related configs for HBase RESTServer (#2642)
HBASE-25267 Make SSL keystore type configurable in HBase RESTServer

In this patch I want to introduce the hbase.rest.ssl.keystore.type parameter,
enabling us to customize the keystore type for the REST server. If the
parameter is not provided, then we should fall-back to the current behaviour
(which assumes keystore type JKS).

This is similar to how we already configure the InfoServer objects with the
ssl.server.keystore.type parameter to set up HTTPS for the various admin UIs.

Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-11-12 12:43:58 +01:00
niuyulin c48f6ebd09 HBASE-25276 Need to throw the original exception in HRegion#openHRegion (#2648)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-12 17:41:39 +08:00
Norbert Kalmar ec3f843b23
HBASE-20598 Upgrade to JRuby 9.2
- upgrade our default jruby to 9.2.13.0
- this major JRuby version update changes the Ruby compatibility from Ruby 2.3 to Ruby 2.5
- use a custom IRB prompt to convey similar information to before
- update the joni and jcoding dependencies to match this version of jruby-complete

closes #2308

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit f0c430aed2)
2020-11-09 16:50:34 -06:00
Mate Szalay-Beko 9fe62c3ec6
HBASE-25181 Add options for disabling column family encryption and choosing hash algorithm for wrapped encryption keys.
Prior to this patch hbase always used the MD5 hash algorithm to store a hash for encryption keys.
This hash is needed to verify the secret key of the subject. (e.g. making
sure that the same secrey key is used during encrypted HFile read and write).
The MD5 algorithm is considered weak, and can not be used in some
(e.g. FIPS compliant) clusters.

In this patch we:
- add a config parameter to globally enable/disable column family encryption (def enabled)
- introduce a backward compatible way of specifying the hash algorithm.
  This enable us to use newer and more secure hash algorithms like SHA-384
  or SHA-512 (which are FIPS compliant).
- add a config parameter to fail if an hfile is encountered that uses a
  different hash algorithm than the one currently configured to ease validation after
  migrating key hash algorithms (def disabled)

Closes #2539

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
(cherry picked from commit 6a5c928539)
2020-11-09 14:15:22 -06:00
gvprathyusha6 21b32d9a6f
HBASE-24667 Rename configs that support atypical DNS set ups to put them in hbase.unsafe
Closes #2542

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-11-09 12:41:04 +05:30
Duo Zhang 7a07dfe059 HBASE-25254 Rewrite TestMultiLogThreshold to remove the LogDelegate in RSRpcServices (#2631)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-08 22:16:03 +08:00
Duo Zhang 38512e6f38 HBASE-25252 Move HMaster inner classes out (#2628)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-07 20:14:39 +08:00
Michael Stack df0009d879
HBASE-25238 Upgrading HBase from 2.2.0 to 2.3.x fails because of “Message missing required fields: state” (#2625)
Make protobuf fields add post-2.0.0 release marked 'required' instead
be 'optional' so migrations from 2.0.x to 2.1+ or 2.2+ succeeds.
2020-11-05 08:36:55 -08:00
WenFeiYi de74312d74
HBASE-25240 gson format of RpcServer.logResponse is abnormal
Closes #2623

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-05 19:59:34 +05:30
niuyulin 3d8152b635 HBASE-25053 WAL replay should ignore 0-length files (#2437)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-04 08:22:14 -08:00
niuyulin 770098128a HBASE-25210 RegionInfo.isOffline is now a duplication with RegionInfo.isSplit (#2580)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-11-04 08:11:00 -08:00
Duo Zhang 04293ad6dd HBASE-25216 The client zk syncer should deal with meta replica count change (#2614)
Signed-off-by: Yu Li <liyu@apache.org>
2020-11-04 18:25:08 +08:00
Andrew Purtell 1cc39ffd59
HBASE-25212 [branch-2] Optionally abort requests in progress after deciding a region should close (#2575)
If hbase.regionserver.close.wait.abort is set to true, interrupt RPC
handler threads holding the region close lock.

Until requests in progress can be aborted, wait on the region close lock for
a configurable interval (specified by hbase.regionserver.close.wait.time.ms,
default 60000 (1 minute)). If we have failed to acquire the close lock after
this interval elapses, if allowed (also specified by
hbase.regionserver.close.wait.abort), abort the regionserver.

We will attempt to interrupt any running handlers every
hbase.regionserver.close.wait.interval.ms (default 10000 (10 seconds)) until
either the close lock is acquired or we reach the maximum wait time.

Define a subset of region operations as interruptible. Track threads holding
the close lock transiting those operations. Set the thread interrupt status
of tracked threads when trying to close the region. Use the thread interrupt
status where safe to break out of request processing.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-03 15:21:32 -08:00
Sandeep Pal 729518af9d
HBASE-24859: Optimize in-memory representation of HBase map reduce table splits (#2610)
Patch fixes the single table input format case.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-11-03 07:40:25 -08:00
Nick Dimiduk ca45b1b45c HBASE-25228 Delete dev-support/jenkins_precommit_jira_yetus.sh (#2611)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-11-02 13:18:57 -08:00
bsglz 3a45fc17e5
HBASE-24967 The table.jsp cost long time to load if the table include closed regions (#2613) 2020-11-01 21:11:15 +08:00
Nick Dimiduk 1c7d472537 HBASE-25167 Normalizer support for hot config reloading (#2523)
Wire up the `ConfigurationObserver` chain for
`RegionNormalizerManager`. The following configuration keys support
hot-reloading:
 * hbase.normalizer.throughput.max_bytes_per_sec
 * hbase.normalizer.split.enabled
 * hbase.normalizer.merge.enabled
 * hbase.normalizer.min.region.count
 * hbase.normalizer.merge.min_region_age.days
 * hbase.normalizer.merge.min_region_size.mb

Note that support for `hbase.normalizer.period` is not provided
here. Support would need to be implemented generally for the `Chore`
subsystem.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Aman Poonia <aman.poonia.29@gmail.com>
2020-10-30 15:32:36 -07:00
Nick Dimiduk 5e3fa7df65 HBASE-24200 Upgrade to Yetus 0.12.0
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-30 14:00:11 -07:00
Nick Dimiduk b84e2f598b HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
The core change here is to the loop in
`SimpleRegionNormalizer#computeMergeNormalizationPlans`. It's a nested
loop that walks the table's region chain once, looking for contiguous
sequences of regions that meet the criteria for merge. The outer loop
tracks the starting point of the next sequence, the inner loop looks
for the end of that sequence. A single sequence becomes an instance of
`MergeNormalizationPlan`.

Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2020-10-30 10:43:43 -07:00
Pankaj 40608c33ef HBASE-24977 Meta table shouldn't be modified as read only (#2537)
Signed-off-by: stack <stack@apache.org>
2020-10-29 16:35:07 -07:00
GeorryHuang 69f282e21a
HBASE-25090 CompactionConfiguration logs unrealistic store file sizes (#2595)
Signed-off-by: stack <stack@apache.org>
2020-10-29 13:09:18 -07:00
Peter Somogyi b155c62f38 HBASE-25224 Maximize sleep for checking meta and namespace regions availability (#2593)
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-28 18:09:16 +01:00
Minji Kim 970aaf38bd HBASE-25223 Use try-with-resources statement (#2592)
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-10-28 09:20:35 -07:00
niuyulin bfe92ea53b HBASE-25201 YouAreDeadException should be moved to hbase-server module (#2581)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-28 15:02:30 +08:00
niuyulin 1bf68514ea HBASE-25176 MasterStoppedException should be moved to hbase-client module (#2538)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-28 15:02:20 +08:00
Reid Chan 70631e901f HBASE-25189 [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level (#2549)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-25 17:47:58 +08:00
Toshihiro Suzuki f7e6143f49 HBASE-25206 Data loss can happen if a cloned table loses original split region(delete table) (#2569)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-24 23:04:33 +08:00
Sandeep Pal 3862dfc930
HBASE-25193: Add support for row prefix and type in the WAL Pretty Printer
Closes #2556

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-23 12:47:31 +05:30
sanjeetnishad95 dd0a14538b HBASE-25128 RSGroupInfo's toString() and hashCode() does not take into account configuration map. (#2484)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-23 08:43:31 +08:00
Duo Zhang fdea847130 HBASE-25207 Revisit the implementation and usage of RegionStates.include (#2571)
Remove the RegionStates.include method as its name is ambiguous.
Add more comments to describe the logic on why we filter region like
this.

Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2020-10-22 22:01:17 +08:00
Duo Zhang 83b189fb46
HBASE-25186 Addendum remove extra sleep to fix TestMasterRegionWALCleaner (#2577)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-22 10:54:57 +08:00
Duo Zhang 9d61afae6e
HBASE-25186 Addendum fix compile error (#2572)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-21 16:47:56 +08:00
Guanghao Zhang aa888f7e38 HBASE-25204 Nightly job failed as the name of jdk and maven changed (#2567)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-20 17:21:08 +08:00
Duo Zhang 02e77e0ba7 HBASE-25203 Change the reference url to flaky list in our jenkins jobs (#2566)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-20 17:02:35 +08:00
Duo Zhang 0c20f2b212 HBASE-25194 Do not publish workspace in flaky find job (#2564)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-10-20 12:31:30 +08:00
shahrs87 06bb12ed83
HBASE-25179 : Fix Assert format in HFilePerformanceEvaluation class
Closes #2551

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-16 12:09:10 +05:30
Nick Dimiduk fc4957f456 HBASE-24628 Region normalizer now respects a rate limit (HMaster chore shutdown NPE ADDENDUM) (#2540)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-15 22:06:40 +08:00
Duo Zhang 2f566dc234 HBASE-25186 TestMasterRegionOnTwoFileSystems is failing after HBASE-25065 (#2544)
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
2020-10-15 10:45:09 +08:00