Commit Graph

18312 Commits

Author SHA1 Message Date
XinSun 9419c78d26
HBASE-25289 [testing] Clean up resources after tests in rsgroup_shell_test.rb (#2659)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-18 09:32:06 +08:00
Sean Busbey ca129e99ce
HBASE-25083 further HBase 1.y releases should have Hadoop 2.10 as a minimum version. (#2656) 2020-11-17 08:49:00 -06:00
Mate Szalay-Beko 6a529d3269 HBASE-25261 Upgrade Bootstrap to 3.4.1
HBase UI is currently using in bootstrap 3.3.7. This version is vulnerable to 4
medium CVEs (CVE-2018-14040, CVE-2018-14041, CVE-2018-14042, and CVE-2019-8331).
Details on all the bootstrap versions and vulnerabilities is
here: https://snyk.io/vuln/npm:bootstrap

Upgrading to bootstrap 4 would be nice, but potentially more work to do. We
should at least upgrade to the latest bootstrap 3, which is 3.4.1 currently.

closes #2661

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-11-17 12:44:20 +01:00
tianhang 322435d413
HBASE-25296 [Documentation] fix duplicate conf entry about upgrading (#2666)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-17 18:05:27 +08:00
gkanade 300b0a650f
HBASE-25026 Create a metric to track full region scans RPCs
Add new metric rpcFullScanRequestCount to track number of requests that are full region scans. Can be used to notify user to check if this is truly intended.
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by Ramkrishna S Vasudevan <ramkrishna@apache.org>
2020-11-17 08:48:59 +05:30
stack c07f27e025 Revert "HBASE-25280 [meta replicas] ArrayIndexOutOfBoundsException in ZKConnectionRegistry (#2652)"
This reverts commit 6210dafc47.

Applied to master when should have been applied to branch. Revert.
2020-11-16 08:59:12 -08:00
Michael Stack 6210dafc47
HBASE-25280 [meta replicas] ArrayIndexOutOfBoundsException in ZKConnectionRegistry (#2652)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-11-16 08:58:05 -08:00
stack 1c85c14994 Revert "HBASE-25280 [meta replicas] ArrayIndexOutOfBoundsException in ZKConnectionRegistry"
This reverts commit adbf81c010a5fbfebfcc4aae68e35dbd12da1601.

Premature. Still some comments to address.
2020-11-16 08:45:20 -08:00
stack 0aff1759a8 HBASE-25280 [meta replicas] ArrayIndexOutOfBoundsException in ZKConnectionRegistry
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-11-16 08:45:20 -08:00
clarax 4ee2270452
HBASE-25127 Enhance PerformanceEvaluation to profile meta replica performance. (#2644)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.com>
2020-11-14 10:45:42 -08:00
Hossein Zolfi f68f3dd799
HBASE-25273 fix typo in StripeStoreFileManager java doc (#2653)
Co-authored-by: Hossein Zolfi <h.zolfi@inside.sahab.ir>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-11-14 18:47:57 +01:00
stack bac459d693 HBASE-25284 Check-in "Enable memstore replication..." design 2020-11-14 09:12:03 -08:00
Duo Zhang 09aaa68be4 HBASE-25255 Addendum wait for meta loaded instead of master initialized for system table creation 2020-11-13 15:19:09 +08:00
Duo Zhang f89faf3ac8
HBASE-25255 Master fails to initialize when creating rs group table (#2638)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-13 14:28:31 +08:00
Peter Somogyi 035c192eb6
HBASE-25275 Upgrade asciidoctor (#2647)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-12 15:37:12 +01:00
WenFeiYi aaeeaa582e
HBASE-25253 Deprecated master carrys regions related methods and configs (#2635)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-12 22:13:24 +08:00
Mate Szalay-Beko 0611ca49ec
HBASE-25267 Add SSL keystore type and truststore related configs for HBase RESTServer (#2642)
HBASE-25267 Make SSL keystore type configurable in HBase RESTServer

In this patch I want to introduce the hbase.rest.ssl.keystore.type parameter,
enabling us to customize the keystore type for the REST server. If the
parameter is not provided, then we should fall-back to the current behaviour
(which assumes keystore type JKS).

This is similar to how we already configure the InfoServer objects with the
ssl.server.keystore.type parameter to set up HTTPS for the various admin UIs.

Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-11-12 12:37:43 +01:00
niuyulin 0b6d6fd773
HBASE-25276 Need to throw the original exception in HRegion#openHRegion (#2648)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-12 17:40:24 +08:00
ramkrish86 57d9cae480
HBASE-25187 Improve SizeCachedKV variants initialization (#2582)
* HBASE-25187 Improve SizeCachedKV variants initialization

* HBASE-25187 Improve SizeCachedKV variants initialization

* The BBKeyValue also can be optimized

* Change for SizeCachedKeyValue

* Addressing revew comments

* Fixing checkstyle and spot bugs comments

* Spot bug fix for hashCode

* Minor updates make the rowLen as short and some consturctor formatting

* Change two more places where there was a cast
2020-11-11 17:39:39 +05:30
Norbert Kalmar f0c430aed2
HBASE-20598 Upgrade to JRuby 9.2
- upgrade our default jruby to 9.2.13.0
- this major JRuby version update changes the Ruby compatibility from Ruby 2.3 to Ruby 2.5
- use a custom IRB prompt to convey similar information to before
- update the joni and jcoding dependencies to match this version of jruby-complete

closes #2308

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-11-09 16:31:44 -06:00
Mate Szalay-Beko 6a5c928539
HBASE-25181 Add options for disabling column family encryption and choosing hash algorithm for wrapped encryption keys.
Prior to this patch hbase always used the MD5 hash algorithm to store a hash for encryption keys.
This hash is needed to verify the secret key of the subject. (e.g. making
sure that the same secrey key is used during encrypted HFile read and write).
The MD5 algorithm is considered weak, and can not be used in some
(e.g. FIPS compliant) clusters.

In this patch we:
- add a config parameter to globally enable/disable column family encryption (def enabled)
- introduce a backward compatible way of specifying the hash algorithm.
  This enable us to use newer and more secure hash algorithms like SHA-384
  or SHA-512 (which are FIPS compliant).
- add a config parameter to fail if an hfile is encountered that uses a
  different hash algorithm than the one currently configured to ease validation after
  migrating key hash algorithms (def disabled)

Closes #2539

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
2020-11-09 13:53:50 -06:00
Duo Zhang 58c974888f
HBASE-25257 Remove MirroringTableStateManager (#2634)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-09 21:36:44 +08:00
gvprathyusha6 5c7432f4a9
HBASE-24667 Rename configs that support atypical DNS set ups to put them in hbase.unsafe
Closes #2542

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-09 12:27:34 +05:30
Duo Zhang c36ee4fac0
HBASE-25254 Rewrite TestMultiLogThreshold to remove the LogDelegate in RSRpcServices (#2631)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-08 21:47:18 +08:00
Duo Zhang 671129df56
HBASE-25252 Move HMaster inner classes out (#2628)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-11-07 20:05:04 +08:00
Michael Stack 23e656712b HBASE-25238 Upgrading HBase from 2.2.0 to 2.3.x fails because of “Message missing required fields: state” (#2625)
Make protobuf fields add post-2.0.0 release marked 'required' instead
be 'optional' so migrations from 2.0.x to 2.1+ or 2.2+ succeeds.

Signed-off-by: Viraj Jasani vjasani@apache.org
2020-11-05 08:49:47 -08:00
WenFeiYi 0356e8efd1
HBASE-25240 gson format of RpcServer.logResponse is abnormal
Closes #2623

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-05 19:58:05 +05:30
niuyulin 0e71d6192a
HBASE-25053 WAL replay should ignore 0-length files (#2437)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-04 08:21:31 -08:00
Viraj Jasani 4bd9ee43a4
HBASE-25245 : Fixing incorrect maven and jdk names for generate-hbase-website
Closes #2624

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-11-04 16:09:38 +05:30
Duo Zhang 49774c7e18
HBASE-25216 The client zk syncer should deal with meta replica count change (#2614)
Signed-off-by: Yu Li <liyu@apache.org>
2020-11-04 17:54:18 +08:00
Duo Zhang f37cd05c32
HBASE-25235 Cleanup the deprecated methods in TimeRange (#2616)
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-11-04 11:02:28 +08:00
Andrew Purtell c98e993b23
HBASE-25212 Optionally abort requests in progress after deciding a region should close (#2574)
If hbase.regionserver.close.wait.abort is set to true, interrupt RPC
handler threads holding the region close lock.

Until requests in progress can be aborted, wait on the region close lock for
a configurable interval (specified by hbase.regionserver.close.wait.time.ms,
default 60000 (1 minute)). If we have failed to acquire the close lock after
this interval elapses, if allowed (also specified by
hbase.regionserver.close.wait.abort), abort the regionserver.

We will attempt to interrupt any running handlers every
hbase.regionserver.close.wait.interval.ms (default 10000 (10 seconds)) until
either the close lock is acquired or we reach the maximum wait time.

Define a subset of region operations as interruptible. Track threads holding
the close lock transiting those operations. Set the thread interrupt status
of tracked threads when trying to close the region. Use the thread interrupt
status where safe to break out of request processing.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-11-03 15:20:27 -08:00
niuyulin 1eceab69b5
HBASE-25210 RegionInfo.isOffline is now a duplication with RegionInfo.isSplit (#2580)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-11-03 09:49:38 -08:00
Sandeep Pal 5abbda1969
HBASE-24859: Optimize in-memory representation of HBase map reduce table splits (#2609)
Patch fixes the single table input format case.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-11-03 07:39:23 -08:00
Nick Dimiduk 7a41247ef7
HBASE-25228 Delete dev-support/jenkins_precommit_jira_yetus.sh (#2611)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-11-02 13:11:09 -08:00
Viraj Jasani 85dfd9a349
HBASE-25218 : Add 2.3.3 to the downloads page
Closes #2615

Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-11-02 20:18:36 +05:30
Nick Dimiduk d790bdedde
HBASE-25167 Normalizer support for hot config reloading (#2523)
Wire up the `ConfigurationObserver` chain for
`RegionNormalizerManager`. The following configuration keys support
hot-reloading:
 * hbase.normalizer.throughput.max_bytes_per_sec
 * hbase.normalizer.split.enabled
 * hbase.normalizer.merge.enabled
 * hbase.normalizer.min.region.count
 * hbase.normalizer.merge.min_region_age.days
 * hbase.normalizer.merge.min_region_size.mb

Note that support for `hbase.normalizer.period` is not provided
here. Support would need to be implemented generally for the `Chore`
subsystem.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Aman Poonia <aman.poonia.29@gmail.com>
2020-10-30 10:41:56 -07:00
Pankaj e3beccf1fc
HBASE-24977 Meta table shouldn't be modified as read only (#2537)
Signed-off-by: stack <stack@apache.org>
2020-10-29 16:34:23 -07:00
Nick Dimiduk eee1cf7ff6 HBASE-24200 Upgrade to Yetus 0.12.0
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-29 15:45:01 -07:00
Nick Dimiduk 12d0397015 HBASE-24419 Normalizer merge plans should consider more than 2 regions when possible
The core change here is to the loop in
`SimpleRegionNormalizer#computeMergeNormalizationPlans`. It's a nested
loop that walks the table's region chain once, looking for contiguous
sequences of regions that meet the criteria for merge. The outer loop
tracks the starting point of the next sequence, the inner loop looks
for the end of that sequence. A single sequence becomes an instance of
`MergeNormalizationPlan`.

Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2020-10-29 13:37:23 -07:00
GeorryHuang 35b344c337 HBASE-25090 CompactionConfiguration logs unrealistic store file sizes (#2595)
Signed-off-by: stack <stack@apache.org>
2020-10-29 13:10:35 -07:00
Nick Dimiduk 259fe1984a
HBASE-24845 Git/Jira Release Audit: limit branches when building audit db (#2238)
Populating the audit database with release tag information from git is
time consuming. Until that's sorted out, give the user a flag for
limiting which branches they want to be reviewed.

Signed-off-by:  Andrew Purtell <apurtell@apache.org >
2020-10-29 09:21:18 -07:00
Peter Somogyi bb4a9d335f
HBASE-25224 Maximize sleep for checking meta and namespace regions availability (#2593)
Signed-off-by: Michael Stack <stack@apache.org>
2020-10-28 18:08:05 +01:00
Minji Kim 735689d0f7
HBASE-25223 Use try-with-resources statement (#2592)
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-10-28 09:17:31 -07:00
niuyulin 17f9aded2c
HBASE-25201 YouAreDeadException should be moved to hbase-server module (#2581)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-28 14:58:28 +08:00
niuyulin 485da758bb
HBASE-25176 MasterStoppedException should be moved to hbase-client module (#2538)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-28 14:52:27 +08:00
WenFeiYi 4bbc772ffc
HBASE-25173 Remove owner related methods in TableDescriptor/TableDescriptorBuilder (#2541)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-27 22:01:57 +08:00
Duo Zhang 164cc5a3dc
HBASE-25215 TestClientOperationTimeout.testScanTimeout is flaky (#2583)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-10-26 10:03:43 +08:00
Reid Chan e5d4e2fc81
HBASE-25189 [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level (#2549)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-10-25 17:46:14 +08:00
Toshihiro Suzuki 881c92b892
HBASE-25206 Data loss can happen if a cloned table loses original split region(delete table) (#2569)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-24 22:49:14 +08:00