Commit Graph

294 Commits

Author SHA1 Message Date
Nick Dimiduk 29475d6edb Preparing development version 2.5.1-SNAPSHOT
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-23 14:04:18 +00:00
Nick Dimiduk 6cdbc46c02 Preparing hbase release 2.5.0RC1; tagging and updates to CHANGES.md and RELEASENOTES.md
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-23 14:03:59 +00:00
LiangJun He 78f587f559 HBASE-27104 Add a tool command list_unknownservers (#4714)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 1bd0b581cb)
2022-08-22 21:26:19 +08:00
Duo Zhang b47ca399c5 HBASE-27141 Upgrade hbase-thirdparty dependency to 4.1.1 (#4552)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
(cherry picked from commit 9bae50f83c)
2022-06-22 23:19:10 +08:00
LiangJun He 53b57d686b HBASE-27028 Add a shell command for flushing master local region (#4539)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 666aa064e7)
2022-06-21 13:35:38 +08:00
Andrew Purtell b7158a87ea Preparing development version 2.5.1-SNAPSHOT
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-31 20:06:32 -07:00
Andrew Purtell 2da2dd917f Preparing hbase release 2.5.0RC0; tagging and updates to CHANGES.md and RELEASENOTES.md
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-31 20:06:29 -07:00
Duo Zhang 1aea663c6d HBASE-26899 Run spotless:apply 2022-05-01 22:52:40 +08:00
Bryan Beaudreault dc86de467b HBASE-26807 Unify CallQueueTooBigException special pause with CallDroppedException (#4273)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-04-22 08:58:35 -04:00
Andrew Purtell abb1bf2617 Revert "HBASE-26306 Backport "HBASE-26220 Use P2P communicate between region servers to sync the list for bootstrap node" to branch-2 (#3727)"
This reverts commit 10584d70d2.

See HBASE-26937 for discussion.

Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClusterConnection.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
2022-04-08 09:14:01 -07:00
Duo Zhang 6c43e62163 HBASE-26587 Introduce a new Admin API to change SFT implementation (#4030) (#4080)
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
Reviewed-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang af921c3546 HBASE-26654 ModifyTableDescriptorProcedure shoud load TableDescriptor while executing (#4034)
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2022-03-26 09:30:58 -07:00
BukrosSzabolcs 16ad5f777d HBASE-26286: Add support for specifying store file tracker when restoring or cloning snapshot
Closes #3851

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
GeorryHuang baac21b047 HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables (#3700)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang 60135108f4 HBASE-25988 Store the store file list by a file (#3578)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang 5844b53dea HBASE-26802 Backport the log4j2 changes to branch-2 (#4166)
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-hadoop-compat/pom.xml
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
	hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml
	hbase-shaded/hbase-shaded-client/pom.xml
	hbase-shaded/hbase-shaded-mapreduce/pom.xml
	hbase-shaded/hbase-shaded-testing-util/pom.xml
	hbase-shaded/pom.xml
	hbase-testing-util/pom.xml
2022-03-11 11:38:37 -08:00
Andrew Purtell 492f8f23fd Amend HBASE-26691 Replacing log4j with reload4j for branch-2.x
- Update reload4j version to 1.2.19
- Update shading and assembly configurations

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-04 12:25:41 -08:00
Andrew Purtell a19dcc3be0 HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks' (#4094)
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-02-15 18:39:03 -08:00
Nick Dimiduk f1add382bd HBASE-26542 Apply a `package` to test protobuf files
This is needed in a couple places in order to test that traces over the IPC layer carry correct
span names, and it's good hygiene anyway.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-15 16:29:23 -08:00
Bryan Beaudreault 9cf0af6e79 HBASE-26537: Make HBASE-15676 backwards compatible, using a flag on the proto (#3931)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-12-13 13:34:10 -08:00
Andrew Purtell 4a929175a5 HBASE-26541 hbase-shaded-protobuf not buildable on M1 MacOSX
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-12-08 16:51:11 -08:00
LiangJun He ec52519f7c HBASE-26462 Should persist restoreAcl flag in the procedure state for CloneSnapshotProcedure and RestoreSnapshotProcedure (#3921)
Signed-off-by: Yu Li <liyu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-06 23:16:15 +08:00
Duo Zhang 10584d70d2
HBASE-26306 Backport "HBASE-26220 Use P2P communicate between region servers to sync the list for bootstrap node" to branch-2 (#3727)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-10-10 22:56:12 +08:00
Bryan Beaudreault e07928df4c HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>

(Re-application of HBASE-26147 with the correct author metadata)
2021-09-01 21:58:30 -04:00
Josh Elser 1e74f08403 Revert "HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed"
This reverts commit 5b58d11c91.

I accidentally applied this as myself instead of as Bryan. Reverting it
to apply the patch properly.
2021-09-01 21:57:46 -04:00
Josh Elser 5b58d11c91 HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org
2021-09-01 21:47:53 -04:00
Tak Lon (Stephen) Wu 4dfc59c19e HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 (#3543)
2/17 commits of HBASE-22120

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 665305cc3b HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529)
1/17 commits of HBASE-22120

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
stack 243e861985 Revert "HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned (#3532)"
This reverts commit 8f16e34eb2.
2021-08-11 07:41:22 -07:00
Bryan Beaudreault 8f16e34eb2
HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned (#3532)
Signed-off-by: stack <stack@apache.org>
2021-08-10 20:38:06 -07:00
Tak Lon (Stephen) Wu c11a3e1b39 Revert "HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529)"
This reverts commit f049301606.
2021-08-04 15:55:13 -07:00
Tak Lon (Stephen) Wu fffdcba5bb Revert "HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 (#3543)"
This reverts commit ca096437d7.
2021-08-04 15:55:13 -07:00
Duo Zhang 620806e3fb HBASE-26150 Let region server also carry ClientMetaService (#3550)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-05 00:15:49 +08:00
Tak Lon (Stephen) Wu ca096437d7
HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 (#3543)
2/17 commits of HBASE-22120

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-08-02 11:33:19 -07:00
Tak Lon (Stephen) Wu f049301606
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529)
1/17 commits of HBASE-22120

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-07-29 09:15:10 -07:00
Duo Zhang 748e369629 HBASE-26029 It is not reliable to use nodeDeleted event to track region server's death (#3430)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-06-30 11:46:09 +08:00
Andrew Purtell 2d591ab3c4 HBASE-25869 WAL value compression (#3244)
WAL storage can be expensive, especially if the cell values
represented in the edits are large, consisting of blobs or
significant lengths of text. Such WALs might need to be kept around
for a fairly long time to satisfy replication constraints on a space
limited (or space-contended) filesystem.

We have a custom dictionary compression scheme for cell metadata that
is engaged when WAL compression is enabled in site configuration.
This is fine for that application, where we can expect the universe
of values and their lengths in the custom dictionaries to be
constrained. For arbitrary cell values it is better to use one of the
available compression codecs, which are suitable for arbitrary albeit
compressible data.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-21 11:53:36 -07:00
GeorryHuang 63d49cb7ed
HBASE-25790 NamedQueue 'BalancerRejection' for recent history of balancer skipping (#3182) (#3245)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-09 00:09:23 +05:30
Toshihiro Suzuki 1d3ea38f1e
HBASE-25703 Support conditional update in MultiRowMutationEndpoint (#3107)
Signed-off-by: Michael Stack <stack@apache.org>
2021-03-30 23:04:29 +09:00
Peter Somogyi af8dc2e4df HBASE-25587 [hbck2] Schedule SCP for all unknown servers (#2978)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-03-11 17:22:08 +01:00
Wellington Ramos Chevreuil 7adc00044c HBASE-25548 Optionally allow snapshots to preserve cluster's max file… (#2923)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit 373dc7788d)
2021-03-10 10:35:58 +00:00
Viraj Jasani 0788547fea
HBASE-25474 : Bump HBase version on branch-2 (#2871)
Signed-off-by: stack <stack@apache.org>
2021-01-12 10:20:22 +05:30
Mate Szalay-Beko 9fe62c3ec6
HBASE-25181 Add options for disabling column family encryption and choosing hash algorithm for wrapped encryption keys.
Prior to this patch hbase always used the MD5 hash algorithm to store a hash for encryption keys.
This hash is needed to verify the secret key of the subject. (e.g. making
sure that the same secrey key is used during encrypted HFile read and write).
The MD5 algorithm is considered weak, and can not be used in some
(e.g. FIPS compliant) clusters.

In this patch we:
- add a config parameter to globally enable/disable column family encryption (def enabled)
- introduce a backward compatible way of specifying the hash algorithm.
  This enable us to use newer and more secure hash algorithms like SHA-384
  or SHA-512 (which are FIPS compliant).
- add a config parameter to fail if an hfile is encountered that uses a
  different hash algorithm than the one currently configured to ease validation after
  migrating key hash algorithms (def disabled)

Closes #2539

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
(cherry picked from commit 6a5c928539)
2020-11-09 14:15:22 -06:00
Michael Stack df0009d879
HBASE-25238 Upgrading HBase from 2.2.0 to 2.3.x fails because of “Message missing required fields: state” (#2625)
Make protobuf fields add post-2.0.0 release marked 'required' instead
be 'optional' so migrations from 2.0.x to 2.1+ or 2.2+ succeeds.
2020-11-05 08:36:55 -08:00
bsglz 3a45fc17e5
HBASE-24967 The table.jsp cost long time to load if the table include closed regions (#2613) 2020-11-01 21:11:15 +08:00
bsglz dd6974fda8
HBASE-24875 Remove the force param for unassign since it dose not take effect any more (#2254)
Modified compared to main branch to deprecate obviated MasterObserver interface methods instead of remove them.

Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit c5ca191921)

 Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/AdminOverAsyncAdmin.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/rsgroup/VerifyingRSGroupAdmin.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestWithDisabledAuthorization.java
2020-10-13 16:29:03 -05:00
Duo Zhang 703b0613bd HBASE-25124 Support changing region replica count without disabling table (#2497)
Signed-off-by: stack <stack@apache.org>
2020-10-08 22:00:05 +08:00
Viraj Jasani 283bb52f7e
HBASE-25147 : Serialize regionNames in ReopenTableRegionsProcedure
Closes #2494

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-05 15:50:57 +05:30
Viraj Jasani 4316dc738c
HBASE-24528 : BalancerDecision queue implementation in HMaster with Admin API (#2411)
* Admin API getLogEntries() for ring buffer use-cases: so far, provides balancerDecision and slowLogResponse
* Refactor RPC call for similar use-cases
* Single RPC API getLogEntries() for both Master.proto and Admin.proto

Closes #2261

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-09-17 18:51:42 +05:30
Bharath Vissapragada bb9121da77
HBASE-24765: Dynamic master discovery (#2314)
This patch adds the ability to discover newly added masters
dynamically on the master registry side. The trigger for the
re-fetch is either periodic (5 mins) or any registry RPC failure.
Master server information is cached in masters to avoid repeated
ZK lookups.

Updates the client side connection metrics to maintain a counter
per RPC type so that clients have visibility into counts grouped
by RPC method name.

I didn't add the method to ZK registry interface since there
is a design discussion going on in splittable meta doc. We can
add it later if needed.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 275a38e153)
2020-08-26 09:13:34 -07:00