Commit Graph

296 Commits

Author SHA1 Message Date
BukrosSzabolcs 771e552cf4 HBASE-26286: Add support for specifying store file tracker when restoring or cloning snapshot
Closes #3851

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2021-12-22 15:05:59 -05:00
Duo Zhang 43b40e9374 HBASE-25988 Store the store file list by a file (#3578)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-12-22 15:05:58 -05:00
Nick Dimiduk c93e4572c1 HBASE-26542 Apply a `package` to test protobuf files
This is needed in a couple places in order to test that traces over the IPC layer carry correct
span names, and it's good hygiene anyway.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-15 15:52:43 -08:00
Duo Zhang 8bca21b47d
HBASE-26558 Set version as 3.0.0-alpha-2 in master in prep for first RC of 3.0.0-alpha-2 (#3935)
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
2021-12-11 20:52:35 +08:00
Andrew Purtell b5cf3cdc5c HBASE-26541 hbase-shaded-protobuf not buildable on M1 MacOSX
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-12-08 16:57:39 -08:00
LiangJun He 9cf224d4fe
HBASE-26462 Should persist restoreAcl flag in the procedure state for CloneSnapshotProcedure and RestoreSnapshotProcedure (#3921)
Signed-off-by: Yu Li <liyu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-06 22:58:38 +08:00
Duo Zhang 6e7e69a0a7
HBASE-26348 Implement a special procedure to migrate rs group information when upgrading (#3746)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Reviewed-by: GeorryHuang <215175212@qq.com>
2021-10-14 22:51:37 +08:00
Duo Zhang 1152a61b5b
HBASE-26220 Use P2P communicate between region servers to sync the list for bootstrap node (#3697)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-09-29 21:12:58 +08:00
Mallikarjun 04c38883af HBASE-25891 Remove dependence on storing WAL filenames for backup (#3359)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-13 09:47:51 +08:00
Duo Zhang fc5a8fd3bf Revert "HBASE-25891 Remove dependence on storing WAL filenames for backup (#3359)"
This reverts commit df57b1ca6b.
2021-09-13 09:47:25 +08:00
Mallikarjun df57b1ca6b
HBASE-25891 Remove dependence on storing WAL filenames for backup (#3359)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-13 09:46:43 +08:00
Bryan Beaudreault a15e94a47f HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
Closes #3630

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org
2021-09-01 18:01:13 -04:00
Duo Zhang 63d4970de4
HBASE-26150 Let region server also carry ClientMetaService (#3550)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-04 23:44:10 +08:00
GeorryHuang 8ae394285a
HBASE-26091 Remove FirstKeyValueMatchingQualifiersFilter (#3497)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-07-25 21:12:20 +08:00
Duo Zhang 16721239e7
HBASE-26100 Set version as 3.0.0-alpha-2-SNAPSHOT in master (#3508)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-20 23:04:08 +08:00
Duo Zhang 5118321ec9
HBASE-26059 Set version as 3.0.0-alpha-1 in master in prep for first RC of 3.0.0-alpha-1 (#3453)
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-07-02 07:50:41 +08:00
Duo Zhang 51893b9ba3
HBASE-26029 It is not reliable to use nodeDeleted event to track region server's death (#3430)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-06-30 08:44:19 +08:00
Toshihiro Suzuki ec31818574
HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update (#3384) 2021-06-15 07:43:27 +09:00
xijiawen 426c3c16f3
HBASE-25799 add clusterReadRequests and clusterWriteRequests jmx (#3188)
* HBASE-25799 add clusterReadRequests and clusterWriteRequests jmx

Co-authored-by: stevenxi <stevenxi@tencent.com>
2021-06-03 15:48:03 +08:00
Andrew Purtell 8ec6fd9459
HBASE-25869 WAL value compression (#3244)
WAL storage can be expensive, especially if the cell values
represented in the edits are large, consisting of blobs or
significant lengths of text. Such WALs might need to be kept around
for a fairly long time to satisfy replication constraints on a space
limited (or space-contended) filesystem.

We have a custom dictionary compression scheme for cell metadata that
is engaged when WAL compression is enabled in site configuration.
This is fine for that application, where we can expect the universe
of values and their lengths in the custom dictionaries to be
constrained. For arbitrary cell values it is better to use one of the
available compression codecs, which are suitable for arbitrary albeit
compressible data.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-21 11:05:52 -07:00
GeorryHuang 00fec24c90
HBASE-25790 NamedQueue 'BalancerRejection' for recent history of balancer skipping (#3182)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-02 21:30:48 +05:30
Duo Zhang 2420286715 HBASE-25401 Add trace support for async call in rpc client (#2790)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2021-04-25 09:23:23 +08:00
Duo Zhang 302d9ea8b8 HBASE-25373 Remove HTrace completely in code base and try to make use of OpenTelemetry
Signed-off-by: stack <stack@apache.org>
2021-04-25 09:23:23 +08:00
Toshihiro Suzuki 46f7d9dd4b
HBASE-25703 Support conditional update in MultiRowMutationEndpoint (#3098)
Signed-off-by: Michael Stack <stack@apache.org>
2021-03-30 09:18:56 +09:00
Toshihiro Suzuki 93b1163a8b
HBASE-25702 Remove RowProcessor (#3097)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-03-28 07:38:42 +09:00
Peter Somogyi 0cc1ae48ed
HBASE-25587 [hbck2] Schedule SCP for all unknown servers (#2978)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-03-11 17:20:36 +01:00
Wellington Ramos Chevreuil 373dc7788d
HBASE-25548 Optionally allow snapshots to preserve cluster's max file… (#2923)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-03-10 10:20:10 +00:00
Mate Szalay-Beko 6a5c928539
HBASE-25181 Add options for disabling column family encryption and choosing hash algorithm for wrapped encryption keys.
Prior to this patch hbase always used the MD5 hash algorithm to store a hash for encryption keys.
This hash is needed to verify the secret key of the subject. (e.g. making
sure that the same secrey key is used during encrypted HFile read and write).
The MD5 algorithm is considered weak, and can not be used in some
(e.g. FIPS compliant) clusters.

In this patch we:
- add a config parameter to globally enable/disable column family encryption (def enabled)
- introduce a backward compatible way of specifying the hash algorithm.
  This enable us to use newer and more secure hash algorithms like SHA-384
  or SHA-512 (which are FIPS compliant).
- add a config parameter to fail if an hfile is encountered that uses a
  different hash algorithm than the one currently configured to ease validation after
  migrating key hash algorithms (def disabled)

Closes #2539

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
2020-11-09 13:53:50 -06:00
Michael Stack 23e656712b HBASE-25238 Upgrading HBase from 2.2.0 to 2.3.x fails because of “Message missing required fields: state” (#2625)
Make protobuf fields add post-2.0.0 release marked 'required' instead
be 'optional' so migrations from 2.0.x to 2.1+ or 2.2+ succeeds.

Signed-off-by: Viraj Jasani vjasani@apache.org
2020-11-05 08:49:47 -08:00
Duo Zhang 0d63318f10
HBASE-25124 Support changing region replica count without disabling table (#2497)
Signed-off-by: stack <stack@apache.org>
2020-10-08 21:35:29 +08:00
Viraj Jasani 23ce91819a
HBASE-25147 : Serialize regionNames in ReopenTableRegionsProcedure
Closes #2494

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-10-05 15:49:26 +05:30
bsglz 5d926627ae
HBASE-24967 The table.jsp cost long time to load if the table include… (#2326)
* HBASE-24967 The table.jsp cost long time to load if the table include closed regions

* fix it  by another way

* fix review issue

* fix checkstyle warnings

* fix checkstyle warning
2020-09-29 15:36:03 +08:00
Viraj Jasani 80ffac29c1
HBASE-25070 : With new generic API getLogEntries, cleaning up unused RPC APIs
Closes #2426

Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-28 12:54:35 +05:30
Viraj Jasani 35d59dc570
HBASE-24528 : BalancerDecision queue implementation in HMaster with Admin API
* Admin API getLogEntries() for ring buffer use-cases: so far, provides balancerDecision and slowLogResponse
* Refactor RPC call for similar use-cases
* Single RPC API getLogEntries() for both Master.proto and Admin.proto

Closes #2261

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-09-16 20:56:54 +05:30
bsglz c5ca191921
HBASE-24875 Remove the force param for unassign since it dose not take effect any more (#2254)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-09-15 22:05:03 -05:00
Bharath Vissapragada 01cf60067c
HBASE-24765: Dynamic master discovery (#2130)
This patch adds the ability to discover newly added masters
dynamically on the master registry side. The trigger for the
re-fetch is either periodic (5 mins) or any registry RPC failure.
Master server information is cached in masters to avoid repeated
ZK lookups.

Updates the client side connection metrics to maintain a counter
per RPC type so that clients have visibility into counts grouped
by RPC method name.

I didn't add the method to ZK registry interface since there
is a design discussion going on in splittable meta doc. We can
add it later if needed.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-08-25 15:09:03 -07:00
Nick Dimiduk ecc0c67077 HBASE-24627 Normalize one table at a time
Introduce an additional method to our Admin interface that allow an
operator to selectivly run the normalizer. The IPC protocol supports
general table name select via compound filter.

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-18 09:46:29 -07:00
Viraj Jasani f35c5eaadd
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses) (#2109)
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-21 20:24:13 +05:30
Viraj Jasani f2c087caeb
Revert "HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses)"
This reverts commit 16f306b4a1.

TestAdminShell and TestThriftHBaseServiceHandler are failing consistently
2020-07-21 13:50:27 +05:30
Viraj Jasani 16f306b4a1
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses)
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-17 19:16:22 +05:30
bsglz 724f0478ed
HBASE-24404 Support flush a single column family of region (#2032)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-10 18:13:50 +01:00
Zheng Wang b2321b3583
HBASE-24586 Add table level locality in table.jsp
Closes #1926

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-10 21:59:59 +05:30
XinSun 4b89ee24f3
HBASE-24431 RSGroupInfo add configuration map to store something extra (#1782)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-24 18:46:09 +08:00
bsglz b83d38fb94
HBASE-24038 Add a metric to show the locality of ssd in table.jsp (#1337)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-14 22:46:18 +05:30
Toshihiro Suzuki efd5a962e5 HBASE-8458 Support for batch version of checkAndMutate()
Closes #1648

Signed-off-by: Josh Elser <elserj@apache.org>
2020-06-10 22:34:36 -04:00
Duo Zhang b2ec4c1ea0
HBASE-24505 Reimplement Hbck.setRegionStateInMeta (#1852)
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2020-06-05 08:48:39 +08:00
Wellington Ramos Chevreuil e5345b3a7c
HBASE-21406 "status 'replication'" should not show SINK if the cluste… (#1761)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-06-02 09:27:16 +01:00
Duo Zhang 36a6ef9cf9
HBASE-24471 The way we bootstrap meta table is confusing (#1806)
Signed-off-by: Michael Stack <stack@apache.org>
2020-05-30 16:03:52 +08:00
Viraj Jasani f0f9a20cd7
HBASE-23941 : FilterBy operator support in get_slowlog_responses API (#1793)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-05-30 12:59:59 +05:30
Nick Dimiduk e37aafcfc2 HBASE-24260 Add a ClusterManager that issues commands via coprocessor
Implements `ClusterManager` that relies on the new
`ShellExecEndpointCoprocessor` for remote shell command execution.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-04 10:53:02 -07:00