Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,408 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

8844 Commits

Author SHA1 Message Date
Bryan Beaudreault 206100f515 HBASE-26814 Default StoreHotnessProtector to off, with logs to guide when to turn it on (#4194) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-09 14:44:29 -08:00
Bryan Beaudreault 8251e8d2c9 HBASE-26783 ScannerCallable doubly clears meta cache on retries (#4147) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-09 12:03:12 -08:00
Bryan Beaudreault 6facc82585 HBASE-26784 Use HIGH_QOS for ResultScanner.close requests (#4163) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-08 11:42:12 -08:00
Nick Dimiduk 84eb868f98 HBASE-26764 Implement generic exception support for TraceUtil methods over Callables and Runnables 
For the `TraceUtil` methods that accept `Callable` and `Runnable` types, make them generic over a
child of `Throwable`. This allows us to consolidate the two method signatures into a single more
flexible definition.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-07 13:57:22 +01:00
Rushabh Shah 9ddc90e258 HBASE-26792 Implement ScanInfo#toString (#4154) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-05 17:49:26 +08:00
Nick Dimiduk 09bd726d46 HBASE-26759 Fix trace continuity through CallRunner (#4126) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-04 16:03:02 -08:00
Andrew Purtell 6c70e4b4c5 [branch-2.5] Amend HBASE-26731 Add metrics for active and expired scanners 
Compliation fixes

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-04 14:11:33 -08:00
Bryan Beaudreault 0f28fa8433 HBASE-26731 Add metrics for active and expired scanners (#4145) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java
 2022-03-04 14:07:16 -08:00
Istvan Toth d4060aeda7 HBASE-26777 BufferedDataBlockEncoder$OffheapDecodedExtendedCell.deepC… (#4139) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-04 13:53:29 -08:00
Duo Zhang 71ddf74dda HBASE-26691 Replacing log4j with reload4j for branch-2.x (#4050) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-04 12:08:36 -08:00
Mike Drob d971a9d298 HBASE-26622 Update error-prone to 2.10 
Author:    Mike Drob <mdrob@apache.org>
Co-authored-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-03 11:52:00 +01:00
Niels Basjes 4858b2543c HBASE-26762 Un-Deprecate and improve documentation for Scan#setRowPrefixFilter (#4119) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-02 23:34:02 +08:00
Nick Dimiduk ff7f18923a HBASE-26776 RpcServer failure to SASL handshake always logs user "unknown" to audit log (#4138) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2022-03-02 15:52:22 +01:00
Bryan Beaudreault 5f11bc4e50 HBASE-26703 Allow configuration of IPC queue balancer (#4063) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java
 2022-02-18 17:55:06 -08:00
Bryan Beaudreault 1fb4ee37be HBASE-26727 Fix CallDroppedException reporting (#4088) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-02-18 17:27:59 -08:00
Richard Marscher 2be2072704 HBASE-26576 Allow pluggable queue to be used with the fast path executor or normal balanced executor (#3944) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-02-18 10:26:35 -08:00
Yutong Xiao 932bf591b3 HBASE-26659 The ByteBuffer of metadata in RAMQueueEntry in BucketCache could be reused. (#4026) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-02-19 00:42:04 +08:00
chenglei af338d2c99
HBASE-26712 Balancer encounters NPE in rare case (#4112) (#4092) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-02-16 12:15:12 +05:30
Xiaolin Ha 42bb0afaf8 HBASE-26742 Comparator of NOT_EQUAL NULL is invalid for checkAndMutate (#4105) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-02-16 11:23:48 +08:00
Andrew Purtell a19dcc3be0 HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks' (#4094) 
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-02-15 18:39:03 -08:00
Xiaolin Ha 40882b5681 HBASE-26434 Do compact when all L0 files are expired (#3830) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-02-15 22:39:14 +08:00
Yutong Xiao 3803b8be2c HBASE-26688 Threads shared EMPTY_RESULT may lead to unexpected client job down (#4073) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-02-15 20:31:19 +08:00
Nick Dimiduk 600a6a8faf HBASE-26521 Name RPC spans as `$package.$service/$method` (#4024) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-02-10 12:58:01 +01:00
Duo Zhang b64a507340 HBASE-26709 Ban the usage of junit 3 TestCase (#4065) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-02-10 01:06:21 +08:00
Duo Zhang 12002e6a79 HBASE-26724 Backport the UT changes in HBASE-24510 to branch-2.x (#4081) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-02-10 00:53:14 +08:00
Andrew Purtell d091751ab0 HBASE-26726 Allow disable of region warmup before graceful move (#4086) 
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
 2022-02-01 15:39:56 -08:00
Bryan Beaudreault 59d60fd421 HBASE-26713 Default to LATEST_TIMESTAMP if no timestamp sent along on Increment/Append (#4075) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-01-28 12:57:59 -08:00
Duo Zhang 9f1f705686 HBASE-26523 Upgrade hbase-thirdparty dependency to 4.0.1 (#3988) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2022-01-27 17:33:37 -08:00
Nick Dimiduk 4d86d3be08 HBASE-26614 Refactor code related to "dump"ing ZK nodes (#3969) 
The code starting at `ZKUtil.dump(ZKWatcher)` is a small mess – it has cyclic dependencies woven
through itself, `ZKWatcher` and `RecoverableZooKeeper`. It also initializes a static variable in
`ZKUtil` through the factory for `RecoverableZooKeeper` instances. Let's decouple and clean it
up.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2022-01-26 09:00:52 -08:00
Peter Somogyi 2c82cf3207 HBASE-26689 Backport HBASE-24443 Refactor TestCustomSaslAuthenticationProvider (#4049) 
Duo Zhang <zhangduo@apache.org>
 2022-01-20 15:05:45 +01:00
Nick Dimiduk 0ee15e0865 HBASE-26474 Implement connection-level attributes (#4014) 
Add support for `db.system`, `db.connection_string`, `db.user`.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
Co-authored-by: Josh Elser <josh.elser@gmail.com>
 2022-01-18 16:09:08 -08:00
Yutong Xiao 540fbe51b0
HBASE-26551 Add FastPath feature to HBase RWQueueRpcExecutor (#4036) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2022-01-18 11:13:13 +08:00
xianjingfeng b840ed68f5 HBASE-26671 Misspellings of hbck usage (#4033) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-16 16:34:34 +08:00
Nick Dimiduk 9ef7f5977e HBASE-26616 Refactor code related to ZooKeeper authentication (#3973) 
This refactor reduces the size and scope of the `ZKUtil` class. The core of this refactor is
moving the `login*` methods from `ZKUtil` into their own class, `ZKAuthentication`. The class
`JaasConfiguration` is also moved along with them.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-10 16:08:17 -08:00
Xiaolin Ha 6a060adc8b HBASE-26397 Display the excluded datanodes on regionserver UI (#3990) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-10 13:03:49 +08:00
Duo Zhang 2abd09f6ac HBASE-26643 LoadBalancer should not return empty map (#4005) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2022-01-08 00:39:59 +08:00
Xiaolin Ha 76bc412fc4 HBASE-26347 Support detect and exclude slow DNs in fan-out of WAL (#3800) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-06 11:48:50 +08:00
Xiaolin Ha 2b05e68ae1 HBASE-26578 CompactedHFilesDischarger should be shutdown when aborting regionserver (#4001) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-05 16:33:59 +08:00
chenglei e50f4991da HBASE-26567 Remove IndexType from ChunkCreator (#3947) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-05 00:08:29 +08:00
Duo Zhang deec701475 HBASE-26641 Split TestMasterFailoverWithProcedures (#3994) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2022-01-03 00:41:36 +08:00
chenglei efeec919de HBASE-26494 Using RefCnt to fix the flawed MemStoreLABImpl (#3983) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-01 20:57:36 +08:00
Richard Marscher 0001ad6b8b HBASE-26623 Report CallDroppedException in exception metrics (#3980) 
`CallDroppedException` can be thrown with `CallRunner.drop()` by queue implementations that decide to drop calls to groom the RPC call backlog. The LifoCoDel queue does this I believe and with Pluggable queue it's possible for 3rd party queue implementations to be using `drop()` for similar reasons. It would be nice for the server to be tracking these exceptions in metrics since otherwise you might have to do some extra lifting on the client side.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
 2021-12-30 00:05:12 +08:00
Yutong Xiao be1d3d856f HBASE-26564 Retire the method visitLogEntryBeforeWrite without RegionInfo in WALActionListner (#3985) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-12-28 21:55:56 +08:00
meiyi afabefb051 HBASE-26615 Snapshot referenced data files are deleted when delete a table with merge regions (#3976) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2021-12-25 22:32:15 -08:00
Duo Zhang 51d38b5e4f HBASE-26609 Round the size to MB or KB at the end of calculation in HRegionServer.createRegionLoad (#3967) 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-12-23 11:58:35 +08:00
Samir Khan a57f4976c1 HBASE-26598 Fix excessive connections in MajorCompactor (#3961) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-12-23 11:55:31 +08:00
zhengzhuobinzzb e28ccf092b HBASE-26580 The message of StoreTooBusy is confused (#3949) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
 2021-12-18 23:34:08 +08:00
zhengzhuobinzzb ef3854225a HBASE-26579 Set storage policy of recovered edits when hbase.wal.storage.type is configured (#3948) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-12-18 23:27:54 +08:00
chenglei 482ef6904e HBASE-26488 Memory leak when MemStore retry flushing (#3899) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-12-18 12:22:52 +08:00
Norbert Kalmar d072a2e694 HBASE-26340 TableSplit returns false size under 1MB (#3872) 
Signed-off-by: Peter Somogyi <psomogyi@apache.org
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-12-18 12:21:51 +08:00
Nick Dimiduk 37397637ad HBASE-26542 Apply a `package` to test protobuf files (addendum) 
RpcServer identifies the services it hosts by unqualified service name. Thus, use `getName()`
instead of `getFullName()`. See also HBASE-26589.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-12-16 16:39:48 -08:00
Josh Elser 2bfb743d79 HBASE-26550 Make sure the master is running normally before accepting a balance command (#3928) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-12-14 23:32:34 +08:00
bsglz 49d4c4a4c0 HBASE-26027 The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException (#3925) 
* HBASE-26027 The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException
 2021-12-13 16:07:42 -08:00
Bryan Beaudreault 9cf0af6e79 HBASE-26537: Make HBASE-15676 backwards compatible, using a flag on the proto (#3931) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-12-13 13:34:10 -08:00
Guanghao Zhang 140b5d8d26 HBASE-24870 Ignore TestAsyncTableRSCrashPublish (#2248) 
Signed-off-by: meiyi <myimeiyi@gmail.com>
 2021-12-06 19:07:07 -08:00
LiangJun He ec52519f7c HBASE-26462 Should persist restoreAcl flag in the procedure state for CloneSnapshotProcedure and RestoreSnapshotProcedure (#3921) 
Signed-off-by: Yu Li <liyu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-12-06 23:16:15 +08:00
RamanChodzka 074456e190 HBASE-26533 KeyValueScanner might not be properly closed when using InternalScan.checkOnlyMemStore() (#3917) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-12-05 22:35:07 +08:00
Tomu Tsuruhara 19d991d291 HBASE-26517 Add auth method information to AccessChecker audit log (#3897) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-12-04 23:14:42 +08:00
Bryan Beaudreault d504504875
HBASE-26304 Reflect out of band locality improvements in metrics and balancer (#3895) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-12-04 12:23:09 +08:00
Nick Dimiduk d1c68a2d5c HBASE-26471 Move tracing semantic attributes to their own class (#3896) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
 2021-12-01 13:24:08 -08:00
Rushabh Shah 6fc5bbc19a
HBASE-26468 Region Server doesn't exit cleanly incase it crashes. (#3865) (#3862) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-12-01 11:06:07 +05:30
Duo Zhang 774484ed44 HBASE-25905 Shutdown of WAL stuck at waitForSafePoint (#3898) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-12-01 08:34:08 +08:00
Duo Zhang 799217ee69 HBASE-26455 TestStochasticLoadBalancerRegionReplicaWithRacks fails consistently (#3879) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2021-11-28 22:23:21 +08:00
chenglei 5437532910 HBASE-26476 Make DefaultMemStore extensible for HStore.memstore (#3869) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-27 12:06:35 +08:00
chenglei 5101f372e4 HBASE-26465 MemStoreLAB may be released early when its SegmentScanner is scanning (#3859) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Anoop Sam John <anoopsamjohn@apache.org>
 2021-11-25 21:35:55 +08:00
Yutong Xiao 6aa2daa2c2 HBASE-26477 Fix MiniMapReduceCluster failure under Java17 (#3878) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-25 16:27:23 +08:00
Duo Zhang fa34cfc7d1 Revert "Backport HBASE-26477 Fix MiniMapReduceCluster failure under Java17 to branch-2 (#3878)" 
This reverts commit ccf150434d.
 2021-11-25 16:27:11 +08:00
Yutong Xiao ccf150434d
Backport HBASE-26477 Fix MiniMapReduceCluster failure under Java17 to branch-2 (#3878) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-25 16:25:44 +08:00
liangxs 2798ef132b HBASE-26479 Print too slow/big scan's operation_id in region server log (#3870) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-25 11:43:01 +08:00
Andrew Purtell 52ce87780a Revert "HBASE-26479 Print too slow/big scan's operation_id in region server log (#3870)" 
This reverts commit b437a96272.
 2021-11-24 17:34:01 -05:00
liangxs b437a96272 HBASE-26479 Print too slow/big scan's operation_id in region server log (#3870) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-24 23:38:16 +08:00
Duo Zhang 0a51d2b5be HBASE-26475 The flush and compact methods in HTU should skip processing secondary replicas (#3868) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-11-23 18:51:25 +08:00
Xiaolin Ha f8d03d1227 HBASE-26249 Ameliorate compaction made by bulk-loading files (#3831) (#3874) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-23 15:52:47 +08:00
haxiaolin 61406252a5 Revert "Backport HBASE-26249 Ameliorate compaction made by bulk-loading files (#3831) (#3874)" 
This reverts commit 5843a9a44c.
 2021-11-23 15:52:34 +08:00
Xiaolin Ha 5843a9a44c
Backport HBASE-26249 Ameliorate compaction made by bulk-loading files (#3831) (#3874) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-23 15:45:21 +08:00
zhengzhuobinzzb 4312f817f3 HBASE-26467 Fix bug for MemStoreLABImpl.forceCopyOfBigCellInto(Cell) (#3858) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: chenglei <chenglei@apache.org>
 2021-11-21 21:02:18 +08:00
ss77892 e549384437 HBASE-26463 Unreadable table names after HBASE-24605 (#3853) 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-21 20:49:13 +08:00
Xiaolin Ha 8d3562c6fa
HBASE-26421 Use HFileLink file to replace entire file's reference when splitting (#3854) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-18 21:37:10 +08:00
Josh Elser 4a8da47f74 HBASE-26267 Don't try to recover WALs from a WAL dir which doesn't exist (#3679) 
We currently cause an error to be thrown by trying to list a
non-existent directory. We see that the master region directory exists
on the filesystem, but forget to make sure that the master region's WAL
directory also exists before we try to list it.
 2021-11-16 16:27:45 -05:00
Andrew Purtell e65f28cc57 HBASE-26444 BucketCacheWriter should log only the BucketAllocatorException message, not the full stack trace (#3840) 
Signed-off-by: Anoop <anoopsamjohn@apache.org>
 2021-11-15 16:13:36 -08:00
Andrew Purtell 23024edba2 HBASE-26443 Some BaseLoadBalancer log lines should be at DEBUG level (#3838) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-11-15 16:06:59 -08:00
chenglei 98a4fb8086 HBASE-26438 Fix flaky test TestHStore.testCompactingMemStoreCellExceedInmemoryFlushSize (#3834) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-12 23:03:37 +08:00
Zhuoyue Huang ad36941932 HBASE-26450 Server configuration will overwrite HStore configuration after using shell command 'update_config' (#3843) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
 2021-11-12 11:43:48 +08:00
Yutong Xiao 9db7630d23 HBASE-26410 Fix HBase TestCanaryTool for Java17 (#3809) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-09 22:21:01 +08:00
clarax cc2e4aec87
HBASE-26337 Optimization for weighted random generators (#3828) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-09 21:27:44 +08:00
Andrew Purtell 1302073dfe HBASE-26430 Increase log level in DefaultHeapMemoryTuner from DEBUG to INFO (#3820) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-11-08 18:35:39 -08:00
Andrew Purtell 201b3c7fc7 HBASE-26429 HeapMemoryManager fails memstore flushes with NPE if enabled (#3819) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-11-08 18:32:17 -08:00
Xiaolin Ha 2fbb375cd5 HBASE-25322 Redundant Reference file in bottom region of split (#3814) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-08 14:48:55 +08:00
haxiaolin ebd89b58eb Revert "HBASE-25322 Redundant Reference file in bottom region of split (#3814)" 
This reverts commit a6edf44175.
 2021-11-08 11:40:43 +08:00
Xiaolin Ha a6edf44175 HBASE-25322 Redundant Reference file in bottom region of split (#3814) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-05 18:17:47 +08:00
clarax e06208f3d6
HBASE-26311 Balancer gets stuck in cohosted replica distribution (#3804) 
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-11-03 11:58:38 -07:00
clarax 9eb3706021
HBASE-26309 Balancer tends to move regions to the server at the end of list (#3811) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-02 21:58:50 +08:00
XinSun 063e0e2e92 HBASE-26406 Can not add peer replicating to non-HBase (#3806) 
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>

(cherry picked from commit b9b7fec57f)
 2021-11-02 15:03:33 +08:00
chenglei e89f62e611
HBASE-26384 Segment already flushed to hfile may still be remained in CompactingMemStore (#3779) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-11-01 23:05:57 +08:00
Andrew Purtell f2c58fcf68 HBASE-26353 Support loadable dictionaries in hbase-compression-zstd (#3787) 
ZStandard supports initialization of compressors and decompressors with a
precomputed dictionary, which can dramatically improve and speed up compression
of tables with small values. For more details, please see

  The Case For Small Data Compression
  https://github.com/facebook/zstd#the-case-for-small-data-compression

Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdCodec.java
 2021-10-27 07:57:16 -07:00
chenglei b4177577ac HBASE-26394 Cache in RSRpcServices.executeProcedures does not take effect (#3794) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-10-26 15:44:17 +08:00
clarax 32eb23c8f3
HBASE-26327 Replicas cohosted on a rack shouldn't keep triggering balancer (#3788) 
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-10-25 15:44:11 -07:00
Anoop Sam John 444dec9289 HBASE-26190 High rate logging of BucketAllocatorException: Allocation too big - Addendum (#3792) 2021-10-24 09:22:26 +05:30
clarax f299d56934
HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger (#3783) 2021-10-22 09:46:58 -07:00
Sourabh Dhanotia eea2ea8576
HBASE-26368 Fix checkstyle issues for HRegionServer for branch-2 (#3781) (#3764) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-10-22 12:51:29 +05:30
binlijin 0eaadd92d8 HBASE-26385 Clear CellScanner when replay (#3773) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2021-10-21 21:56:39 +08:00
Andrew Purtell dce45ec16a HBASE-26383 HBCK incorrectly reports inconsistencies for recently split regions following a master failover (#3775) 
A regression was introduced by HBASE-25847 which changed regionInfo#isParentSplit
to regionState#isSplit. The region state after restart is CLOSED instead of SPLIT.
We need to check both regionState and regionInfo for split status.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-10-20 15:55:07 -07:00
Viraj Jasani 5c4a053f2a
HBASE-26371 Prioritize meta region move over other region moves in region_mover (#3771) (#3767) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-10-20 23:06:25 +05:30
Andrew Purtell b6bb18022f HBASE-26316 Per-table or per-CF compression codec setting overrides (#3730) 
We get and retain Compressor instances in HFileBlockDefaultEncodingContext,
and could in theory call Compressor#reinit when setting up the context,
to update compression parameters like level and buffer size, but we do
not plumb through the CompoundConfiguration from the Store into the
encoding context. As a consequence we can only update codec parameters
globally in system site conf files.

Fine grained configurability is important for algorithms like ZStandard
(ZSTD), which offers more than 20 compression levels, where at level 1
it is almost as fast as LZ4, and where at higher levels it utilizes
computationally expensive techniques to rival LZMA at compression ratio
but trades off significantly for reduced compresson throughput. The ZSTD
level that should be set for a given column family or table will vary by
use case.

Signed-off-by: Viraj Jasani <vjasani@apache.org>

Conflicts:
	hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdDecompressor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/io/compress/HFileTestBase.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestDataBlockEncoders.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestSeekToBlockWithEncoders.java
 2021-10-19 13:59:04 -07:00
KulwantSingh011 7e574c9f1d
HBASE-26190 High rate logging of BucketAllocatorException: Allocation too big (#3752) 
Reduce the frequency of allocation failed traces by printing them
preiodically (once per minute). Record the allocation failures in the
Bucket Cache Stats and let the stat thread dump cumulative allocation
failures alongside other traces it dumps.

Also, this change adds trace for the Table name, Column Family and
HFileName for the most recent allocation failure in last 1 minute.

Signed-off-by: Anoop <anoopsamjohn@apache.org>
 2021-10-19 13:21:11 +05:30
Josh Elser 45b09aa4d7 HBASE-26350 Add a DEBUG when we fail the SASL handshake 
Closes #3743

Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-10-14 15:27:32 -04:00
Tak Lon (Stephen) Wu 3ce9052188
HBASE-24833: Bootstrap should not delete the META table directory if … (#2237) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
 2021-10-12 10:56:00 -07:00
Duo Zhang 10584d70d2
HBASE-26306 Backport "HBASE-26220 Use P2P communicate between region servers to sync the list for bootstrap node" to branch-2 (#3727) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-10-10 22:56:12 +08:00
Andrew Purtell 5af02201d2 HBASE-26324 Reuse compressors and decompressors in WAL CompressionContext (#3728) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-10-07 09:46:38 -07:00
Andrew Purtell 18b9fa8a3c HBASE-26259 Fallback support to pure Java compression (#3691) 
This change introduces provided compression codecs to HBase as
new Maven modules. Each module provides compression codec support
that formerly required Hadoop native codecs, which in turn relies
on native code integration, which may or may not be available on
a given hardware platform or in an operational environment. We
now provide codecs in the HBase distribution for users whom for
whatever reason cannot or do not wish to deploy the Hadoop native
codecs.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileEncryption.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestCompressedWAL.java
 2021-10-06 13:48:33 -07:00
Duo Zhang 706082d513
HBASE-26294 Backport "HBASE-26181 Region server and master could use itself as ConnectionRegistry" to branch-2 (#3708) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-10-05 22:11:01 +08:00
imbajin 9a33e234e7 HBASE-24601: Change default Hfile storage policy from HOT to NONE for HDFS (#1940) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-10-02 16:37:39 +08:00
Yutong Xiao a2c09ae5bc HBASE-26305 Move NavigableSet add operation to writer thread in BucketCache (#3705) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: chenglei <chenglei@apache.org>
 2021-10-01 21:06:57 +08:00
chenglei 0748165ab0 HBASE-26295 BucketCache could not free BucketEntry which restored fro… (#3699) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-29 21:46:57 +08:00
GeorryHuang 08800e19d6
HBASE-26251 StochasticLoadBalancer metrics should update even if balancer doesn't run (#3706) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
 2021-09-29 21:25:39 +08:00
richardantal 69749ed01e HBASE-26289 Hbase scan setMaxResultsPerColumnFamily not giving right results (#3688) 
Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-09-29 10:22:24 +02:00
Duo Zhang 08a2efde16 HBASE-26293 Use reservoir sampling when selecting bootstrap nodes (#3702) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-09-28 23:41:28 +08:00
bitterfox 786e09a936 HBASE-26238 Short message by Result#compareResults for VerifyReplication to avoid OOME (#3647) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-28 23:16:20 +08:00
clarax 73f42b27bf HBASE-26297 Balancer run is improperly triggered by accuracy error of double comparison (#3698) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-26 23:35:50 +08:00
clarax 404789adde HBASE-26178 Improve data structure and algorithm for BalanceClusterState to improve computation speed for large cluster (#3682) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-25 21:48:51 +08:00
Andrew Purtell 59d133ef95 Amend HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job 
Compliation fixes for CombinedBlockCache and TestClientSideRegionScanner.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-09-22 12:27:59 -07:00
Tak Lon (Stephen) Wu e0325c40f6 HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job (#3684) 
Introduce `hfile.onheap.block.cache.fixed.size`
and default to disable. when using ClientSideRegionScanner
it will be enabled with a fixed size for caching
INDEX/LEAF_INDEX block when a client, e.g.
snapshot scanner, scans the entire HFile
and does not need to seek/reseek to index
block multiple times.

Signed-off-by: Josh Elser <elserj@apache.org>
 2021-09-22 09:38:20 -07:00
meiyi 84b5511d13 HBASE-26261 Store configuration loss when use update_config (#3664) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-22 18:35:28 +08:00
meiyi 8f6c5e1f02 HBASE-26270 Provide getConfiguration method for Region and Store interface (#3671) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
 2021-09-22 11:32:32 +08:00
Duo Zhang e327fa43c0 HBASE-26288 Revisit the usage of MetaTableLocator when HRegionServer.TEST_SKIP_REPORTING_TRANSITION is true (#3689) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-09-22 00:27:11 +08:00
Duo Zhang ef85204522 HBASE-26285 Remove MetaTableLocator usages in non-migration code (#3686) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-09-21 23:32:13 +08:00
chenglei b21b250570 HBASE-26281 DBB got from BucketCache would be freed unexpectedly before RPC completed (#3680) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-17 22:28:56 +08:00
Xiaolin Ha f670fc5d4d HBASE-26229 Limit count and size of L0 files compaction in StripeCompactionPolicy (#3646) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-09 18:28:45 +08:00
hapihu 68db7f5ed0 [HBASE-26243]Fix typo for file "hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java" (#3650) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-09 12:21:54 +08:00
Rushabh Shah 820ba21b2a HBASE-26106 AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory. (#3636) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
 2021-09-02 10:26:22 -07:00
XinSun ccd90269d2 HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky (#3651) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 345d7256c8)
 2021-09-02 10:33:41 +08:00
Bryan Beaudreault e07928df4c HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>

(Re-application of HBASE-26147 with the correct author metadata)
 2021-09-01 21:58:30 -04:00
Josh Elser 1e74f08403 Revert "HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed" 
This reverts commit 5b58d11c91.

I accidentally applied this as myself instead of as Bryan. Reverting it
to apply the patch properly.
 2021-09-01 21:57:46 -04:00
Josh Elser 5b58d11c91 HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org
 2021-09-01 21:47:53 -04:00
Tak Lon (Stephen) Wu c639d6f353 HBASE-26136 Backport HBASE-25723 "Temporarily remove the trace support for RegionScanner.next" to branch-2 (#3623) 
13/17 commits of HBASE-22120, original commit 7f90c2201f

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu d0a53e3f29 HBASE-26133 Backport HBASE-25591 "Upgrade opentelemetry to 0.17.1" to branch-2 (#3608) 
10/17 commits of HBASE-22120, original commit f6ff519dd0

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 8b7450f5d8 HBASE-26132 Backport HBASE-25535 "Set span kind to CLIENT in AbstractRpcClient" to branch-2 (#3607) 
9/17 commits of HBASE-22120, original commit bb8c4967f8

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 9cce94a2d0 HBASE-26131 Backport HBASE-25484 "Add trace support for WAL sync" to branch-2 (#3597) 
8/17 commits of HBASE-22120, original commit 2be2c63f0d

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by:  Peter Somogyi <psomogyi@apache.org>
 2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 321dcd4e77 HBASE-26130 Backport HBASE-25455 "Add trace support for HRegion read/… (#3594) 
7/17 commits of HBASE-22120, original commit 03e12bfa4a

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by:  Duo Zhang <zhangduo@apache.org>
 2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 83a27389a0 HBASE-26129 Backport HBASE-25481 "Add host and port attribute when tracing rpc call at client side" to branch-2 (#3589) 
6/17 commits of HBASE-22120, original commit ae2c62ffaa

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by:  Peter Somogyi  <psomogyi@apache.org>
 2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 58b5c007b9 HBASE-26127 Backport HBASE-23898 "Add trace support for simple apis i… (#3556) 
4/17 commits of HBASE-22120, original commit 805b2ae2ad

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 4dfc59c19e HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 (#3543) 
2/17 commits of HBASE-22120

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 665305cc3b HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529) 
1/17 commits of HBASE-22120

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-09-01 15:29:09 -07:00
chenglei dbf43dcfbd HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore (#3604) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-01 18:17:13 +08:00
Andrew Purtell 8be2ad2d42 HBASE-25642 Fix or stop warning about already cached block (#3638) 
Our logs have as a fairly common occurrence: 2021-03-05 22:24:31,034 WARN
[StoreFileOpener-foo-1] hfile.BlockCacheUtil: Caching an already cached
block: blah.bub. This is harmless and can happen in rare cases (see HBASE-8547)

Because it is harmless, log at DEBUG level, not WARN.

Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
 2021-08-31 09:26:10 -07:00
Duo Zhang c08e877d6a HBASE-26235 We could start RegionServerTracker before becoming active master (#3645) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-08-30 12:22:41 +08:00
keyserbrian1 1e49e94051 HBASE-26232 SEEK_NEXT_USING_HINT is ignored on reversed Scans (#3639) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-29 21:42:48 +08:00
GeorryHuang 033ff757fd HBASE-26216 Move HRegionServer.abort(String) to Abortable as a default method (#3642) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-28 23:06:54 +08:00
Andrew Purtell 3b1482ef60 HBASE-26189 Reduce log level of CompactionProgress notice to DEBUG (#3579) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-08-27 10:15:28 -07:00
Pankaj 30bb72bdb8 HBASE-26227 Forward port HBASE-26223 test code to branch-2.4+ (#3632) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-27 16:46:54 +08:00
lpalasthy cd9e45ce8d HBASE-26219 Negative time is logged while waiting on regionservers (#3622) 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-25 16:25:24 +02:00
Duo Zhang 931a7a5a61 HBASE-26180 Introduce a initial refresh interval for RpcConnectionRegistry (#3601) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2021-08-25 18:51:40 +08:00
GeorryHuang dacd3404a4 HBASE-26215 The backup master status page should use ActiveMasterManager instead of MasterAddressTracker (#3621) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-25 16:42:45 +08:00
Duo Zhang 5dad395869 HBASE-26173 Return only a sub set of region servers as bootstrap nodes (#3599) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-24 23:17:56 +08:00
Duo Zhang 6ad7eb88c3 HBASE-26172 Deprecated MasterRegistry (#3566) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-19 16:30:48 +08:00
Michael Stack 101602007a
HBASE-24337 Backport HBASE-23968 to branch-2 (#3588) 
Signed-off-by: stack <stack@apache.org>
 2021-08-18 10:19:35 -07:00
Wellington Ramos Chevreuil f2f99aea34 Amend HBASE-26187 UTs, adding a check for Split procedure completion before proceeding with test logic. (#3598) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f2e2140b57)
 2021-08-18 16:41:08 +01:00
Wellington Ramos Chevreuil 30c20d7627
HBASE-26187 Write straight into the store directory when Splitting an… (#3574) (#3595) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-18 09:42:03 +01:00
Xiaolin Ha 4ccd215354 HBASE-26087 JVM crash when displaying RPC params by MonitoredRPCHandler (#3489) 
Signed-off-by: stack <stack@apache.org>
 2021-08-18 14:39:28 +08:00
Duo Zhang d99ca37307 HBASE-26193 Do not store meta region location as permanent state on zookeeper (#3583) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
 2021-08-17 18:58:39 +08:00
eomiks 5eac407944 HBASE-24652 master-status UI make date type fields sortable 2021-08-16 16:03:29 -07:00
stack 63c57d7476 HBASE-26200 Revert "HBASE-25165 Change 'State time' in UI so sorts (#2508)" 
This reverts commit d07d181ea4.
 2021-08-16 13:49:55 -07:00
lzx404243 4aec407c02 HBASE-25680 Non-idempotent test in TestReplicationHFileCleaner (#2984) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-14 11:11:34 +08:00
chenglei 725474bbe3 HBASE-26026 HBase Write may be stuck forever when using CompactingMemStore (#3421) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-12 23:19:32 +08:00
Xiaolin Ha e8ecafb22b HBASE-26155 JVM crash when scan (#3553) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-12 17:17:03 +08:00
Rushabh Shah 6b9114d95e
HBASE-26185 Return mutable list in AssignmentManager#getExcludedServersForSystemTable (#3577) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-08-12 13:47:52 +05:30
stack 243e861985 Revert "HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned (#3532)" 
This reverts commit 8f16e34eb2.
 2021-08-11 07:41:22 -07:00
Bryan Beaudreault 8f16e34eb2
HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned (#3532) 
Signed-off-by: stack <stack@apache.org>
 2021-08-10 20:38:06 -07:00
Viraj Jasani 40a7a75a19
HBASE-26185 Fix TestMaster#testMoveRegionWhenNotInitialized in the presence of hbase.min.version.move.system.tables (#3573) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-08-10 10:58:31 +05:30
Richard Marscher 79a6dc6bfd HBASE-6908: Add pluggable rpc queue implementation (#3522) 
Can pass in a FQCN to load as the call queue implementation.

Standardized arguments to the constructor are the max queue length, the
PriorityFunction, and the Configuration.

`PluggableBlockingQueue` abstract class provided to help guide the
correct constructor signature

Hard fails if the class fails to load as a `BlockingQueue<CallRunner>`

Signed-off-by: stack <stack@apache.org>
 2021-08-09 12:55:27 -07:00
Duo Zhang 611ffa7da9 HBASE-26179 TestRequestTooBigException spends too much time to finish (#3571) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-08-07 23:05:53 +08:00
Xiaolin Ha 61fbe6f480 HBASE-26170 handleTooBigRequest in NettyRpcServer didn't skip enough bytes (#3564) 
Signed-off-by: stack <stack@apache.com>
 2021-08-07 23:05:27 +08:00
chenglei 4f0f519f9f HBASE-26142 NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero (#3531) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-05 21:04:42 +08:00
caroliney14 effde97a88 HBASE-25469 Add detailed RIT info in JSON format for consumption as metrics (#3555) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-04 18:23:17 -07:00
Andrew Purtell 8e9253edd8 Revert "Add detailed RIT info in JSON format for consumption as metrics (#3555)" 
Commit is missing JIRA identifier.

This reverts commit 31b5e09ac2.
 2021-08-04 18:23:08 -07:00
caroliney14 31b5e09ac2
Add detailed RIT info in JSON format for consumption as metrics (#3555) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-04 18:04:44 -07:00
Tak Lon (Stephen) Wu c11a3e1b39 Revert "HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529)" 
This reverts commit f049301606.
 2021-08-04 15:55:13 -07:00
Tak Lon (Stephen) Wu fffdcba5bb Revert "HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 (#3543)" 
This reverts commit ca096437d7.
 2021-08-04 15:55:13 -07:00
Duo Zhang 620806e3fb HBASE-26150 Let region server also carry ClientMetaService (#3550) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-05 00:15:49 +08:00
Ruanhui 04d3c7a9d0 HBASE-26166 Fix a minor bug of table list on master ui (#3554) 
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
 2021-08-04 20:42:15 +08:00
Tak Lon (Stephen) Wu ca096437d7
HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 (#3543) 
2/17 commits of HBASE-22120

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-08-02 11:33:19 -07:00
Bryan Beaudreault aa3b07f6bb
HBASE-26154: Adds exception metrics for QuotaExceededException and RpcThrottlingException (#3544) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-08-02 09:51:34 +05:30
Duo Zhang f7baa195e3 HBASE-26151 Reimplement MasterAddressTracker to also cache backup master addresses (#3548) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-31 15:54:06 +08:00
Duo Zhang aff880e147 HBASE-26144 The HStore.snapshot method is never called in main code (#3533) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-31 14:39:13 +08:00
YutSean f3c022740a HBASE-26105 Rectify the expired TODO comment in CombinedBC. (#3509) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-30 21:43:31 +08:00
liever18 6df4cb1cf4
HBASE-26114 when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster cannot start normally (#3541) 
Signed-off-by: Anoop <anoopsamjohn@apache.org>
 2021-07-30 17:13:50 +05:30
Tak Lon (Stephen) Wu f049301606
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 (#3529) 
1/17 commits of HBASE-22120

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-07-29 09:15:10 -07:00
Duo Zhang dd22fecc2b
HBASE-26120 New replication gets stuck or data loss when multiwal groups more than 10 (#3528) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-27 08:14:15 -07:00
Duo Zhang 20a4aaedcc HBASE-26118 The HStore.commitFile and HStore.moveFileIntoPlace almost have the same logic (#3525) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-26 21:47:12 +08:00
Wei-Chiu Chuang 37c3f7b446
HBASE-21946 Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable (#3434) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 4a3c7d73b0)
 2021-07-26 17:43:10 +08:00
Duo Zhang be0a6cea40 HBASE-26119 Polish TestAsyncNonMetaRegionLocator (#3526) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2021-07-26 17:34:01 +08:00
Rushabh Shah 59e2825c93 HBASE-26093 Replication is stuck due to zero length wal file in oldWALs directory (#3504) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-23 09:33:43 -07:00
meiyi 729da5a6d3
HBASE-24734 RegionInfo#containsRange should support check meta table (#3496) (#3517) 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2021-07-22 18:04:24 +08:00
gkanade 8bc180c773
HBASE-24984 WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used with multi operation (#3512) 
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-07-22 08:30:25 +05:30
Aman Poonia cabf0dc51d
HBASE-25986 set default value of normalization enabled from hbase site (#3476) (#3372) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-07-19 15:26:52 +05:30
clarax 55d93fbf2d
HBASE-25973 Balancer should explain progress in a better way in log -… (#3483) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-18 13:04:33 -07:00
YutSean fb01a6c276
HBASE-26083 In branch-2 & branch-1 L1 miss metric is always 0 when using CombinedBlockCache (#3473) 
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by Reid Chan <reidchan@apache.org>
 2021-07-16 10:28:27 +08:00
Xiaolin Ha 692d384ec6
HBASE-26036 DBB released too early in HRegion.get() and dirty data for some operations (#3436) (#3486) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-14 16:30:46 +08:00
binlijin 12f2a16280
HBASE-26086 TestHRegionReplayEvents do not pass in branch-2 and throws NullPointerException (#3480) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-07-14 11:37:25 +08:00
Clara Xiong dfc9ac8ffb HBASE-25739 TableSkewCostFunction need to use aggregated deviation - backport 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@duboce.net>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-07-13 08:37:29 -07:00
mokai 3294d32f46 HBASE-25700 Enhance znode parent validation when add_peer (#3458) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2021-07-11 21:15:57 +08:00
Duo Zhang b86d2d4020 HBASE-26069 Remove HStore.compactRecentForTestingAssumingDefaultPolic… (#3462) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-07 21:57:59 +08:00
Duo Zhang 1ba03cae4b HBASE-26068 The last assertion in TestHStore.testRefreshStoreFilesNotChanged is wrong (#3461) 
Signed-off-by: Yulin Niu <niuyulin@apache.org
 2021-07-07 14:39:03 +08:00
Duo Zhang bde5d98338 HBASE-26065 StripeStoreFileManager does not need to throw IOException for most methods (#3459) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-07-05 22:19:17 +08:00
Viraj Jasani 342a478753
HBASE-22923 Consider minVersionToMoveSysTables while moving region and creating regionPlan (ADDENDUM) (#3455) 
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-07-03 22:44:46 +05:30
Wei-Chiu Chuang db75ef265e
HBASE-26050 Remove the reflection used in FSUtils.isInSafeMode (#3445) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 82c44b49f5)
 2021-07-02 14:21:28 +08:00
Viraj Jasani 1381df4c80
HBASE-22923 min version of RegionServer to move system table regions (#3439) (#3438) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-07-01 16:24:14 +05:30
Viraj Jasani 75cd25386a
HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade (#3417) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-01 15:20:38 +05:30
bsglz b90fbb77c6 HBASE-26028 The view as json page shows exception when using TinyLfuBlockCache (#3420) 
(cherry picked from commit 147b030c1f)
 2021-07-01 10:27:24 +08:00
Duo Zhang 748e369629 HBASE-26029 It is not reliable to use nodeDeleted event to track region server's death (#3430) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2021-06-30 11:46:09 +08:00
Duo Zhang f827a4c5f8 HBASE-26039 TestReplicationKillRS is useless after HBASE-23956 (#3440) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-06-30 08:27:17 +08:00
GeorryHuang 166becfd66
HBASE-25980 Master table.jsp pointed at meta throws 500 when no all r… (#3373) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-27 22:30:46 +08:00
GeorryHuang c707a9ffff
HBASE-25914 Provide slow/large logs on RegionServer UI (#3321) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-27 22:27:06 +08:00
GeorryHuang c7c5d56084 HBASE-26015 Should implement getRegionServers(boolean) method in Asyn… (#3406) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-27 22:13:21 +08:00
YutSean 193a94e523
HBASE-26013 Get operations readRows metrics becomes zero after HBASE-25677 (#3411) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-06-24 15:24:50 +08:00
Duo Zhang 9fe1781037 HBASE-26020 Split TestWALEntryStream.testDifferentCounts out (#3409) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-06-23 22:51:23 +08:00
litao c81a5ab565
HBASE-25934 Add username for RegionScannerHolder (#3325) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-06-23 12:54:41 +05:30
lujiefsi 2ae69a883c HBASE-25877 Add access check for compactionSwitch (#3253) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-22 23:45:36 +08:00
belugabehr bda7afe97d HBASE-25937: Clarify UnknownRegionException (#3330) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-22 22:44:05 +08:00
Duo Zhang 12d707c880 HBASE-25992 Addendum add missing catch WALEntryFilterRetryableException back 2021-06-21 23:58:42 +08:00
Viraj Jasani 41a68325c2
HBASE-25698 Fixing IllegalReferenceCountException when using TinyLfuBlockCache (#3215) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-06-21 12:51:59 +05:30
YutSean 029dc81b39
HBASE-26001 When turn on access control, the cell level TTL of Increment and Append operations is invalid (#3403) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-06-21 15:00:50 +08:00
Duo Zhang c800e6c825 HBASE-25992 Polish the ReplicationSourceWALReader code for 2.x after HBASE-25596 (#3376) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-20 16:38:13 +08:00
Bharath Vissapragada 3b58919246
HBASE-25998: Redo synchronization in SyncFuture (#3399) 
Currently uses coarse grained synchronized approach that seems to
create a lot of contention. This patch

- Uses a reentrant lock instead of synchronized monitor
- Switches to a condition variable based waiting rather than busy wait
- Removed synchronization for unnecessary fields

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 6bafb59642)
 2021-06-18 10:37:10 -07:00
Bharath Vissapragada 836fd3c52a
HBASE-25984: Avoid premature reuse of sync futures in FSHLog (#3371) (#3392) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 5a19bcfa98)
 2021-06-17 12:27:23 -07:00
Duo Zhang 26c4e86ebd HBASE-25976 Implement a master based ReplicationTracker (#3390) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-06-17 18:38:43 +08:00
binlijin 4719a2078e HBASE-25997 NettyRpcFrameDecoder decode request header wrong when han… (#3380) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-15 14:39:47 +08:00
Toshihiro Suzuki 6f1e652091
HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update (#3386) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-15 13:55:21 +09:00
Duo Zhang cb5f0dfe1e HBASE-25995 Change the method name for DoubleArrayCost.setCosts (#3381) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-15 11:55:54 +08:00
Andrew Purtell a1d59bf1a5 HBASE-25994 Active WAL tailing fails when WAL value compression is enabled (#3377) 
Depending on which compression codec is used, a short read of the
compressed bytes can cause catastrophic errors that confuse the WAL reader.
This problem can manifest when the reader is actively tailing the WAL for
replication. To avoid these issues when WAL value compression is enabled,
BoundedDelegatingInputStream should assume enough bytes are available to
supply a reader up to its bound. This behavior is valid per the contract
of available(), which provides an _estimate_ of available bytes, and
equivalent to IOUtils.readFully but without requiring an intermediate
buffer.

Added TestReplicationCompressedWAL and TestReplicationValueCompressedWAL.
Without the WALCellCodec change TestReplicationValueCompressedWAL will
fail.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-06-14 17:18:53 -07:00
bsglz 88f031eeec HBASE-25967 The readRequestsCount does not calculate when the outResu… (#3351) 
* HBASE-25967 The readRequestsCount does not calculate when the outResults is empty

Co-authored-by: Zheng Wang <wangzheng@apache.org>
(cherry picked from commit 329f0baa98)
 2021-06-10 10:24:18 +08:00
Xiaolin Ha b6837295f3 HBASE-25981 JVM crash when displaying RegionServer UI (#3364) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-09 18:16:33 +08:00
Duo Zhang e4dd4901a8 HBASE-25963 HBaseCluster should be marked as IA.Public (#3348) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-06 21:27:02 +08:00
meiyi 1267bdb1a4 HBASE-25929 RegionServer JVM crash when compaction (#3318) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-03 17:20:08 +08:00
Andrew Purtell a4e8ee183e HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime (#3302) 
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

Conflicts:
	hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBase.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java
	hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupSystemTable.java
	hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
	hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTracing.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestManyRegions.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/MoveRegionsOfTableAction.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationCrossDiffHdfs.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestEnableRSGroups.java
	hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractWALRoller.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaUpdatesGoToPriorityQueue.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/RestoreSnapshotFromClientSimpleTestBase.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnection.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClientCloneLinksAfterDelete.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMutationGetCellBuilder.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotMetadata.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetricsWrapper.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaAssignmentWithStopMaster.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionStateStore.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/janitor/TestCatalogJanitor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureSchedulerPerformanceEvaluation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCloneSnapshotProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRestoreSnapshotProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/procedure2/store/region/RegionProcedureStorePerformanceEvaluation.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMajorCompaction.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionOpen.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSimpleTimeRangeTracker.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCloseChecker.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/replication/master/TestRecoverStandbyProcedure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
	hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
	hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 2021-06-01 12:41:15 -07:00
Bharath Vissapragada d4285be5c1
HBASE-25932: Ensure replication reads the trailer bytes from WAL. (#3332) (#3345) 
This bug was exposed by the test from HBASE-25924. Since this wal
implementations close the wal asynchronously, replication can potentially
miss the trailer bytes. (see jira comment for detailed analysis).

While this is not a correctness problem (since trailer does not have any entry data),
it erroneously bumps a metric that is used to track skipped bytes in WAL resulting
in false alarms which is something we should avoid.

Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
(cherry picked from commit b04c3c7786)
 2021-06-01 08:04:12 -07:00
Michael Stack 1a45b743a0 HBASE-25940 Update Compression/TestCompressionTest: LZ4, SNAPPY, LZO (#3334) 
Undo asserts that LZ4 and SNAPPY fails if their native libs are NOT
loaded; as of hadoop 3.3.1, LZ4 and SNAPPY can work w/o native libs.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-29 09:04:08 -07:00
Sandeep Pal 5157521213
HBASE-25927: Fix the log messages by not stringifying the exceptions in log (#3338) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 9a2027bf71)
 2021-05-28 21:34:14 -07:00
Duo Zhang 2a84fd40d3 HBASE-25938 The SnapshotOfRegionAssignmentFromMeta.initialize call in FavoredNodeLoadBalancer is just a dummy one (#3329) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-29 11:10:52 +08:00
Baiqiang Zhao bbe9558040
HBASE-25745 Deprecate/Rename config `hbase.normalizer.min.region.count` to `hbase.normalizer.merge.min.region.count` 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-05-28 13:10:20 -07:00
Victor 5aa39a975c
HBASE-25910 - Fix port assignment test (#3308) 
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-28 20:53:20 +05:30
Wellington Ramos Chevreuil cad9ae386a HBASE-25933 Log trace raw exception, instead of cause message in NettyRpcServerRequestDecoder (#3323) 
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2021-05-27 20:03:17 +01:00
Duo Zhang 3cf0678aee HBASE-25926 Cleanup MetaTableAccessor references in FavoredNodeBalancer related code (#3313) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-27 16:25:01 +08:00
Rushabh Shah e265eccf20
HBASE-25924 Re-compute size of WAL file while removing from WALEntryStream (#3315) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-05-26 10:42:03 -07:00
Baiqiang Zhao 563ebc214a
HBASE-25534 Honor TableDescriptor settings earlier in normalization (#2917) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-05-25 08:24:05 -07:00
Duo Zhang 2cb6cc8a6c HBASE-25894 Improve the performance for region load and region count related cost functions (#3276) 
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
 2021-05-25 18:07:33 +08:00
GeorryHuang 8754e88fa7
HBASE-25906 UI of master-status to show recent history of balancer desicion (#3303) (#3296) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-25 12:39:47 +05:30
Baiqiang Zhao 073f23e2bd HBASE-25513 When the table is turned on normalize, the first region may not be merged even the size is 0 (#2887) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-05-24 17:27:06 -07:00
Anoop Sam John 1649013b99 HBASE-25898 RS getting aborted due to NPE in Replication WALEntryStream (#3292) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
 2021-05-24 23:44:32 +05:30
Xiaolin Ha 76d56bc42e HBASE-25899 Improve efficiency of SnapshotHFileCleaner (#3280) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-24 22:18:43 +08:00