d851133b6f
HBASE-26670 HFileLinkCleaner should be added even if snapshot is disabled ( #4032 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-14 12:25:44 +08:00
a1994ee152
HBASE-26830 Rewrite TestLruBlockCache to make it more stable ( #4212 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit c3d0cc7040
2022-03-14 11:24:45 +08:00
6cef9e1602
HBASE-25465 Use javac --release option for supporting cross version compilation ( #4164 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-12 18:30:40 +08:00
5844b53dea
HBASE-26802 Backport the log4j2 changes to branch-2 ( #4166 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-hadoop-compat/pom.xml
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml
hbase-shaded/hbase-shaded-client/pom.xml
hbase-shaded/hbase-shaded-mapreduce/pom.xml
hbase-shaded/hbase-shaded-testing-util/pom.xml
hbase-shaded/pom.xml
hbase-testing-util/pom.xml
2022-03-11 11:38:37 -08:00
bf9233f511
HBASE-25709 Close region may stuck when region is compacting and skipped most cells read ( #3117 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-11 09:28:59 -08:00
70ec3beef4
HBASE-26242 Allow split when store file count larger than the configed blocking file count ( #3652 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-11 09:28:59 -08:00
b08321c08c
HBASE-26729 Backport "HBASE-26714 Introduce path configuration for system coprocessors" to branch-2 ( #4093 )
...
- rename from HBaseCommonTestingUtil to HBaseCommonTestingUtility
Signed-off-by: Ankit Singhal <ankit@apache.org>
2022-03-11 09:28:59 -08:00
eebc2e0bbb
HBASE-26819 Minor code cleanup in and around RpcScheduler
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-11 11:16:05 +01:00
775c020625
HBASE-26817 Mark RpcExecutor as IA.LimitedPrivate COPROC and PHOENIX
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-11 11:08:54 +01:00
057bc95268
HBASE-26761 TestMobStoreScanner (testGetMassive) can OOME ( #4204 )
...
Change the timing of the test but not the objective by storing three large
MOB values into the row with three separate puts, each randomizing the
data. Increases running time of the testGetMassive case 2x but avoids OOME
in the test environment where the OOME was consistently reproductable.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2022-03-11 15:13:06 +08:00
cc256bd78f
HBASE-26824 TestHBaseTestingUtil.testResolvePortConflict failing after HBASE-26582 ( #4203 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-03-11 15:07:01 +08:00
5386325acd
HBASE-26582 Prune use of Random and SecureRandom objects ( #4118 )
...
Avoid the pattern where a Random object is allocated, used once or twice, and
then left for GC. This pattern triggers warnings from some static analysis tools
because this pattern leads to poor effective randomness. In a few cases we were
legitimately suffering from this issue; in others a change is still good to
reduce noise in analysis results.
Use ThreadLocalRandom where there is no requirement to set the seed to gain
good reuse.
Where useful relax use of SecureRandom to simply Random or ThreadLocalRandom,
which are unlikely to block if the system entropy pool is low, if we don't need
crypographically strong randomness for the use case. The exception to this is
normalization of use of Bytes#random to fill byte arrays with randomness.
Because Bytes#random may be used to generate key material it must be backed by
SecureRandom.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-11 15:06:48 +08:00
e8cca6d111
HBASE-26715 Blocked on SyncFuture in AsyncProtobufLogWriter#write ( #4184 )
...
Co-authored-by: Bryan Beaudreault <bbeaudreault@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-03-10 12:16:03 -08:00
99912c9783
HBASE-26552 Introduce retry to logroller to avoid abort ( #4170 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-10 12:16:03 -08:00
1f9e5a1f7c
HBASE-26814 Addendum: Fix spotbugs warning and clarify intent ( #4201 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-10 10:10:16 -08:00
9359449a3a
HBASE-26782 Minor code cleanup in and around RpcExecutor
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-10 14:32:47 +01:00
206100f515
HBASE-26814 Default StoreHotnessProtector to off, with logs to guide when to turn it on ( #4194 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-09 14:44:29 -08:00
8251e8d2c9
HBASE-26783 ScannerCallable doubly clears meta cache on retries ( #4147 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-09 12:03:12 -08:00
6facc82585
HBASE-26784 Use HIGH_QOS for ResultScanner.close requests ( #4163 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-03-08 11:42:12 -08:00
84eb868f98
HBASE-26764 Implement generic exception support for TraceUtil methods over Callables and Runnables
...
For the `TraceUtil` methods that accept `Callable` and `Runnable` types, make them generic over a
child of `Throwable`. This allows us to consolidate the two method signatures into a single more
flexible definition.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-07 13:57:22 +01:00
9ddc90e258
HBASE-26792 Implement ScanInfo#toString ( #4154 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-05 17:49:26 +08:00
09bd726d46
HBASE-26759 Fix trace continuity through CallRunner ( #4126 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-04 16:03:02 -08:00
6c70e4b4c5
[branch-2.5] Amend HBASE-26731 Add metrics for active and expired scanners
...
Compliation fixes
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-04 14:11:33 -08:00
0f28fa8433
HBASE-26731 Add metrics for active and expired scanners ( #4145 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionServer.java
2022-03-04 14:07:16 -08:00
d4060aeda7
HBASE-26777 BufferedDataBlockEncoder$OffheapDecodedExtendedCell.deepC… ( #4139 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-04 13:53:29 -08:00
71ddf74dda
HBASE-26691 Replacing log4j with reload4j for branch-2.x ( #4050 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-04 12:08:36 -08:00
d971a9d298
HBASE-26622 Update error-prone to 2.10
...
Author: Mike Drob <mdrob@apache.org>
Co-authored-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-03 11:52:00 +01:00
4858b2543c
HBASE-26762 Un-Deprecate and improve documentation for Scan#setRowPrefixFilter ( #4119 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-02 23:34:02 +08:00
ff7f18923a
HBASE-26776 RpcServer failure to SASL handshake always logs user "unknown" to audit log ( #4138 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2022-03-02 15:52:22 +01:00
5f11bc4e50
HBASE-26703 Allow configuration of IPC queue balancer ( #4063 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java
2022-02-18 17:55:06 -08:00
1fb4ee37be
HBASE-26727 Fix CallDroppedException reporting ( #4088 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-02-18 17:27:59 -08:00
2be2072704
HBASE-26576 Allow pluggable queue to be used with the fast path executor or normal balanced executor ( #3944 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-02-18 10:26:35 -08:00
932bf591b3
HBASE-26659 The ByteBuffer of metadata in RAMQueueEntry in BucketCache could be reused. ( #4026 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-02-19 00:42:04 +08:00
af338d2c99
HBASE-26712 Balancer encounters NPE in rare case ( #4112 ) ( #4092 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-02-16 12:15:12 +05:30
42bb0afaf8
HBASE-26742 Comparator of NOT_EQUAL NULL is invalid for checkAndMutate ( #4105 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-02-16 11:23:48 +08:00
a19dcc3be0
HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks' ( #4094 )
...
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-02-15 18:39:03 -08:00
40882b5681
HBASE-26434 Do compact when all L0 files are expired ( #3830 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-02-15 22:39:14 +08:00
3803b8be2c
HBASE-26688 Threads shared EMPTY_RESULT may lead to unexpected client job down ( #4073 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-02-15 20:31:19 +08:00
600a6a8faf
HBASE-26521 Name RPC spans as `$package.$service/$method` ( #4024 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-02-10 12:58:01 +01:00
b64a507340
HBASE-26709 Ban the usage of junit 3 TestCase ( #4065 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-02-10 01:06:21 +08:00
12002e6a79
HBASE-26724 Backport the UT changes in HBASE-24510 to branch-2.x ( #4081 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2022-02-10 00:53:14 +08:00
d091751ab0
HBASE-26726 Allow disable of region warmup before graceful move ( #4086 )
...
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
2022-02-01 15:39:56 -08:00
59d60fd421
HBASE-26713 Default to LATEST_TIMESTAMP if no timestamp sent along on Increment/Append ( #4075 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-01-28 12:57:59 -08:00
9f1f705686
HBASE-26523 Upgrade hbase-thirdparty dependency to 4.0.1 ( #3988 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2022-01-27 17:33:37 -08:00
4d86d3be08
HBASE-26614 Refactor code related to "dump"ing ZK nodes ( #3969 )
...
The code starting at `ZKUtil.dump(ZKWatcher)` is a small mess – it has cyclic dependencies woven
through itself, `ZKWatcher` and `RecoverableZooKeeper`. It also initializes a static variable in
`ZKUtil` through the factory for `RecoverableZooKeeper` instances. Let's decouple and clean it
up.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2022-01-26 09:00:52 -08:00
2c82cf3207
HBASE-26689 Backport HBASE-24443 Refactor TestCustomSaslAuthenticationProvider ( #4049 )
...
Duo Zhang <zhangduo@apache.org>
2022-01-20 15:05:45 +01:00
0ee15e0865
HBASE-26474 Implement connection-level attributes ( #4014 )
...
Add support for `db.system`, `db.connection_string`, `db.user`.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
Co-authored-by: Josh Elser <josh.elser@gmail.com>
2022-01-18 16:09:08 -08:00
540fbe51b0
HBASE-26551 Add FastPath feature to HBase RWQueueRpcExecutor ( #4036 )
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2022-01-18 11:13:13 +08:00
b840ed68f5
HBASE-26671 Misspellings of hbck usage ( #4033 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-01-16 16:34:34 +08:00
9ef7f5977e
HBASE-26616 Refactor code related to ZooKeeper authentication ( #3973 )
...
This refactor reduces the size and scope of the `ZKUtil` class. The core of this refactor is
moving the `login*` methods from `ZKUtil` into their own class, `ZKAuthentication`. The class
`JaasConfiguration` is also moved along with them.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-01-10 16:08:17 -08:00
6a060adc8b
HBASE-26397 Display the excluded datanodes on regionserver UI ( #3990 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-01-10 13:03:49 +08:00
2abd09f6ac
HBASE-26643 LoadBalancer should not return empty map ( #4005 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2022-01-08 00:39:59 +08:00
76bc412fc4
HBASE-26347 Support detect and exclude slow DNs in fan-out of WAL ( #3800 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-01-06 11:48:50 +08:00
2b05e68ae1
HBASE-26578 CompactedHFilesDischarger should be shutdown when aborting regionserver ( #4001 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-01-05 16:33:59 +08:00
e50f4991da
HBASE-26567 Remove IndexType from ChunkCreator ( #3947 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-01-05 00:08:29 +08:00
deec701475
HBASE-26641 Split TestMasterFailoverWithProcedures ( #3994 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2022-01-03 00:41:36 +08:00
efeec919de
HBASE-26494 Using RefCnt to fix the flawed MemStoreLABImpl ( #3983 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-01-01 20:57:36 +08:00
0001ad6b8b
HBASE-26623 Report CallDroppedException in exception metrics ( #3980 )
...
`CallDroppedException` can be thrown with `CallRunner.drop()` by queue implementations that decide to drop calls to groom the RPC call backlog. The LifoCoDel queue does this I believe and with Pluggable queue it's possible for 3rd party queue implementations to be using `drop()` for similar reasons. It would be nice for the server to be tracking these exceptions in metrics since otherwise you might have to do some extra lifting on the client side.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2021-12-30 00:05:12 +08:00
be1d3d856f
HBASE-26564 Retire the method visitLogEntryBeforeWrite without RegionInfo in WALActionListner ( #3985 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-28 21:55:56 +08:00
afabefb051
HBASE-26615 Snapshot referenced data files are deleted when delete a table with merge regions ( #3976 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2021-12-25 22:32:15 -08:00
51d38b5e4f
HBASE-26609 Round the size to MB or KB at the end of calculation in HRegionServer.createRegionLoad ( #3967 )
...
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-12-23 11:58:35 +08:00
a57f4976c1
HBASE-26598 Fix excessive connections in MajorCompactor ( #3961 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-23 11:55:31 +08:00
e28ccf092b
HBASE-26580 The message of StoreTooBusy is confused ( #3949 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2021-12-18 23:34:08 +08:00
ef3854225a
HBASE-26579 Set storage policy of recovered edits when hbase.wal.storage.type is configured ( #3948 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-12-18 23:27:54 +08:00
482ef6904e
HBASE-26488 Memory leak when MemStore retry flushing ( #3899 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-18 12:22:52 +08:00
d072a2e694
HBASE-26340 TableSplit returns false size under 1MB ( #3872 )
...
Signed-off-by: Peter Somogyi <psomogyi@apache.org
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-18 12:21:51 +08:00
37397637ad
HBASE-26542 Apply a `package` to test protobuf files (addendum)
...
RpcServer identifies the services it hosts by unqualified service name. Thus, use `getName()`
instead of `getFullName()`. See also HBASE-26589.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-12-16 16:39:48 -08:00
2bfb743d79
HBASE-26550 Make sure the master is running normally before accepting a balance command ( #3928 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-14 23:32:34 +08:00
49d4c4a4c0
HBASE-26027 The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException ( #3925 )
...
* HBASE-26027 The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException
2021-12-13 16:07:42 -08:00
9cf0af6e79
HBASE-26537: Make HBASE-15676 backwards compatible, using a flag on the proto ( #3931 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-12-13 13:34:10 -08:00
140b5d8d26
HBASE-24870 Ignore TestAsyncTableRSCrashPublish ( #2248 )
...
Signed-off-by: meiyi <myimeiyi@gmail.com>
2021-12-06 19:07:07 -08:00
ec52519f7c
HBASE-26462 Should persist restoreAcl flag in the procedure state for CloneSnapshotProcedure and RestoreSnapshotProcedure ( #3921 )
...
Signed-off-by: Yu Li <liyu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-06 23:16:15 +08:00
074456e190
HBASE-26533 KeyValueScanner might not be properly closed when using InternalScan.checkOnlyMemStore() ( #3917 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-05 22:35:07 +08:00
19d991d291
HBASE-26517 Add auth method information to AccessChecker audit log ( #3897 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-04 23:14:42 +08:00
d504504875
HBASE-26304 Reflect out of band locality improvements in metrics and balancer ( #3895 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-04 12:23:09 +08:00
d1c68a2d5c
HBASE-26471 Move tracing semantic attributes to their own class ( #3896 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
2021-12-01 13:24:08 -08:00
6fc5bbc19a
HBASE-26468 Region Server doesn't exit cleanly incase it crashes. ( #3865 ) ( #3862 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-12-01 11:06:07 +05:30
774484ed44
HBASE-25905 Shutdown of WAL stuck at waitForSafePoint ( #3898 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-12-01 08:34:08 +08:00
799217ee69
HBASE-26455 TestStochasticLoadBalancerRegionReplicaWithRacks fails consistently ( #3879 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2021-11-28 22:23:21 +08:00
5437532910
HBASE-26476 Make DefaultMemStore extensible for HStore.memstore ( #3869 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-27 12:06:35 +08:00
5101f372e4
HBASE-26465 MemStoreLAB may be released early when its SegmentScanner is scanning ( #3859 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Anoop Sam John <anoopsamjohn@apache.org>
2021-11-25 21:35:55 +08:00
6aa2daa2c2
HBASE-26477 Fix MiniMapReduceCluster failure under Java17 ( #3878 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-25 16:27:23 +08:00
fa34cfc7d1
Revert "Backport HBASE-26477 Fix MiniMapReduceCluster failure under Java17 to branch-2 ( #3878 )"
...
This reverts commit ccf150434d
2021-11-25 16:27:11 +08:00
ccf150434d
Backport HBASE-26477 Fix MiniMapReduceCluster failure under Java17 to branch-2 ( #3878 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-25 16:25:44 +08:00
2798ef132b
HBASE-26479 Print too slow/big scan's operation_id in region server log ( #3870 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-25 11:43:01 +08:00
52ce87780a
Revert "HBASE-26479 Print too slow/big scan's operation_id in region server log ( #3870 )"
...
This reverts commit b437a96272
2021-11-24 17:34:01 -05:00
b437a96272
HBASE-26479 Print too slow/big scan's operation_id in region server log ( #3870 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-24 23:38:16 +08:00
0a51d2b5be
HBASE-26475 The flush and compact methods in HTU should skip processing secondary replicas ( #3868 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-11-23 18:51:25 +08:00
f8d03d1227
HBASE-26249 Ameliorate compaction made by bulk-loading files ( #3831 ) ( #3874 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-23 15:52:47 +08:00
61406252a5
Revert "Backport HBASE-26249 Ameliorate compaction made by bulk-loading files ( #3831 ) ( #3874 )"
...
This reverts commit 5843a9a44c
2021-11-23 15:52:34 +08:00
5843a9a44c
Backport HBASE-26249 Ameliorate compaction made by bulk-loading files ( #3831 ) ( #3874 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-23 15:45:21 +08:00
4312f817f3
HBASE-26467 Fix bug for MemStoreLABImpl.forceCopyOfBigCellInto(Cell) ( #3858 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: chenglei <chenglei@apache.org>
2021-11-21 21:02:18 +08:00
e549384437
HBASE-26463 Unreadable table names after HBASE-24605 ( #3853 )
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-21 20:49:13 +08:00
8d3562c6fa
HBASE-26421 Use HFileLink file to replace entire file's reference when splitting ( #3854 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-18 21:37:10 +08:00
4a8da47f74
HBASE-26267 Don't try to recover WALs from a WAL dir which doesn't exist ( #3679 )
...
We currently cause an error to be thrown by trying to list a
non-existent directory. We see that the master region directory exists
on the filesystem, but forget to make sure that the master region's WAL
directory also exists before we try to list it.
2021-11-16 16:27:45 -05:00
e65f28cc57
HBASE-26444 BucketCacheWriter should log only the BucketAllocatorException message, not the full stack trace ( #3840 )
...
Signed-off-by: Anoop <anoopsamjohn@apache.org>
2021-11-15 16:13:36 -08:00
23024edba2
HBASE-26443 Some BaseLoadBalancer log lines should be at DEBUG level ( #3838 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-11-15 16:06:59 -08:00
98a4fb8086
HBASE-26438 Fix flaky test TestHStore.testCompactingMemStoreCellExceedInmemoryFlushSize ( #3834 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-12 23:03:37 +08:00
ad36941932
HBASE-26450 Server configuration will overwrite HStore configuration after using shell command 'update_config' ( #3843 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
2021-11-12 11:43:48 +08:00
9db7630d23
HBASE-26410 Fix HBase TestCanaryTool for Java17 ( #3809 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-09 22:21:01 +08:00
cc2e4aec87
HBASE-26337 Optimization for weighted random generators ( #3828 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-09 21:27:44 +08:00
1302073dfe
HBASE-26430 Increase log level in DefaultHeapMemoryTuner from DEBUG to INFO ( #3820 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-11-08 18:35:39 -08:00
201b3c7fc7
HBASE-26429 HeapMemoryManager fails memstore flushes with NPE if enabled ( #3819 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-11-08 18:32:17 -08:00
2fbb375cd5
HBASE-25322 Redundant Reference file in bottom region of split ( #3814 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-08 14:48:55 +08:00
ebd89b58eb
Revert "HBASE-25322 Redundant Reference file in bottom region of split ( #3814 )"
...
This reverts commit a6edf44175
2021-11-08 11:40:43 +08:00
a6edf44175
HBASE-25322 Redundant Reference file in bottom region of split ( #3814 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-05 18:17:47 +08:00
e06208f3d6
HBASE-26311 Balancer gets stuck in cohosted replica distribution ( #3804 )
...
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-11-03 11:58:38 -07:00
9eb3706021
HBASE-26309 Balancer tends to move regions to the server at the end of list ( #3811 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-02 21:58:50 +08:00
063e0e2e92
HBASE-26406 Can not add peer replicating to non-HBase ( #3806 )
...
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b9b7fec57f
2021-11-02 15:03:33 +08:00
e89f62e611
HBASE-26384 Segment already flushed to hfile may still be remained in CompactingMemStore ( #3779 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-01 23:05:57 +08:00
f2c58fcf68
HBASE-26353 Support loadable dictionaries in hbase-compression-zstd ( #3787 )
...
ZStandard supports initialization of compressors and decompressors with a
precomputed dictionary, which can dramatically improve and speed up compression
of tables with small values. For more details, please see
The Case For Small Data Compression
https://github.com/facebook/zstd#the-case-for-small-data-compression
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdCodec.java
2021-10-27 07:57:16 -07:00
b4177577ac
HBASE-26394 Cache in RSRpcServices.executeProcedures does not take effect ( #3794 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-10-26 15:44:17 +08:00
32eb23c8f3
HBASE-26327 Replicas cohosted on a rack shouldn't keep triggering balancer ( #3788 )
...
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-10-25 15:44:11 -07:00
444dec9289
HBASE-26190 High rate logging of BucketAllocatorException: Allocation too big - Addendum ( #3792 )
2021-10-24 09:22:26 +05:30
f299d56934
HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger ( #3783 )
2021-10-22 09:46:58 -07:00
eea2ea8576
HBASE-26368 Fix checkstyle issues for HRegionServer for branch-2 ( #3781 ) ( #3764 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-10-22 12:51:29 +05:30
0eaadd92d8
HBASE-26385 Clear CellScanner when replay ( #3773 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-10-21 21:56:39 +08:00
dce45ec16a
HBASE-26383 HBCK incorrectly reports inconsistencies for recently split regions following a master failover ( #3775 )
...
A regression was introduced by HBASE-25847 which changed regionInfo#isParentSplit
to regionState#isSplit. The region state after restart is CLOSED instead of SPLIT.
We need to check both regionState and regionInfo for split status.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-10-20 15:55:07 -07:00
5c4a053f2a
HBASE-26371 Prioritize meta region move over other region moves in region_mover ( #3771 ) ( #3767 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-10-20 23:06:25 +05:30
b6bb18022f
HBASE-26316 Per-table or per-CF compression codec setting overrides ( #3730 )
...
We get and retain Compressor instances in HFileBlockDefaultEncodingContext,
and could in theory call Compressor#reinit when setting up the context,
to update compression parameters like level and buffer size, but we do
not plumb through the CompoundConfiguration from the Store into the
encoding context. As a consequence we can only update codec parameters
globally in system site conf files.
Fine grained configurability is important for algorithms like ZStandard
(ZSTD), which offers more than 20 compression levels, where at level 1
it is almost as fast as LZ4, and where at higher levels it utilizes
computationally expensive techniques to rival LZMA at compression ratio
but trades off significantly for reduced compresson throughput. The ZSTD
level that should be set for a given column family or table will vary by
use case.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdDecompressor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/compress/HFileTestBase.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestDataBlockEncoders.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestSeekToBlockWithEncoders.java
2021-10-19 13:59:04 -07:00
7e574c9f1d
HBASE-26190 High rate logging of BucketAllocatorException: Allocation too big ( #3752 )
...
Reduce the frequency of allocation failed traces by printing them
preiodically (once per minute). Record the allocation failures in the
Bucket Cache Stats and let the stat thread dump cumulative allocation
failures alongside other traces it dumps.
Also, this change adds trace for the Table name, Column Family and
HFileName for the most recent allocation failure in last 1 minute.
Signed-off-by: Anoop <anoopsamjohn@apache.org>
2021-10-19 13:21:11 +05:30
45b09aa4d7
HBASE-26350 Add a DEBUG when we fail the SASL handshake
...
Closes #3743
Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-10-14 15:27:32 -04:00
3ce9052188
HBASE-24833: Bootstrap should not delete the META table directory if … ( #2237 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
2021-10-12 10:56:00 -07:00
10584d70d2
HBASE-26306 Backport "HBASE-26220 Use P2P communicate between region servers to sync the list for bootstrap node" to branch-2 ( #3727 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-10-10 22:56:12 +08:00
5af02201d2
HBASE-26324 Reuse compressors and decompressors in WAL CompressionContext ( #3728 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-10-07 09:46:38 -07:00
18b9fa8a3c
HBASE-26259 Fallback support to pure Java compression ( #3691 )
...
This change introduces provided compression codecs to HBase as
new Maven modules. Each module provides compression codec support
that formerly required Hadoop native codecs, which in turn relies
on native code integration, which may or may not be available on
a given hardware platform or in an operational environment. We
now provide codecs in the HBase distribution for users whom for
whatever reason cannot or do not wish to deploy the Hadoop native
codecs.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileEncryption.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestCompressedWAL.java
2021-10-06 13:48:33 -07:00
706082d513
HBASE-26294 Backport "HBASE-26181 Region server and master could use itself as ConnectionRegistry" to branch-2 ( #3708 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-10-05 22:11:01 +08:00
9a33e234e7
HBASE-24601: Change default Hfile storage policy from HOT to NONE for HDFS ( #1940 )
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-10-02 16:37:39 +08:00
a2c09ae5bc
HBASE-26305 Move NavigableSet add operation to writer thread in BucketCache ( #3705 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: chenglei <chenglei@apache.org>
2021-10-01 21:06:57 +08:00
0748165ab0
HBASE-26295 BucketCache could not free BucketEntry which restored fro… ( #3699 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-29 21:46:57 +08:00
08800e19d6
HBASE-26251 StochasticLoadBalancer metrics should update even if balancer doesn't run ( #3706 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2021-09-29 21:25:39 +08:00
69749ed01e
HBASE-26289 Hbase scan setMaxResultsPerColumnFamily not giving right results ( #3688 )
...
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-29 10:22:24 +02:00
08a2efde16
HBASE-26293 Use reservoir sampling when selecting bootstrap nodes ( #3702 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-09-28 23:41:28 +08:00
786e09a936
HBASE-26238 Short message by Result#compareResults for VerifyReplication to avoid OOME ( #3647 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-28 23:16:20 +08:00
73f42b27bf
HBASE-26297 Balancer run is improperly triggered by accuracy error of double comparison ( #3698 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-26 23:35:50 +08:00
404789adde
HBASE-26178 Improve data structure and algorithm for BalanceClusterState to improve computation speed for large cluster ( #3682 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-25 21:48:51 +08:00
59d133ef95
Amend HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job
...
Compliation fixes for CombinedBlockCache and TestClientSideRegionScanner.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-09-22 12:27:59 -07:00
e0325c40f6
HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job ( #3684 )
...
Introduce `hfile.onheap.block.cache.fixed.size`
and default to disable. when using ClientSideRegionScanner
it will be enabled with a fixed size for caching
INDEX/LEAF_INDEX block when a client, e.g.
snapshot scanner, scans the entire HFile
and does not need to seek/reseek to index
block multiple times.
Signed-off-by: Josh Elser <elserj@apache.org>
2021-09-22 09:38:20 -07:00
84b5511d13
HBASE-26261 Store configuration loss when use update_config ( #3664 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-22 18:35:28 +08:00
8f6c5e1f02
HBASE-26270 Provide getConfiguration method for Region and Store interface ( #3671 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2021-09-22 11:32:32 +08:00
e327fa43c0
HBASE-26288 Revisit the usage of MetaTableLocator when HRegionServer.TEST_SKIP_REPORTING_TRANSITION is true ( #3689 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-09-22 00:27:11 +08:00
ef85204522
HBASE-26285 Remove MetaTableLocator usages in non-migration code ( #3686 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-09-21 23:32:13 +08:00
b21b250570
HBASE-26281 DBB got from BucketCache would be freed unexpectedly before RPC completed ( #3680 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-17 22:28:56 +08:00
f670fc5d4d
HBASE-26229 Limit count and size of L0 files compaction in StripeCompactionPolicy ( #3646 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-09 18:28:45 +08:00
68db7f5ed0
[HBASE-26243]Fix typo for file "hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java" ( #3650 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-09 12:21:54 +08:00
820ba21b2a
HBASE-26106 AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory. ( #3636 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
2021-09-02 10:26:22 -07:00
ccd90269d2
HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky ( #3651 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 345d7256c8
2021-09-02 10:33:41 +08:00
e07928df4c
HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
(Re-application of HBASE-26147 with the correct author metadata)
2021-09-01 21:58:30 -04:00
1e74f08403
Revert "HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed"
...
This reverts commit 5b58d11c91
2021-09-01 21:57:46 -04:00
5b58d11c91
HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org
2021-09-01 21:47:53 -04:00
c639d6f353
HBASE-26136 Backport HBASE-25723 "Temporarily remove the trace support for RegionScanner.next" to branch-2 ( #3623 )
...
13/17 commits of HBASE-22120, original commit 7f90c2201f
2021-09-01 15:29:09 -07:00
d0a53e3f29
HBASE-26133 Backport HBASE-25591 "Upgrade opentelemetry to 0.17.1" to branch-2 ( #3608 )
...
10/17 commits of HBASE-22120, original commit f6ff519dd0
2021-09-01 15:29:09 -07:00
8b7450f5d8
HBASE-26132 Backport HBASE-25535 "Set span kind to CLIENT in AbstractRpcClient" to branch-2 ( #3607 )
...
9/17 commits of HBASE-22120, original commit bb8c4967f8
2021-09-01 15:29:09 -07:00
9cce94a2d0
HBASE-26131 Backport HBASE-25484 "Add trace support for WAL sync" to branch-2 ( #3597 )
...
8/17 commits of HBASE-22120, original commit 2be2c63f0d
2021-09-01 15:29:09 -07:00
321dcd4e77
HBASE-26130 Backport HBASE-25455 "Add trace support for HRegion read/… ( #3594 )
...
7/17 commits of HBASE-22120, original commit 03e12bfa4a
2021-09-01 15:29:09 -07:00
83a27389a0
HBASE-26129 Backport HBASE-25481 "Add host and port attribute when tracing rpc call at client side" to branch-2 ( #3589 )
...
6/17 commits of HBASE-22120, original commit ae2c62ffaa
2021-09-01 15:29:09 -07:00
58b5c007b9
HBASE-26127 Backport HBASE-23898 "Add trace support for simple apis i… ( #3556 )
...
4/17 commits of HBASE-22120, original commit 805b2ae2ad
2021-09-01 15:29:09 -07:00
4dfc59c19e
HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 ( #3543 )
...
2/17 commits of HBASE-22120
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
665305cc3b
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 ( #3529 )
...
1/17 commits of HBASE-22120
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
dbf43dcfbd
HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore ( #3604 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 18:17:13 +08:00
8be2ad2d42
HBASE-25642 Fix or stop warning about already cached block ( #3638 )
...
Our logs have as a fairly common occurrence: 2021-03-05 22:24:31,034 WARN
[StoreFileOpener-foo-1] hfile.BlockCacheUtil: Caching an already cached
block: blah.bub. This is harmless and can happen in rare cases (see HBASE-8547)
Because it is harmless, log at DEBUG level, not WARN.
Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
2021-08-31 09:26:10 -07:00
c08e877d6a
HBASE-26235 We could start RegionServerTracker before becoming active master ( #3645 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-08-30 12:22:41 +08:00
1e49e94051
HBASE-26232 SEEK_NEXT_USING_HINT is ignored on reversed Scans ( #3639 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-29 21:42:48 +08:00
033ff757fd
HBASE-26216 Move HRegionServer.abort(String) to Abortable as a default method ( #3642 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-28 23:06:54 +08:00
3b1482ef60
HBASE-26189 Reduce log level of CompactionProgress notice to DEBUG ( #3579 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-08-27 10:15:28 -07:00
30bb72bdb8
HBASE-26227 Forward port HBASE-26223 test code to branch-2.4+ ( #3632 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-27 16:46:54 +08:00
cd9e45ce8d
HBASE-26219 Negative time is logged while waiting on regionservers ( #3622 )
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-25 16:25:24 +02:00
931a7a5a61
HBASE-26180 Introduce a initial refresh interval for RpcConnectionRegistry ( #3601 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-08-25 18:51:40 +08:00
dacd3404a4
HBASE-26215 The backup master status page should use ActiveMasterManager instead of MasterAddressTracker ( #3621 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-25 16:42:45 +08:00
5dad395869
HBASE-26173 Return only a sub set of region servers as bootstrap nodes ( #3599 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-24 23:17:56 +08:00
6ad7eb88c3
HBASE-26172 Deprecated MasterRegistry ( #3566 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-19 16:30:48 +08:00
101602007a
HBASE-24337 Backport HBASE-23968 to branch-2 ( #3588 )
...
Signed-off-by: stack <stack@apache.org>
2021-08-18 10:19:35 -07:00
f2f99aea34
Amend HBASE-26187 UTs, adding a check for Split procedure completion before proceeding with test logic. ( #3598 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f2e2140b57
2021-08-18 16:41:08 +01:00
30c20d7627
HBASE-26187 Write straight into the store directory when Splitting an… ( #3574 ) ( #3595 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-18 09:42:03 +01:00
4ccd215354
HBASE-26087 JVM crash when displaying RPC params by MonitoredRPCHandler ( #3489 )
...
Signed-off-by: stack <stack@apache.org>
2021-08-18 14:39:28 +08:00
d99ca37307
HBASE-26193 Do not store meta region location as permanent state on zookeeper ( #3583 )
...
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
2021-08-17 18:58:39 +08:00
5eac407944
HBASE-24652 master-status UI make date type fields sortable
2021-08-16 16:03:29 -07:00
63c57d7476
HBASE-26200 Revert "HBASE-25165 Change 'State time' in UI so sorts ( #2508 )"
...
This reverts commit d07d181ea4
2021-08-16 13:49:55 -07:00
4aec407c02
HBASE-25680 Non-idempotent test in TestReplicationHFileCleaner ( #2984 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-14 11:11:34 +08:00
725474bbe3
HBASE-26026 HBase Write may be stuck forever when using CompactingMemStore ( #3421 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-12 23:19:32 +08:00
e8ecafb22b
HBASE-26155 JVM crash when scan ( #3553 )
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-12 17:17:03 +08:00
6b9114d95e
HBASE-26185 Return mutable list in AssignmentManager#getExcludedServersForSystemTable ( #3577 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-08-12 13:47:52 +05:30
243e861985
Revert "HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned ( #3532 )"
...
This reverts commit 8f16e34eb2
2021-08-11 07:41:22 -07:00
8f16e34eb2
HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned ( #3532 )
...
Signed-off-by: stack <stack@apache.org>
2021-08-10 20:38:06 -07:00
40a7a75a19
HBASE-26185 Fix TestMaster#testMoveRegionWhenNotInitialized in the presence of hbase.min.version.move.system.tables ( #3573 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-08-10 10:58:31 +05:30
79a6dc6bfd
HBASE-6908: Add pluggable rpc queue implementation ( #3522 )
...
Can pass in a FQCN to load as the call queue implementation.
Standardized arguments to the constructor are the max queue length, the
PriorityFunction, and the Configuration.
`PluggableBlockingQueue` abstract class provided to help guide the
correct constructor signature
Hard fails if the class fails to load as a `BlockingQueue<CallRunner>`
Signed-off-by: stack <stack@apache.org>
2021-08-09 12:55:27 -07:00
611ffa7da9
HBASE-26179 TestRequestTooBigException spends too much time to finish ( #3571 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-08-07 23:05:53 +08:00
61fbe6f480
HBASE-26170 handleTooBigRequest in NettyRpcServer didn't skip enough bytes ( #3564 )
...
Signed-off-by: stack <stack@apache.com>
2021-08-07 23:05:27 +08:00
4f0f519f9f
HBASE-26142 NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero ( #3531 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-05 21:04:42 +08:00
effde97a88
HBASE-25469 Add detailed RIT info in JSON format for consumption as metrics ( #3555 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-04 18:23:17 -07:00
8e9253edd8
Revert "Add detailed RIT info in JSON format for consumption as metrics ( #3555 )"
...
Commit is missing JIRA identifier.
This reverts commit 31b5e09ac2
2021-08-04 18:23:08 -07:00
31b5e09ac2
Add detailed RIT info in JSON format for consumption as metrics ( #3555 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-04 18:04:44 -07:00
c11a3e1b39
Revert "HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 ( #3529 )"
...
This reverts commit f049301606
2021-08-04 15:55:13 -07:00
fffdcba5bb
Revert "HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 ( #3543 )"
...
This reverts commit ca096437d7
2021-08-04 15:55:13 -07:00
620806e3fb
HBASE-26150 Let region server also carry ClientMetaService ( #3550 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-05 00:15:49 +08:00
04d3c7a9d0
HBASE-26166 Fix a minor bug of table list on master ui ( #3554 )
...
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
2021-08-04 20:42:15 +08:00
ca096437d7
HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 ( #3543 )
...
2/17 commits of HBASE-22120
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-08-02 11:33:19 -07:00
aa3b07f6bb
HBASE-26154: Adds exception metrics for QuotaExceededException and RpcThrottlingException ( #3544 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2021-08-02 09:51:34 +05:30
f7baa195e3
HBASE-26151 Reimplement MasterAddressTracker to also cache backup master addresses ( #3548 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-31 15:54:06 +08:00
aff880e147
HBASE-26144 The HStore.snapshot method is never called in main code ( #3533 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-31 14:39:13 +08:00
f3c022740a
HBASE-26105 Rectify the expired TODO comment in CombinedBC. ( #3509 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-07-30 21:43:31 +08:00
6df4cb1cf4
HBASE-26114 when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster cannot start normally ( #3541 )
...
Signed-off-by: Anoop <anoopsamjohn@apache.org>
2021-07-30 17:13:50 +05:30
f049301606
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 ( #3529 )
...
1/17 commits of HBASE-22120
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-07-29 09:15:10 -07:00
dd22fecc2b
HBASE-26120 New replication gets stuck or data loss when multiwal groups more than 10 ( #3528 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-07-27 08:14:15 -07:00
20a4aaedcc
HBASE-26118 The HStore.commitFile and HStore.moveFileIntoPlace almost have the same logic ( #3525 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-26 21:47:12 +08:00
37c3f7b446
HBASE-21946 Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable ( #3434 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 4a3c7d73b0
2021-07-26 17:43:10 +08:00
be0a6cea40
HBASE-26119 Polish TestAsyncNonMetaRegionLocator ( #3526 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-07-26 17:34:01 +08:00
59e2825c93
HBASE-26093 Replication is stuck due to zero length wal file in oldWALs directory ( #3504 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-07-23 09:33:43 -07:00
729da5a6d3
HBASE-24734 RegionInfo#containsRange should support check meta table ( #3496 ) ( #3517 )
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2021-07-22 18:04:24 +08:00
8bc180c773
HBASE-24984 WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used with multi operation ( #3512 )
...
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-07-22 08:30:25 +05:30
cabf0dc51d
HBASE-25986 set default value of normalization enabled from hbase site ( #3476 ) ( #3372 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-07-19 15:26:52 +05:30
55d93fbf2d
HBASE-25973 Balancer should explain progress in a better way in log -… ( #3483 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-07-18 13:04:33 -07:00
fb01a6c276
HBASE-26083 In branch-2 & branch-1 L1 miss metric is always 0 when using CombinedBlockCache ( #3473 )
...
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by Reid Chan <reidchan@apache.org>
2021-07-16 10:28:27 +08:00
692d384ec6
HBASE-26036 DBB released too early in HRegion.get() and dirty data for some operations ( #3436 ) ( #3486 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-07-14 16:30:46 +08:00
12f2a16280
HBASE-26086 TestHRegionReplayEvents do not pass in branch-2 and throws NullPointerException ( #3480 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-07-14 11:37:25 +08:00
dfc9ac8ffb
HBASE-25739 TableSkewCostFunction need to use aggregated deviation - backport
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@duboce.net>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
2021-07-13 08:37:29 -07:00
3294d32f46
HBASE-25700 Enhance znode parent validation when add_peer ( #3458 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-07-11 21:15:57 +08:00
b86d2d4020
HBASE-26069 Remove HStore.compactRecentForTestingAssumingDefaultPolic… ( #3462 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-07 21:57:59 +08:00
1ba03cae4b
HBASE-26068 The last assertion in TestHStore.testRefreshStoreFilesNotChanged is wrong ( #3461 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org
2021-07-07 14:39:03 +08:00
bde5d98338
HBASE-26065 StripeStoreFileManager does not need to throw IOException for most methods ( #3459 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-07-05 22:19:17 +08:00
342a478753
HBASE-22923 Consider minVersionToMoveSysTables while moving region and creating regionPlan (ADDENDUM) ( #3455 )
...
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-07-03 22:44:46 +05:30
db75ef265e
HBASE-26050 Remove the reflection used in FSUtils.isInSafeMode ( #3445 )
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 82c44b49f5
2021-07-02 14:21:28 +08:00
1381df4c80
HBASE-22923 min version of RegionServer to move system table regions ( #3439 ) ( #3438 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-07-01 16:24:14 +05:30
75cd25386a
HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade ( #3417 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-07-01 15:20:38 +05:30
b90fbb77c6
HBASE-26028 The view as json page shows exception when using TinyLfuBlockCache ( #3420 )
...
(cherry picked from commit 147b030c1f
2021-07-01 10:27:24 +08:00
748e369629
HBASE-26029 It is not reliable to use nodeDeleted event to track region server's death ( #3430 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-06-30 11:46:09 +08:00
f827a4c5f8
HBASE-26039 TestReplicationKillRS is useless after HBASE-23956 ( #3440 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-06-30 08:27:17 +08:00
166becfd66
HBASE-25980 Master table.jsp pointed at meta throws 500 when no all r… ( #3373 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:30:46 +08:00
c707a9ffff
HBASE-25914 Provide slow/large logs on RegionServer UI ( #3321 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:27:06 +08:00
c7c5d56084
HBASE-26015 Should implement getRegionServers(boolean) method in Asyn… ( #3406 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:13:21 +08:00
193a94e523
HBASE-26013 Get operations readRows metrics becomes zero after HBASE-25677 ( #3411 )
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-06-24 15:24:50 +08:00
9fe1781037
HBASE-26020 Split TestWALEntryStream.testDifferentCounts out ( #3409 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-06-23 22:51:23 +08:00
c81a5ab565
HBASE-25934 Add username for RegionScannerHolder ( #3325 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-06-23 12:54:41 +05:30
2ae69a883c
HBASE-25877 Add access check for compactionSwitch ( #3253 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-22 23:45:36 +08:00
bda7afe97d
HBASE-25937: Clarify UnknownRegionException ( #3330 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-22 22:44:05 +08:00
12d707c880
HBASE-25992 Addendum add missing catch WALEntryFilterRetryableException back
2021-06-21 23:58:42 +08:00
41a68325c2
HBASE-25698 Fixing IllegalReferenceCountException when using TinyLfuBlockCache ( #3215 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-06-21 12:51:59 +05:30
029dc81b39
HBASE-26001 When turn on access control, the cell level TTL of Increment and Append operations is invalid ( #3403 )
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-06-21 15:00:50 +08:00
c800e6c825
HBASE-25992 Polish the ReplicationSourceWALReader code for 2.x after HBASE-25596 ( #3376 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-20 16:38:13 +08:00
3b58919246
HBASE-25998: Redo synchronization in SyncFuture ( #3399 )
...
Currently uses coarse grained synchronized approach that seems to
create a lot of contention. This patch
- Uses a reentrant lock instead of synchronized monitor
- Switches to a condition variable based waiting rather than busy wait
- Removed synchronization for unnecessary fields
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 6bafb59642
2021-06-18 10:37:10 -07:00
836fd3c52a
HBASE-25984: Avoid premature reuse of sync futures in FSHLog ( #3371 ) ( #3392 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 5a19bcfa98
2021-06-17 12:27:23 -07:00
26c4e86ebd
HBASE-25976 Implement a master based ReplicationTracker ( #3390 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-06-17 18:38:43 +08:00
4719a2078e
HBASE-25997 NettyRpcFrameDecoder decode request header wrong when han… ( #3380 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-15 14:39:47 +08:00
6f1e652091
HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update ( #3386 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-15 13:55:21 +09:00
cb5f0dfe1e
HBASE-25995 Change the method name for DoubleArrayCost.setCosts ( #3381 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-15 11:55:54 +08:00
a1d59bf1a5
HBASE-25994 Active WAL tailing fails when WAL value compression is enabled ( #3377 )
...
Depending on which compression codec is used, a short read of the
compressed bytes can cause catastrophic errors that confuse the WAL reader.
This problem can manifest when the reader is actively tailing the WAL for
replication. To avoid these issues when WAL value compression is enabled,
BoundedDelegatingInputStream should assume enough bytes are available to
supply a reader up to its bound. This behavior is valid per the contract
of available(), which provides an _estimate_ of available bytes, and
equivalent to IOUtils.readFully but without requiring an intermediate
buffer.
Added TestReplicationCompressedWAL and TestReplicationValueCompressedWAL.
Without the WALCellCodec change TestReplicationValueCompressedWAL will
fail.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-06-14 17:18:53 -07:00
88f031eeec
HBASE-25967 The readRequestsCount does not calculate when the outResu… ( #3351 )
...
* HBASE-25967 The readRequestsCount does not calculate when the outResults is empty
Co-authored-by: Zheng Wang <wangzheng@apache.org>
(cherry picked from commit 329f0baa98
2021-06-10 10:24:18 +08:00
b6837295f3
HBASE-25981 JVM crash when displaying RegionServer UI ( #3364 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-09 18:16:33 +08:00
e4dd4901a8
HBASE-25963 HBaseCluster should be marked as IA.Public ( #3348 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-06 21:27:02 +08:00
1267bdb1a4
HBASE-25929 RegionServer JVM crash when compaction ( #3318 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-03 17:20:08 +08:00
meiyi
Duo Zhang
Xiaolin Ha
Tak Lon (Stephen) Wu
Nick Dimiduk
Andrew Purtell
Bryan Beaudreault
Rushabh Shah
Istvan Toth
Mike Drob
Niels Basjes
Richard Marscher
Yutong Xiao
chenglei
Peter Somogyi
xianjingfeng
Samir Khan
zhengzhuobinzzb
Norbert Kalmar
Josh Elser
bsglz
Guanghao Zhang
LiangJun He
RamanChodzka
Tomu Tsuruhara
liangxs
ss77892
Zhuoyue Huang
clarax
Xiaolin Ha
XinSun
Anoop Sam John
Sourabh Dhanotia
binlijin
Viraj Jasani
KulwantSingh011
imbajin
GeorryHuang
richardantal
bitterfox
hapihu
Josh Elser
keyserbrian1
Pankaj
lpalasthy
Michael Stack
Wellington Ramos Chevreuil
eomiks
stack
lzx404243
caroliney14
Ruanhui
liever18
Wei-Chiu Chuang
gkanade
Aman Poonia
mokai
YutSean
litao
lujiefsi
belugabehr
Bharath Vissapragada
Toshihiro Suzuki