chenglei
a67fb0a85c
HBASE-26494 Using RefCnt to fix the flawed MemStoreLABImpl ( #3983 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-01-01 20:57:25 +08:00
Duo Zhang
e53712ae99
HBASE-26523 Upgrade hbase-thirdparty dependency to 4.0.1 ( #3988 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2021-12-31 12:10:08 +08:00
meiyi
5b08db92c7
HBASE-26625 ExportSnapshot tool failed to copy data files for tables with merge region ( #3981 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-30 11:08:36 +08:00
Richard Marscher
9890ac9639
HBASE-26623 Report CallDroppedException in exception metrics ( #3980 )
...
`CallDroppedException` can be thrown with `CallRunner.drop()` by queue implementations that decide to drop calls to groom the RPC call backlog. The LifoCoDel queue does this I believe and with Pluggable queue it's possible for 3rd party queue implementations to be using `drop()` for similar reasons. It would be nice for the server to be tracking these exceptions in metrics since otherwise you might have to do some extra lifting on the client side.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2021-12-30 00:05:04 +08:00
Yutong Xiao
c6a287930b
HBASE-26564 Retire the method visitLogEntryBeforeWrite without RegionInfo in WALActionListner ( #3985 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-28 21:42:00 +08:00
meiyi
7fcf8e84a5
HBASE-26615 Snapshot referenced data files are deleted when delete a table with merge regions ( #3976 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2021-12-23 18:51:28 +08:00
Duo Zhang
c6ac99684c
HBASE-26609 Round the size to MB or KB at the end of calculation in HRegionServer.createRegionLoad ( #3967 )
...
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-12-23 11:58:29 +08:00
Samir Khan
55ddcbaaf5
HBASE-26598 Fix excessive connections in MajorCompactor ( #3961 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-23 11:55:21 +08:00
zhengzhuobinzzb
68d8573c09
HBASE-26580 The message of StoreTooBusy is confused ( #3949 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2021-12-18 23:33:58 +08:00
zhengzhuobinzzb
12a286ca66
HBASE-26579 Set storage policy of recovered edits when hbase.wal.storage.type is configured ( #3948 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-12-18 23:27:42 +08:00
chenglei
2d5d5b2011
HBASE-26488 Memory leak when MemStore retry flushing ( #3899 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-18 12:22:42 +08:00
Norbert Kalmar
471e398df0
HBASE-26340 TableSplit returns false size under 1MB ( #3872 )
...
Signed-off-by: Peter Somogyi <psomogyi@apache.org
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-18 12:21:45 +08:00
Nick Dimiduk
9d6823236d
HBASE-26542 Apply a `package` to test protobuf files (addendum)
...
RpcServer identifies the services it hosts by unqualified service name. Thus, use `getName()`
instead of `getFullName()`. See also HBASE-26589.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-12-16 16:39:40 -08:00
Josh Elser
cb3155459f
HBASE-26550 Make sure the master is running normally before accepting a balance command ( #3928 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-14 23:32:24 +08:00
Bryan Beaudreault
ef48a6975d
HBASE-26537: Make HBASE-15676 backwards compatible, using a flag on the proto ( #3931 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-12-13 13:31:14 -08:00
bsglz
26e37bcbfe
HBASE-26027 The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException ( #3925 )
...
* HBASE-26027 The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException
2021-12-10 14:34:17 +08:00
Andrew Purtell
b1bc5f3a5c
Renumber to 2.6.0-SNAPSHOT after branching branch-2.5
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-12-08 16:54:32 -08:00
Guanghao Zhang
140b5d8d26
HBASE-24870 Ignore TestAsyncTableRSCrashPublish ( #2248 )
...
Signed-off-by: meiyi <myimeiyi@gmail.com>
2021-12-06 19:07:07 -08:00
LiangJun He
ec52519f7c
HBASE-26462 Should persist restoreAcl flag in the procedure state for CloneSnapshotProcedure and RestoreSnapshotProcedure ( #3921 )
...
Signed-off-by: Yu Li <liyu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-06 23:16:15 +08:00
RamanChodzka
074456e190
HBASE-26533 KeyValueScanner might not be properly closed when using InternalScan.checkOnlyMemStore() ( #3917 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-05 22:35:07 +08:00
Tomu Tsuruhara
19d991d291
HBASE-26517 Add auth method information to AccessChecker audit log ( #3897 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-04 23:14:42 +08:00
Bryan Beaudreault
d504504875
HBASE-26304 Reflect out of band locality improvements in metrics and balancer ( #3895 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-04 12:23:09 +08:00
Nick Dimiduk
d1c68a2d5c
HBASE-26471 Move tracing semantic attributes to their own class ( #3896 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
2021-12-01 13:24:08 -08:00
Rushabh Shah
6fc5bbc19a
HBASE-26468 Region Server doesn't exit cleanly incase it crashes. ( #3865 ) ( #3862 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-12-01 11:06:07 +05:30
Duo Zhang
774484ed44
HBASE-25905 Shutdown of WAL stuck at waitForSafePoint ( #3898 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-12-01 08:34:08 +08:00
Duo Zhang
799217ee69
HBASE-26455 TestStochasticLoadBalancerRegionReplicaWithRacks fails consistently ( #3879 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2021-11-28 22:23:21 +08:00
chenglei
5437532910
HBASE-26476 Make DefaultMemStore extensible for HStore.memstore ( #3869 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-27 12:06:35 +08:00
chenglei
5101f372e4
HBASE-26465 MemStoreLAB may be released early when its SegmentScanner is scanning ( #3859 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Anoop Sam John <anoopsamjohn@apache.org>
2021-11-25 21:35:55 +08:00
Yutong Xiao
6aa2daa2c2
HBASE-26477 Fix MiniMapReduceCluster failure under Java17 ( #3878 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-25 16:27:23 +08:00
Duo Zhang
fa34cfc7d1
Revert "Backport HBASE-26477 Fix MiniMapReduceCluster failure under Java17 to branch-2 ( #3878 )"
...
This reverts commit ccf150434d
.
2021-11-25 16:27:11 +08:00
Yutong Xiao
ccf150434d
Backport HBASE-26477 Fix MiniMapReduceCluster failure under Java17 to branch-2 ( #3878 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-25 16:25:44 +08:00
liangxs
2798ef132b
HBASE-26479 Print too slow/big scan's operation_id in region server log ( #3870 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-25 11:43:01 +08:00
Andrew Purtell
52ce87780a
Revert "HBASE-26479 Print too slow/big scan's operation_id in region server log ( #3870 )"
...
This reverts commit b437a96272
.
2021-11-24 17:34:01 -05:00
liangxs
b437a96272
HBASE-26479 Print too slow/big scan's operation_id in region server log ( #3870 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-24 23:38:16 +08:00
Duo Zhang
0a51d2b5be
HBASE-26475 The flush and compact methods in HTU should skip processing secondary replicas ( #3868 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-11-23 18:51:25 +08:00
Xiaolin Ha
f8d03d1227
HBASE-26249 Ameliorate compaction made by bulk-loading files ( #3831 ) ( #3874 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-23 15:52:47 +08:00
haxiaolin
61406252a5
Revert "Backport HBASE-26249 Ameliorate compaction made by bulk-loading files ( #3831 ) ( #3874 )"
...
This reverts commit 5843a9a44c
.
2021-11-23 15:52:34 +08:00
Xiaolin Ha
5843a9a44c
Backport HBASE-26249 Ameliorate compaction made by bulk-loading files ( #3831 ) ( #3874 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-23 15:45:21 +08:00
zhengzhuobinzzb
4312f817f3
HBASE-26467 Fix bug for MemStoreLABImpl.forceCopyOfBigCellInto(Cell) ( #3858 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: chenglei <chenglei@apache.org>
2021-11-21 21:02:18 +08:00
ss77892
e549384437
HBASE-26463 Unreadable table names after HBASE-24605 ( #3853 )
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-21 20:49:13 +08:00
Xiaolin Ha
8d3562c6fa
HBASE-26421 Use HFileLink file to replace entire file's reference when splitting ( #3854 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-18 21:37:10 +08:00
Josh Elser
4a8da47f74
HBASE-26267 Don't try to recover WALs from a WAL dir which doesn't exist ( #3679 )
...
We currently cause an error to be thrown by trying to list a
non-existent directory. We see that the master region directory exists
on the filesystem, but forget to make sure that the master region's WAL
directory also exists before we try to list it.
2021-11-16 16:27:45 -05:00
Andrew Purtell
e65f28cc57
HBASE-26444 BucketCacheWriter should log only the BucketAllocatorException message, not the full stack trace ( #3840 )
...
Signed-off-by: Anoop <anoopsamjohn@apache.org>
2021-11-15 16:13:36 -08:00
Andrew Purtell
23024edba2
HBASE-26443 Some BaseLoadBalancer log lines should be at DEBUG level ( #3838 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-11-15 16:06:59 -08:00
chenglei
98a4fb8086
HBASE-26438 Fix flaky test TestHStore.testCompactingMemStoreCellExceedInmemoryFlushSize ( #3834 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-12 23:03:37 +08:00
Zhuoyue Huang
ad36941932
HBASE-26450 Server configuration will overwrite HStore configuration after using shell command 'update_config' ( #3843 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
2021-11-12 11:43:48 +08:00
Yutong Xiao
9db7630d23
HBASE-26410 Fix HBase TestCanaryTool for Java17 ( #3809 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-09 22:21:01 +08:00
clarax
cc2e4aec87
HBASE-26337 Optimization for weighted random generators ( #3828 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-09 21:27:44 +08:00
Andrew Purtell
1302073dfe
HBASE-26430 Increase log level in DefaultHeapMemoryTuner from DEBUG to INFO ( #3820 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-11-08 18:35:39 -08:00
Andrew Purtell
201b3c7fc7
HBASE-26429 HeapMemoryManager fails memstore flushes with NPE if enabled ( #3819 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-11-08 18:32:17 -08:00
Xiaolin Ha
2fbb375cd5
HBASE-25322 Redundant Reference file in bottom region of split ( #3814 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-08 14:48:55 +08:00
haxiaolin
ebd89b58eb
Revert "HBASE-25322 Redundant Reference file in bottom region of split ( #3814 )"
...
This reverts commit a6edf44175
.
2021-11-08 11:40:43 +08:00
Xiaolin Ha
a6edf44175
HBASE-25322 Redundant Reference file in bottom region of split ( #3814 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-05 18:17:47 +08:00
clarax
e06208f3d6
HBASE-26311 Balancer gets stuck in cohosted replica distribution ( #3804 )
...
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-11-03 11:58:38 -07:00
clarax
9eb3706021
HBASE-26309 Balancer tends to move regions to the server at the end of list ( #3811 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-02 21:58:50 +08:00
XinSun
063e0e2e92
HBASE-26406 Can not add peer replicating to non-HBase ( #3806 )
...
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b9b7fec57f
)
2021-11-02 15:03:33 +08:00
chenglei
e89f62e611
HBASE-26384 Segment already flushed to hfile may still be remained in CompactingMemStore ( #3779 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-01 23:05:57 +08:00
Andrew Purtell
f2c58fcf68
HBASE-26353 Support loadable dictionaries in hbase-compression-zstd ( #3787 )
...
ZStandard supports initialization of compressors and decompressors with a
precomputed dictionary, which can dramatically improve and speed up compression
of tables with small values. For more details, please see
The Case For Small Data Compression
https://github.com/facebook/zstd#the-case-for-small-data-compression
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdCodec.java
2021-10-27 07:57:16 -07:00
chenglei
b4177577ac
HBASE-26394 Cache in RSRpcServices.executeProcedures does not take effect ( #3794 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-10-26 15:44:17 +08:00
clarax
32eb23c8f3
HBASE-26327 Replicas cohosted on a rack shouldn't keep triggering balancer ( #3788 )
...
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-10-25 15:44:11 -07:00
Anoop Sam John
444dec9289
HBASE-26190 High rate logging of BucketAllocatorException: Allocation too big - Addendum ( #3792 )
2021-10-24 09:22:26 +05:30
clarax
f299d56934
HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger ( #3783 )
2021-10-22 09:46:58 -07:00
Sourabh Dhanotia
eea2ea8576
HBASE-26368 Fix checkstyle issues for HRegionServer for branch-2 ( #3781 ) ( #3764 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-10-22 12:51:29 +05:30
binlijin
0eaadd92d8
HBASE-26385 Clear CellScanner when replay ( #3773 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-10-21 21:56:39 +08:00
Andrew Purtell
dce45ec16a
HBASE-26383 HBCK incorrectly reports inconsistencies for recently split regions following a master failover ( #3775 )
...
A regression was introduced by HBASE-25847 which changed regionInfo#isParentSplit
to regionState#isSplit. The region state after restart is CLOSED instead of SPLIT.
We need to check both regionState and regionInfo for split status.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-10-20 15:55:07 -07:00
Viraj Jasani
5c4a053f2a
HBASE-26371 Prioritize meta region move over other region moves in region_mover ( #3771 ) ( #3767 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-10-20 23:06:25 +05:30
Andrew Purtell
b6bb18022f
HBASE-26316 Per-table or per-CF compression codec setting overrides ( #3730 )
...
We get and retain Compressor instances in HFileBlockDefaultEncodingContext,
and could in theory call Compressor#reinit when setting up the context,
to update compression parameters like level and buffer size, but we do
not plumb through the CompoundConfiguration from the Store into the
encoding context. As a consequence we can only update codec parameters
globally in system site conf files.
Fine grained configurability is important for algorithms like ZStandard
(ZSTD), which offers more than 20 compression levels, where at level 1
it is almost as fast as LZ4, and where at higher levels it utilizes
computationally expensive techniques to rival LZMA at compression ratio
but trades off significantly for reduced compresson throughput. The ZSTD
level that should be set for a given column family or table will vary by
use case.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdDecompressor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/compress/HFileTestBase.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestDataBlockEncoders.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestSeekToBlockWithEncoders.java
2021-10-19 13:59:04 -07:00
KulwantSingh011
7e574c9f1d
HBASE-26190 High rate logging of BucketAllocatorException: Allocation too big ( #3752 )
...
Reduce the frequency of allocation failed traces by printing them
preiodically (once per minute). Record the allocation failures in the
Bucket Cache Stats and let the stat thread dump cumulative allocation
failures alongside other traces it dumps.
Also, this change adds trace for the Table name, Column Family and
HFileName for the most recent allocation failure in last 1 minute.
Signed-off-by: Anoop <anoopsamjohn@apache.org>
2021-10-19 13:21:11 +05:30
Josh Elser
45b09aa4d7
HBASE-26350 Add a DEBUG when we fail the SASL handshake
...
Closes #3743
Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-10-14 15:27:32 -04:00
Tak Lon (Stephen) Wu
3ce9052188
HBASE-24833: Bootstrap should not delete the META table directory if … ( #2237 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
2021-10-12 10:56:00 -07:00
Duo Zhang
10584d70d2
HBASE-26306 Backport "HBASE-26220 Use P2P communicate between region servers to sync the list for bootstrap node" to branch-2 ( #3727 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-10-10 22:56:12 +08:00
Andrew Purtell
5af02201d2
HBASE-26324 Reuse compressors and decompressors in WAL CompressionContext ( #3728 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-10-07 09:46:38 -07:00
Andrew Purtell
18b9fa8a3c
HBASE-26259 Fallback support to pure Java compression ( #3691 )
...
This change introduces provided compression codecs to HBase as
new Maven modules. Each module provides compression codec support
that formerly required Hadoop native codecs, which in turn relies
on native code integration, which may or may not be available on
a given hardware platform or in an operational environment. We
now provide codecs in the HBase distribution for users whom for
whatever reason cannot or do not wish to deploy the Hadoop native
codecs.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileEncryption.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestCompressedWAL.java
2021-10-06 13:48:33 -07:00
Duo Zhang
706082d513
HBASE-26294 Backport "HBASE-26181 Region server and master could use itself as ConnectionRegistry" to branch-2 ( #3708 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-10-05 22:11:01 +08:00
imbajin
9a33e234e7
HBASE-24601: Change default Hfile storage policy from HOT to NONE for HDFS ( #1940 )
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-10-02 16:37:39 +08:00
Yutong Xiao
a2c09ae5bc
HBASE-26305 Move NavigableSet add operation to writer thread in BucketCache ( #3705 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: chenglei <chenglei@apache.org>
2021-10-01 21:06:57 +08:00
chenglei
0748165ab0
HBASE-26295 BucketCache could not free BucketEntry which restored fro… ( #3699 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-29 21:46:57 +08:00
GeorryHuang
08800e19d6
HBASE-26251 StochasticLoadBalancer metrics should update even if balancer doesn't run ( #3706 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2021-09-29 21:25:39 +08:00
richardantal
69749ed01e
HBASE-26289 Hbase scan setMaxResultsPerColumnFamily not giving right results ( #3688 )
...
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-29 10:22:24 +02:00
Duo Zhang
08a2efde16
HBASE-26293 Use reservoir sampling when selecting bootstrap nodes ( #3702 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-09-28 23:41:28 +08:00
bitterfox
786e09a936
HBASE-26238 Short message by Result#compareResults for VerifyReplication to avoid OOME ( #3647 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-28 23:16:20 +08:00
clarax
73f42b27bf
HBASE-26297 Balancer run is improperly triggered by accuracy error of double comparison ( #3698 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-26 23:35:50 +08:00
clarax
404789adde
HBASE-26178 Improve data structure and algorithm for BalanceClusterState to improve computation speed for large cluster ( #3682 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-25 21:48:51 +08:00
Andrew Purtell
59d133ef95
Amend HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job
...
Compliation fixes for CombinedBlockCache and TestClientSideRegionScanner.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-09-22 12:27:59 -07:00
Tak Lon (Stephen) Wu
e0325c40f6
HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job ( #3684 )
...
Introduce `hfile.onheap.block.cache.fixed.size`
and default to disable. when using ClientSideRegionScanner
it will be enabled with a fixed size for caching
INDEX/LEAF_INDEX block when a client, e.g.
snapshot scanner, scans the entire HFile
and does not need to seek/reseek to index
block multiple times.
Signed-off-by: Josh Elser <elserj@apache.org>
2021-09-22 09:38:20 -07:00
meiyi
84b5511d13
HBASE-26261 Store configuration loss when use update_config ( #3664 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-22 18:35:28 +08:00
meiyi
8f6c5e1f02
HBASE-26270 Provide getConfiguration method for Region and Store interface ( #3671 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2021-09-22 11:32:32 +08:00
Duo Zhang
e327fa43c0
HBASE-26288 Revisit the usage of MetaTableLocator when HRegionServer.TEST_SKIP_REPORTING_TRANSITION is true ( #3689 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-09-22 00:27:11 +08:00
Duo Zhang
ef85204522
HBASE-26285 Remove MetaTableLocator usages in non-migration code ( #3686 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-09-21 23:32:13 +08:00
chenglei
b21b250570
HBASE-26281 DBB got from BucketCache would be freed unexpectedly before RPC completed ( #3680 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-17 22:28:56 +08:00
Xiaolin Ha
f670fc5d4d
HBASE-26229 Limit count and size of L0 files compaction in StripeCompactionPolicy ( #3646 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-09 18:28:45 +08:00
hapihu
68db7f5ed0
[HBASE-26243]Fix typo for file "hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java" ( #3650 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-09 12:21:54 +08:00
Rushabh Shah
820ba21b2a
HBASE-26106 AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory. ( #3636 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
2021-09-02 10:26:22 -07:00
XinSun
ccd90269d2
HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky ( #3651 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 345d7256c8
)
2021-09-02 10:33:41 +08:00
Bryan Beaudreault
e07928df4c
HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
(Re-application of HBASE-26147 with the correct author metadata)
2021-09-01 21:58:30 -04:00
Josh Elser
1e74f08403
Revert "HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed"
...
This reverts commit 5b58d11c91
.
I accidentally applied this as myself instead of as Bryan. Reverting it
to apply the patch properly.
2021-09-01 21:57:46 -04:00
Josh Elser
5b58d11c91
HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org
2021-09-01 21:47:53 -04:00
Tak Lon (Stephen) Wu
c639d6f353
HBASE-26136 Backport HBASE-25723 "Temporarily remove the trace support for RegionScanner.next" to branch-2 ( #3623 )
...
13/17 commits of HBASE-22120, original commit 7f90c2201f
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu
d0a53e3f29
HBASE-26133 Backport HBASE-25591 "Upgrade opentelemetry to 0.17.1" to branch-2 ( #3608 )
...
10/17 commits of HBASE-22120, original commit f6ff519dd0
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu
8b7450f5d8
HBASE-26132 Backport HBASE-25535 "Set span kind to CLIENT in AbstractRpcClient" to branch-2 ( #3607 )
...
9/17 commits of HBASE-22120, original commit bb8c4967f8
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu
9cce94a2d0
HBASE-26131 Backport HBASE-25484 "Add trace support for WAL sync" to branch-2 ( #3597 )
...
8/17 commits of HBASE-22120, original commit 2be2c63f0d
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu
321dcd4e77
HBASE-26130 Backport HBASE-25455 "Add trace support for HRegion read/… ( #3594 )
...
7/17 commits of HBASE-22120, original commit 03e12bfa4a
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu
83a27389a0
HBASE-26129 Backport HBASE-25481 "Add host and port attribute when tracing rpc call at client side" to branch-2 ( #3589 )
...
6/17 commits of HBASE-22120, original commit ae2c62ffaa
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu
58b5c007b9
HBASE-26127 Backport HBASE-23898 "Add trace support for simple apis i… ( #3556 )
...
4/17 commits of HBASE-22120, original commit 805b2ae2ad
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu
4dfc59c19e
HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 ( #3543 )
...
2/17 commits of HBASE-22120
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu
665305cc3b
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 ( #3529 )
...
1/17 commits of HBASE-22120
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
chenglei
dbf43dcfbd
HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore ( #3604 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 18:17:13 +08:00
Andrew Purtell
8be2ad2d42
HBASE-25642 Fix or stop warning about already cached block ( #3638 )
...
Our logs have as a fairly common occurrence: 2021-03-05 22:24:31,034 WARN
[StoreFileOpener-foo-1] hfile.BlockCacheUtil: Caching an already cached
block: blah.bub. This is harmless and can happen in rare cases (see HBASE-8547)
Because it is harmless, log at DEBUG level, not WARN.
Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
2021-08-31 09:26:10 -07:00
Duo Zhang
c08e877d6a
HBASE-26235 We could start RegionServerTracker before becoming active master ( #3645 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-08-30 12:22:41 +08:00
keyserbrian1
1e49e94051
HBASE-26232 SEEK_NEXT_USING_HINT is ignored on reversed Scans ( #3639 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-29 21:42:48 +08:00
GeorryHuang
033ff757fd
HBASE-26216 Move HRegionServer.abort(String) to Abortable as a default method ( #3642 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-28 23:06:54 +08:00
Andrew Purtell
3b1482ef60
HBASE-26189 Reduce log level of CompactionProgress notice to DEBUG ( #3579 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-08-27 10:15:28 -07:00
Pankaj
30bb72bdb8
HBASE-26227 Forward port HBASE-26223 test code to branch-2.4+ ( #3632 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-27 16:46:54 +08:00
lpalasthy
cd9e45ce8d
HBASE-26219 Negative time is logged while waiting on regionservers ( #3622 )
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-25 16:25:24 +02:00
Duo Zhang
931a7a5a61
HBASE-26180 Introduce a initial refresh interval for RpcConnectionRegistry ( #3601 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-08-25 18:51:40 +08:00
GeorryHuang
dacd3404a4
HBASE-26215 The backup master status page should use ActiveMasterManager instead of MasterAddressTracker ( #3621 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-25 16:42:45 +08:00
Duo Zhang
5dad395869
HBASE-26173 Return only a sub set of region servers as bootstrap nodes ( #3599 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-24 23:17:56 +08:00
Duo Zhang
6ad7eb88c3
HBASE-26172 Deprecated MasterRegistry ( #3566 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-19 16:30:48 +08:00
Michael Stack
101602007a
HBASE-24337 Backport HBASE-23968 to branch-2 ( #3588 )
...
Signed-off-by: stack <stack@apache.org>
2021-08-18 10:19:35 -07:00
Wellington Ramos Chevreuil
f2f99aea34
Amend HBASE-26187 UTs, adding a check for Split procedure completion before proceeding with test logic. ( #3598 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f2e2140b57
)
2021-08-18 16:41:08 +01:00
Wellington Ramos Chevreuil
30c20d7627
HBASE-26187 Write straight into the store directory when Splitting an… ( #3574 ) ( #3595 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-18 09:42:03 +01:00
Xiaolin Ha
4ccd215354
HBASE-26087 JVM crash when displaying RPC params by MonitoredRPCHandler ( #3489 )
...
Signed-off-by: stack <stack@apache.org>
2021-08-18 14:39:28 +08:00
Duo Zhang
d99ca37307
HBASE-26193 Do not store meta region location as permanent state on zookeeper ( #3583 )
...
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
2021-08-17 18:58:39 +08:00
eomiks
5eac407944
HBASE-24652 master-status UI make date type fields sortable
2021-08-16 16:03:29 -07:00
stack
63c57d7476
HBASE-26200 Revert "HBASE-25165 Change 'State time' in UI so sorts ( #2508 )"
...
This reverts commit d07d181ea4
.
2021-08-16 13:49:55 -07:00
lzx404243
4aec407c02
HBASE-25680 Non-idempotent test in TestReplicationHFileCleaner ( #2984 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-14 11:11:34 +08:00
chenglei
725474bbe3
HBASE-26026 HBase Write may be stuck forever when using CompactingMemStore ( #3421 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-12 23:19:32 +08:00
Xiaolin Ha
e8ecafb22b
HBASE-26155 JVM crash when scan ( #3553 )
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-12 17:17:03 +08:00
Rushabh Shah
6b9114d95e
HBASE-26185 Return mutable list in AssignmentManager#getExcludedServersForSystemTable ( #3577 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-08-12 13:47:52 +05:30
stack
243e861985
Revert "HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned ( #3532 )"
...
This reverts commit 8f16e34eb2
.
2021-08-11 07:41:22 -07:00
Bryan Beaudreault
8f16e34eb2
HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned ( #3532 )
...
Signed-off-by: stack <stack@apache.org>
2021-08-10 20:38:06 -07:00
Viraj Jasani
40a7a75a19
HBASE-26185 Fix TestMaster#testMoveRegionWhenNotInitialized in the presence of hbase.min.version.move.system.tables ( #3573 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-08-10 10:58:31 +05:30
Richard Marscher
79a6dc6bfd
HBASE-6908: Add pluggable rpc queue implementation ( #3522 )
...
Can pass in a FQCN to load as the call queue implementation.
Standardized arguments to the constructor are the max queue length, the
PriorityFunction, and the Configuration.
`PluggableBlockingQueue` abstract class provided to help guide the
correct constructor signature
Hard fails if the class fails to load as a `BlockingQueue<CallRunner>`
Signed-off-by: stack <stack@apache.org>
2021-08-09 12:55:27 -07:00
Duo Zhang
611ffa7da9
HBASE-26179 TestRequestTooBigException spends too much time to finish ( #3571 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-08-07 23:05:53 +08:00
Xiaolin Ha
61fbe6f480
HBASE-26170 handleTooBigRequest in NettyRpcServer didn't skip enough bytes ( #3564 )
...
Signed-off-by: stack <stack@apache.com>
2021-08-07 23:05:27 +08:00
chenglei
4f0f519f9f
HBASE-26142 NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero ( #3531 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-05 21:04:42 +08:00
caroliney14
effde97a88
HBASE-25469 Add detailed RIT info in JSON format for consumption as metrics ( #3555 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-04 18:23:17 -07:00
Andrew Purtell
8e9253edd8
Revert "Add detailed RIT info in JSON format for consumption as metrics ( #3555 )"
...
Commit is missing JIRA identifier.
This reverts commit 31b5e09ac2
.
2021-08-04 18:23:08 -07:00
caroliney14
31b5e09ac2
Add detailed RIT info in JSON format for consumption as metrics ( #3555 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-04 18:04:44 -07:00
Tak Lon (Stephen) Wu
c11a3e1b39
Revert "HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 ( #3529 )"
...
This reverts commit f049301606
.
2021-08-04 15:55:13 -07:00
Tak Lon (Stephen) Wu
fffdcba5bb
Revert "HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 ( #3543 )"
...
This reverts commit ca096437d7
.
2021-08-04 15:55:13 -07:00
Duo Zhang
620806e3fb
HBASE-26150 Let region server also carry ClientMetaService ( #3550 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-05 00:15:49 +08:00
Ruanhui
04d3c7a9d0
HBASE-26166 Fix a minor bug of table list on master ui ( #3554 )
...
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
2021-08-04 20:42:15 +08:00
Tak Lon (Stephen) Wu
ca096437d7
HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 ( #3543 )
...
2/17 commits of HBASE-22120
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-08-02 11:33:19 -07:00
Bryan Beaudreault
aa3b07f6bb
HBASE-26154: Adds exception metrics for QuotaExceededException and RpcThrottlingException ( #3544 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2021-08-02 09:51:34 +05:30
Duo Zhang
f7baa195e3
HBASE-26151 Reimplement MasterAddressTracker to also cache backup master addresses ( #3548 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-31 15:54:06 +08:00
Duo Zhang
aff880e147
HBASE-26144 The HStore.snapshot method is never called in main code ( #3533 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-31 14:39:13 +08:00
YutSean
f3c022740a
HBASE-26105 Rectify the expired TODO comment in CombinedBC. ( #3509 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-07-30 21:43:31 +08:00
liever18
6df4cb1cf4
HBASE-26114 when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster cannot start normally ( #3541 )
...
Signed-off-by: Anoop <anoopsamjohn@apache.org>
2021-07-30 17:13:50 +05:30
Tak Lon (Stephen) Wu
f049301606
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 ( #3529 )
...
1/17 commits of HBASE-22120
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-07-29 09:15:10 -07:00
Duo Zhang
dd22fecc2b
HBASE-26120 New replication gets stuck or data loss when multiwal groups more than 10 ( #3528 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-07-27 08:14:15 -07:00
Duo Zhang
20a4aaedcc
HBASE-26118 The HStore.commitFile and HStore.moveFileIntoPlace almost have the same logic ( #3525 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-26 21:47:12 +08:00
Wei-Chiu Chuang
37c3f7b446
HBASE-21946 Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable ( #3434 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 4a3c7d73b0
)
2021-07-26 17:43:10 +08:00
Duo Zhang
be0a6cea40
HBASE-26119 Polish TestAsyncNonMetaRegionLocator ( #3526 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-07-26 17:34:01 +08:00
Rushabh Shah
59e2825c93
HBASE-26093 Replication is stuck due to zero length wal file in oldWALs directory ( #3504 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-07-23 09:33:43 -07:00
meiyi
729da5a6d3
HBASE-24734 RegionInfo#containsRange should support check meta table ( #3496 ) ( #3517 )
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2021-07-22 18:04:24 +08:00
gkanade
8bc180c773
HBASE-24984 WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used with multi operation ( #3512 )
...
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-07-22 08:30:25 +05:30
Aman Poonia
cabf0dc51d
HBASE-25986 set default value of normalization enabled from hbase site ( #3476 ) ( #3372 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-07-19 15:26:52 +05:30
clarax
55d93fbf2d
HBASE-25973 Balancer should explain progress in a better way in log -… ( #3483 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-07-18 13:04:33 -07:00
YutSean
fb01a6c276
HBASE-26083 In branch-2 & branch-1 L1 miss metric is always 0 when using CombinedBlockCache ( #3473 )
...
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by Reid Chan <reidchan@apache.org>
2021-07-16 10:28:27 +08:00
Xiaolin Ha
692d384ec6
HBASE-26036 DBB released too early in HRegion.get() and dirty data for some operations ( #3436 ) ( #3486 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-07-14 16:30:46 +08:00
binlijin
12f2a16280
HBASE-26086 TestHRegionReplayEvents do not pass in branch-2 and throws NullPointerException ( #3480 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-07-14 11:37:25 +08:00
Clara Xiong
dfc9ac8ffb
HBASE-25739 TableSkewCostFunction need to use aggregated deviation - backport
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@duboce.net>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
2021-07-13 08:37:29 -07:00
mokai
3294d32f46
HBASE-25700 Enhance znode parent validation when add_peer ( #3458 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-07-11 21:15:57 +08:00
Duo Zhang
b86d2d4020
HBASE-26069 Remove HStore.compactRecentForTestingAssumingDefaultPolic… ( #3462 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-07 21:57:59 +08:00
Duo Zhang
1ba03cae4b
HBASE-26068 The last assertion in TestHStore.testRefreshStoreFilesNotChanged is wrong ( #3461 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org
2021-07-07 14:39:03 +08:00
Duo Zhang
bde5d98338
HBASE-26065 StripeStoreFileManager does not need to throw IOException for most methods ( #3459 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-07-05 22:19:17 +08:00
Viraj Jasani
342a478753
HBASE-22923 Consider minVersionToMoveSysTables while moving region and creating regionPlan (ADDENDUM) ( #3455 )
...
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-07-03 22:44:46 +05:30
Wei-Chiu Chuang
db75ef265e
HBASE-26050 Remove the reflection used in FSUtils.isInSafeMode ( #3445 )
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 82c44b49f5
)
2021-07-02 14:21:28 +08:00
Viraj Jasani
1381df4c80
HBASE-22923 min version of RegionServer to move system table regions ( #3439 ) ( #3438 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-07-01 16:24:14 +05:30
Viraj Jasani
75cd25386a
HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade ( #3417 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-07-01 15:20:38 +05:30
bsglz
b90fbb77c6
HBASE-26028 The view as json page shows exception when using TinyLfuBlockCache ( #3420 )
...
(cherry picked from commit 147b030c1f
)
2021-07-01 10:27:24 +08:00
Duo Zhang
748e369629
HBASE-26029 It is not reliable to use nodeDeleted event to track region server's death ( #3430 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-06-30 11:46:09 +08:00
Duo Zhang
f827a4c5f8
HBASE-26039 TestReplicationKillRS is useless after HBASE-23956 ( #3440 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-06-30 08:27:17 +08:00
GeorryHuang
166becfd66
HBASE-25980 Master table.jsp pointed at meta throws 500 when no all r… ( #3373 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:30:46 +08:00
GeorryHuang
c707a9ffff
HBASE-25914 Provide slow/large logs on RegionServer UI ( #3321 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:27:06 +08:00
GeorryHuang
c7c5d56084
HBASE-26015 Should implement getRegionServers(boolean) method in Asyn… ( #3406 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:13:21 +08:00
YutSean
193a94e523
HBASE-26013 Get operations readRows metrics becomes zero after HBASE-25677 ( #3411 )
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-06-24 15:24:50 +08:00
Duo Zhang
9fe1781037
HBASE-26020 Split TestWALEntryStream.testDifferentCounts out ( #3409 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-06-23 22:51:23 +08:00
litao
c81a5ab565
HBASE-25934 Add username for RegionScannerHolder ( #3325 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-06-23 12:54:41 +05:30
lujiefsi
2ae69a883c
HBASE-25877 Add access check for compactionSwitch ( #3253 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-22 23:45:36 +08:00
belugabehr
bda7afe97d
HBASE-25937: Clarify UnknownRegionException ( #3330 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-22 22:44:05 +08:00
Duo Zhang
12d707c880
HBASE-25992 Addendum add missing catch WALEntryFilterRetryableException back
2021-06-21 23:58:42 +08:00
Viraj Jasani
41a68325c2
HBASE-25698 Fixing IllegalReferenceCountException when using TinyLfuBlockCache ( #3215 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-06-21 12:51:59 +05:30
YutSean
029dc81b39
HBASE-26001 When turn on access control, the cell level TTL of Increment and Append operations is invalid ( #3403 )
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-06-21 15:00:50 +08:00
Duo Zhang
c800e6c825
HBASE-25992 Polish the ReplicationSourceWALReader code for 2.x after HBASE-25596 ( #3376 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-20 16:38:13 +08:00
Bharath Vissapragada
3b58919246
HBASE-25998: Redo synchronization in SyncFuture ( #3399 )
...
Currently uses coarse grained synchronized approach that seems to
create a lot of contention. This patch
- Uses a reentrant lock instead of synchronized monitor
- Switches to a condition variable based waiting rather than busy wait
- Removed synchronization for unnecessary fields
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 6bafb59642
)
2021-06-18 10:37:10 -07:00
Bharath Vissapragada
836fd3c52a
HBASE-25984: Avoid premature reuse of sync futures in FSHLog ( #3371 ) ( #3392 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 5a19bcfa98
)
2021-06-17 12:27:23 -07:00
Duo Zhang
26c4e86ebd
HBASE-25976 Implement a master based ReplicationTracker ( #3390 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-06-17 18:38:43 +08:00
binlijin
4719a2078e
HBASE-25997 NettyRpcFrameDecoder decode request header wrong when han… ( #3380 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-15 14:39:47 +08:00
Toshihiro Suzuki
6f1e652091
HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update ( #3386 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-15 13:55:21 +09:00
Duo Zhang
cb5f0dfe1e
HBASE-25995 Change the method name for DoubleArrayCost.setCosts ( #3381 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-15 11:55:54 +08:00
Andrew Purtell
a1d59bf1a5
HBASE-25994 Active WAL tailing fails when WAL value compression is enabled ( #3377 )
...
Depending on which compression codec is used, a short read of the
compressed bytes can cause catastrophic errors that confuse the WAL reader.
This problem can manifest when the reader is actively tailing the WAL for
replication. To avoid these issues when WAL value compression is enabled,
BoundedDelegatingInputStream should assume enough bytes are available to
supply a reader up to its bound. This behavior is valid per the contract
of available(), which provides an _estimate_ of available bytes, and
equivalent to IOUtils.readFully but without requiring an intermediate
buffer.
Added TestReplicationCompressedWAL and TestReplicationValueCompressedWAL.
Without the WALCellCodec change TestReplicationValueCompressedWAL will
fail.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-06-14 17:18:53 -07:00
bsglz
88f031eeec
HBASE-25967 The readRequestsCount does not calculate when the outResu… ( #3351 )
...
* HBASE-25967 The readRequestsCount does not calculate when the outResults is empty
Co-authored-by: Zheng Wang <wangzheng@apache.org>
(cherry picked from commit 329f0baa98
)
2021-06-10 10:24:18 +08:00
Xiaolin Ha
b6837295f3
HBASE-25981 JVM crash when displaying RegionServer UI ( #3364 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-09 18:16:33 +08:00
Duo Zhang
e4dd4901a8
HBASE-25963 HBaseCluster should be marked as IA.Public ( #3348 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-06 21:27:02 +08:00
meiyi
1267bdb1a4
HBASE-25929 RegionServer JVM crash when compaction ( #3318 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-03 17:20:08 +08:00
Andrew Purtell
a4e8ee183e
HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime ( #3302 )
...
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBase.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupSystemTable.java
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTracing.java
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestManyRegions.java
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/MoveRegionsOfTableAction.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationCrossDiffHdfs.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestEnableRSGroups.java
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractWALRoller.java
hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaUpdatesGoToPriorityQueue.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/RestoreSnapshotFromClientSimpleTestBase.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnection.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClientCloneLinksAfterDelete.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMutationGetCellBuilder.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotMetadata.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetricsWrapper.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaAssignmentWithStopMaster.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionStateStore.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/janitor/TestCatalogJanitor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureSchedulerPerformanceEvaluation.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCloneSnapshotProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRestoreSnapshotProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/procedure2/store/region/RegionProcedureStorePerformanceEvaluation.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMajorCompaction.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionOpen.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSimpleTimeRangeTracker.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCloseChecker.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/master/TestRecoverStandbyProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
2021-06-01 12:41:15 -07:00
Bharath Vissapragada
d4285be5c1
HBASE-25932: Ensure replication reads the trailer bytes from WAL. ( #3332 ) ( #3345 )
...
This bug was exposed by the test from HBASE-25924. Since this wal
implementations close the wal asynchronously, replication can potentially
miss the trailer bytes. (see jira comment for detailed analysis).
While this is not a correctness problem (since trailer does not have any entry data),
it erroneously bumps a metric that is used to track skipped bytes in WAL resulting
in false alarms which is something we should avoid.
Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
(cherry picked from commit b04c3c7786
)
2021-06-01 08:04:12 -07:00
Michael Stack
1a45b743a0
HBASE-25940 Update Compression/TestCompressionTest: LZ4, SNAPPY, LZO ( #3334 )
...
Undo asserts that LZ4 and SNAPPY fails if their native libs are NOT
loaded; as of hadoop 3.3.1, LZ4 and SNAPPY can work w/o native libs.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-29 09:04:08 -07:00
Sandeep Pal
5157521213
HBASE-25927: Fix the log messages by not stringifying the exceptions in log ( #3338 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 9a2027bf71
)
2021-05-28 21:34:14 -07:00
Duo Zhang
2a84fd40d3
HBASE-25938 The SnapshotOfRegionAssignmentFromMeta.initialize call in FavoredNodeLoadBalancer is just a dummy one ( #3329 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-29 11:10:52 +08:00
Baiqiang Zhao
bbe9558040
HBASE-25745 Deprecate/Rename config `hbase.normalizer.min.region.count` to `hbase.normalizer.merge.min.region.count`
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-28 13:10:20 -07:00
Victor
5aa39a975c
HBASE-25910 - Fix port assignment test ( #3308 )
...
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-28 20:53:20 +05:30
Wellington Ramos Chevreuil
cad9ae386a
HBASE-25933 Log trace raw exception, instead of cause message in NettyRpcServerRequestDecoder ( #3323 )
...
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2021-05-27 20:03:17 +01:00
Duo Zhang
3cf0678aee
HBASE-25926 Cleanup MetaTableAccessor references in FavoredNodeBalancer related code ( #3313 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-27 16:25:01 +08:00
Rushabh Shah
e265eccf20
HBASE-25924 Re-compute size of WAL file while removing from WALEntryStream ( #3315 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-05-26 10:42:03 -07:00
Baiqiang Zhao
563ebc214a
HBASE-25534 Honor TableDescriptor settings earlier in normalization ( #2917 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-25 08:24:05 -07:00
Duo Zhang
2cb6cc8a6c
HBASE-25894 Improve the performance for region load and region count related cost functions ( #3276 )
...
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
2021-05-25 18:07:33 +08:00
GeorryHuang
8754e88fa7
HBASE-25906 UI of master-status to show recent history of balancer desicion ( #3303 ) ( #3296 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-25 12:39:47 +05:30
Baiqiang Zhao
073f23e2bd
HBASE-25513 When the table is turned on normalize, the first region may not be merged even the size is 0 ( #2887 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-24 17:27:06 -07:00
Anoop Sam John
1649013b99
HBASE-25898 RS getting aborted due to NPE in Replication WALEntryStream ( #3292 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
2021-05-24 23:44:32 +05:30
Xiaolin Ha
76d56bc42e
HBASE-25899 Improve efficiency of SnapshotHFileCleaner ( #3280 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-24 22:18:43 +08:00
Duo Zhang
1c1acee2fb
HBASE-25873 Refactor and cleanup the code for CostFunction ( #3274 )
...
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
2021-05-24 22:13:01 +08:00
Xiaolin Ha
16f865f65d
HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky ( #3160 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-22 22:20:47 +08:00
caoliqing
a19dee1eed
HBASE-25892: 'False' should be 'True' in auditlog of listLabels ( #3273 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-22 17:25:43 +08:00
Sandeep Pal
926c1132c0
HBASE-25848: Add flexibility to backup replication in case replication filter throws an exception
...
Signed-off-by: Xu Cang <xcang@salesforce.com>
2021-05-21 12:14:35 -07:00
Andrew Purtell
2d591ab3c4
HBASE-25869 WAL value compression ( #3244 )
...
WAL storage can be expensive, especially if the cell values
represented in the edits are large, consisting of blobs or
significant lengths of text. Such WALs might need to be kept around
for a fairly long time to satisfy replication constraints on a space
limited (or space-contended) filesystem.
We have a custom dictionary compression scheme for cell metadata that
is engaged when WAL compression is enabled in site configuration.
This is fine for that application, where we can expect the universe
of values and their lengths in the custom dictionaries to be
constrained. For arbitrary cell values it is better to use one of the
available compression codecs, which are suitable for arbitrary albeit
compressible data.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-21 11:53:36 -07:00
Rushabh Shah
7b374b7c76
HBASE-25827 Per Cell TTL tags get duplicated with increments causing tags length overflow ( #3290 )( #3210 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-21 22:58:32 +05:30
Duo Zhang
d78c164429
HBASE-25897 TestRetainAssignmentOnRestart is flaky after HBASE-25032 ( #3281 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-05-20 21:18:09 +08:00
GeorryHuang
3040be1438
HBASE-25791 UI of master-status to show a recent history of that why balancer was rejected to run ( #3278 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-20 15:00:00 +08:00
Nick Dimiduk
aab6e1d0ff
HBASE-25861 Correct the usage of Configuration#addDeprecation ( #3249 )
...
Co-authored-by: Baiqiang Zhao <zbq.dean@gmail.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-18 06:53:31 -07:00
Duo Zhang
8110b18ab0
HBASE-25032 Do not assign regions to region server which has not called regionServerReport yet ( #3268 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-05-18 10:43:01 +08:00
Pankaj
1a8d3c3b6b
HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager ( #3250 )
...
* HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager's start & retrievePassword method
Signed-off-by: stack <stack@apache.com>
(cherry picked from commit 2126ec94f0
)
2021-05-17 12:21:47 +05:30
Duo Zhang
439fd4aff2
HBASE-25872 Add documentation for LoadBalancer about synchronization ( #3267 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-05-17 10:20:48 +08:00
Duo Zhang
d5d972f92e
HBASE-25883 The regionFinder and rackManager fields in BaseLoadBalancer should be volatile ( #3262 )
...
Signed-off-by: meiyi <myimeiyi@gmail.com>
2021-05-17 09:54:40 +08:00
Viraj Jasani
1bb16fea5b
HBASE-25884 Return empty records for disabled balancer in-memory queue ( #3263 )
...
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Aman Poonia <apoonia@salesforce.com>
2021-05-14 12:57:24 +05:30
Duo Zhang
72d7641fb5
HBASE-25852 Move all the intialization work of LoadBalancer implementation to initialize method ( #3248 ) ( #3258 )
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-13 14:19:03 +08:00
Michael Stack
075545731a
HBASE-25876 Add retry if we fail to read all bytes of the protobuf magic marker ( #3251 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2021-05-12 11:20:50 -07:00
Michael Stack
61d9b46aab
HBASE-25867 Extra doc around ITBLL ( #3242 )
...
* HBASE-25867 Extra doc around ITBLL
Minor edits to a few log messages.
Explain how the '-c' option works when passed to ChaosMonkeyRunner.
Some added notes on ITBLL.
Fix whacky 'R' and 'Not r' thing in Master (shows when you run ITBLL).
In HRS, report hostname and port when it checks in (was debugging issue
where Master and HRS had different notions of its hostname).
Spare a dirty FNFException on startup if base dir not yet in place.
* Address Review by Sean
Signed-off-by: Sean Busbey <busbey@apache.org>
2021-05-11 19:24:33 +01:00
Catalin Luca
4e507ccda0
HBASE-25859 Reference class incorrectly parses the protobuf magic marker ( #3236 )
...
Co-authored-by: Catalin Luca <luca@adobe.com>
Signed-off-by: stack <stack@apache.org>
2021-05-10 14:47:18 -05:00
GeorryHuang
63d49cb7ed
HBASE-25790 NamedQueue 'BalancerRejection' for recent history of balancer skipping ( #3182 ) ( #3245 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-09 00:09:23 +05:30
Rushabh Shah
90dc150b1b
HBASE-25860 Add metric for successful wal roll requests. ( #3238 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-08 13:02:52 +05:30
Andrew Purtell
d60aa0a865
HBASE-25774 ServerManager.getOnlineServer may miss some region servers when refreshing state in some procedure implementations
...
Revert "HBASE-25032 Wait for region server to become online before adding it to online servers in Master (#2771 )"
This reverts commit 3bb978894d
.
2021-05-07 17:55:20 -07:00
niuyulin
5e65da64e2
HBASE-25837 TestRollingRestart is flaky ( #3220 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-07 20:28:37 +08:00
xiaozhang0319
bec79d877f
HBASE-25850 Fix spotbugs warnings on branch-2 ( #3239 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-07 20:21:53 +08:00
Andrew Purtell
2938ba421b
HBASE-25854 Remove redundant AM in-memory state changes in CatalogJanitor ( #3234 )
...
In CatalogJanitor we schedule GCRegionProcedure to clean up both
filesystem and in-memory state after a split, and
GCMultipleMergedRegionsProcedure to do the same for merges. Both of these
procedures clean up in-memory state, but CatalogJanitor also does this
redundantly just after scheduling the procedures. The cleanup should be
done in only one place. Presumably we are using the procedures to do it in
a principled way. Remove the redundancy in CatalogJanitor and fix any
follow on issues, like test failures.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-06 09:17:29 -07:00
Duo Zhang
2d4904ceea
HBASE-25851 Make LoadBalancer not extend Configurable interface ( #3233 ) ( #3235 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-06 23:39:43 +08:00
Andrew Purtell
b2c58331f3
HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore ( #3230 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-05-05 17:02:12 -07:00
Nick Dimiduk
6f2842d1af
HBASE-25843 move master http-related code into o.a.h.h.master.http
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-05 08:36:40 -07:00
Duo Zhang
afa9836b87
HBASE-25834 Remove balanceTable method from LoadBalancer interface ( #3217 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-05 16:03:51 +08:00
Nick Dimiduk
3f8d3ba6d9
HBASE-25842 move regionserver http-related code into o.a.h.h.regionserver.http
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-04 15:46:53 -07:00
Andrew Purtell
36994a2c5e
HBASE-25835 Ignore duplicate split requests from regionserver reports ( #3218 )
...
Processing of the RS report happens asynchronously from other activities
which can mutate region state. For example, a split procedure may already
be running. A split procedure cannot succeed if the parent region is no
longer open, so we can ignore it in that case.
Note that submitting more than one split procedure for a given region is
harmless -- the split is fenced in the procedure handling -- but it would
be noisy in the logs. Only one procedure can succeed. The other
procedure(s) would abort during initialization and report failure with
WARN level logging.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
2021-05-04 10:22:57 -07:00
Andrew Purtell
c8f3dd56b7
HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions ( #3219 )
...
RegionStates#getAssignmentsForBalancer is used by the HMaster to
collect all regions of interest to the balancer for the next chore
iteration. We check if a table is in disabled state to exclude
regions that will not be of interest (because disabled regions are
or will be offline) or are in a state where they shouldn't be
mutated (like SPLITTING). The current checks are not actually
comprehensive.
Filter out regions not in OPEN or OPENING state when building the
set of interesting regions for the balancer to consider. Only
regions open (or opening) on the cluster are of interest to
balancing calculations for the current iteration. Regions in all
other states can be expected to not be of interest – either offline
(OFFLINE, or FAILED_*), not subject to balancer decisions now
(SPLITTING, SPLITTING_NEW, MERGING, MERGING_NEW), or will be
offline shortly (CLOSING) – until at least the next chore
iteration.
Add TRACE level logging.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-03 18:24:13 -07:00
Andrew Purtell
60055f5554
HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip ( #3223 )
...
We claim in a WARN level log line to be "Playing-it-safe skipping merge/
split gc'ing of regions from hbase:meta while regions-in-transition (RIT)"
but do not actually skip because of a missing return. Remove the warning.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-03 18:17:41 -07:00
Duo Zhang
e725ef540e
HBASE-25838 Use double instead of Double in StochasticLoadBalancer ( #3221 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-04 09:11:28 +08:00
Duo Zhang
c52c091609
HBASE-25819 Fix style issues for StochasticLoadBalancer ( #3207 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-29 11:26:58 +08:00
Nick Dimiduk
3a01941963
HBASE-25779 HRegionServer#compactSplitThread should be private
...
Minor refactor. Make the `compactSplitThread` member field of `HRegionServer` private, and gate
all access through the getter method.
Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-04-28 16:54:44 -07:00
stack
04c30bda88
HBASE-25792 Filter out o.a.hadoop.thirdparty building shaded jars ( #3184 )
...
Need to add to allowed-licenses list too....
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
2021-04-27 08:41:15 -07:00
Andrew Purtell
01ce44abc4
HBASE-25756 Support alternate compression for major and minor compactions ( #3142 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
hbase-shell/src/main/ruby/hbase/admin.rb
2021-04-23 16:02:41 -07:00
Duo Zhang
f374357cc3
HBASE-25802 Miscellaneous style improvements for load balancer related classes ( #3192 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-23 16:32:10 +08:00
haxiaolin
fd7b2c6eec
HBASE-25754 StripeCompactionPolicy should support compacting cold regions ( #3152 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-04-23 15:31:55 +08:00
Duo Zhang
a02ce95ff1
HBASE-25793 Move BaseLoadBalancer.Cluster to a separated file ( #3185 ) ( #3190 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-23 00:08:16 +08:00
Toshihiro Suzuki
f4f84302fa
HBASE-25766 Introduce RegionSplitRestriction that restricts the pattern of the split point
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-04-22 14:06:28 +09:00
Duo Zhang
a177fb4c62
HBASE-25775 Use a special balancer to deal with maintenance mode ( #3161 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-04-16 13:39:48 +08:00
Nick Dimiduk
32207cd478
HBASE-25770 Http InfoServers should honor gzip encoding when requested ( #3159 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2021-04-15 13:46:53 -07:00
xiaoyu
819062bac3
HBASE-25776 Use Class.asSubclass to fix the warning in StochasticLoadBalancer.loadCustomCostFunctions ( #3163 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-04-15 23:35:33 +05:30
DivyeshChandra
35520a72fb
HBASE-25653 Add units and round off region size to 2 digits after decimal ( #3046 )
...
Signed-off-by: stack <stack@duboce.net>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
2021-04-13 09:06:05 -07:00
Duo Zhang
15659f0e85
HBASE-25767 CandidateGenerator.getRandomIterationOrder is too slow on large cluster ( #3149 )
...
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-13 23:14:48 +08:00
Baiqiang Zhao
ae3daa5c0b
HBASE-25482 Improve SimpleRegionNormalizer#getAverageRegionSizeMb ( #2858 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-04-12 15:19:43 -07:00
Duo Zhang
702220a135
HBASE-25759 The master services field in LocalityBasedCostFunction is never used ( #3144 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-04-12 22:31:05 +08:00
Nick Dimiduk
b071902b10
HBASE-25744 Change default of `hbase.normalizer.merge.min_region_size.mb` to `0`
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-04-11 16:22:02 -07:00
Geoffrey Jacoby
89c9f28d3f
HBASE-25751 - Add writable TimeToPurgeDeletes to ScanOptions ( #3137 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-04-09 13:09:24 -07:00
meiyi
3faa67520c
HBASE-25747 Remove unused getWriteAvailable method in OperationQuota ( #3133 )
...
Signed-off-by: stack <stack@apache.org>
2021-04-09 10:41:51 +08:00
Pankaj
89fa8e7333
HBASE-25717 RegionServer aborted with due to ClassCastException ( #3108 )
...
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-04-08 16:28:41 +05:30
Baiqiang Zhao
8ff17c68e2
HBASE-25687 Backport "HBASE-25681 Add a switch for server/table query… ( #3074 )
...
Signed-off-by: stack <stack@apache.org>
2021-04-07 11:11:46 -07:00
Jan Hentschel
18882d6f9a
HBASE-24305 Prepare deprecations in ServerName ( #1666 ) ( #3121 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-04-05 22:10:04 +02:00
Jan Hentschel
1fa07cea80
HBASE-25199 Remove deprecated HStore#getStoreHomedir methods ( #2562 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-04-03 23:44:23 +08:00
d-c-manning
554e039817
HBASE-25726 MoveCostFunction is not included in the list of cost functions for StochasticLoadBalancer ( #3116 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-04-02 10:43:51 -07:00
stack
b051ad5594
Revert "HBASE-25713 Make an hbase-wal module"
...
This reverts commit e58f1b24a6
.
2021-03-31 16:15:50 -07:00
lujiefsi
d90d0c2231
HBASE-25558:Adding audit log for execMasterService ( #3101 )
...
Signed-off-by: stack <stack@apache.org>
2021-03-31 16:14:50 -07:00
stack
e58f1b24a6
HBASE-25713 Make an hbase-wal module
...
First extract an hbase-coprocessor module used by hbase-client, hbase-server.
This is prerequisite to extracting an hbase-wal module.
M hbase-common/src/main/java/org/apache/hadoop/hbase/Abortable.java
M hbase-common/src/main/java/org/apache/hadoop/hbase/DoNotRetryIOException.java
M hbase-common/src/main/java/org/apache/hadoop/hbase/util/SortedList.java
Move to hbase-common. Its a generic Interface. Need by
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/Coprocessor.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/CoprocessorEnvironment.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseEnvironment.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/CoreCoprocessor.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/ObserverContext.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/ObserverContextImpl.java
M hbase-coprocessor/src/main/java/org/apache/hadoop/hbase/coprocessor/ReadOnlyConfiguration.java
Move to hbase-coprocessor.
M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/client/coprocessor/BigDecimalColumnInterpreter.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/coprocessor/DoubleColumnInterpreter.java
M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/client/coprocessor/LongColumnInterpreter.java
M hbase-endpoint/src/main/java/org/apache/hadoop/hbase/coprocessor/ColumnInterpreter.java
Moved to hbase-endpoint where they are used.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
Include region name when toString'd.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCoprocessorHost.java
Include WAL name when toString'd.
M hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
Add utility used in testing here from CoprocessorHost.
2021-03-31 16:14:45 -07:00
Toshihiro Suzuki
1d3ea38f1e
HBASE-25703 Support conditional update in MultiRowMutationEndpoint ( #3107 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-03-30 23:04:29 +09:00
Josh Elser
fa8b9659b1
HBASE-25692 Always try to close the WAL reader when we catch any exception ( #3090 )
...
There are code paths in which we throw non-IOExceptions when
initializing a WAL reader. However, we only close the InputStream to the
WAL filesystem when the exception is an IOException. Close it if it is
open in all cases.
Co-authored-by: Josh Elser <jelser@cloudera.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-03-29 12:17:16 -07:00
Michael Stack
84eb0220a4
HBASE-25695 Link to the filter on hbase:meta from user tables panel on master page ( #3092 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-03-27 20:35:20 -07:00
caroliney14
3bb978894d
HBASE-25032 Wait for region server to become online before adding it to online servers in Master ( #2771 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-03-25 17:10:43 -07:00
Andrew Purtell
124ea5eee3
HBASE-25693 NPE getting metrics from standby masters (MetricsMasterWrapperImpl.getMergePlanCount) ( #3091 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-03-24 19:10:58 -07:00
huaxiangsun
80529af963
HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile ( #3087 )
...
The issue is that FileInputStream is created with try-with-resources, so its close() is called right after the try sentence.
FileInputStream is a finalize class, when this object is garbage collected, its close() is called again.
To avoid this double-free resources, add guard against it.
Signed-off-by: stack <stack@apache.org>
2021-03-24 14:38:44 -07:00
Toshihiro Suzuki
2b5c2c36d9
HBASE-25678 Support nonce operations for Increment/Append in RowMutations and CheckAndMutate ( #3073 )
...
Signed-off-by: stack <stack@apache.org>
2021-03-21 22:47:53 +09:00
shahrs87
ed8b72627d
HBASE-25679 Size of log queue metric is incorrect ( #3071 )
...
Co-authored-by: Rushabh <rushabh.shah@salesforce.com>
Signed-off-by: stack <stack@apache.org>
2021-03-19 16:14:45 -07:00
haxiaolin
566b0512a7
HBASE-25518 Support separate child regions to different region servers ( #3001 )
...
Signed-off-by: stack <stack@apache.org>
2021-03-18 12:39:00 -07:00
bsglz
19d094b96f
HBASE-25643 The delayed FlushRegionEntry should be removed when we ne… ( #3049 )
...
Signed-off-by: AnoopSamJohn <anoopsamjohn@apache.org>
Signed-off-by: stack <stack@apache.org>
2021-03-18 12:13:34 -07:00
Michael Stack
75e2e7c83d
HBASE-25677 Server+table counters on each scan #nextRaw invocation becomes a bottleneck when heavy load ( #3061 )
...
Don't have every handler update regionserver metrics on each
scan#nextRaw; instead, do a batch update just before Scan
returns. Otherwise, all running handlers end up contending
on metrics update.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
Update of regionserver metrics counters moved out to caller where
can be done as a batch update instead of per-next.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java
Class doc to encourage batch updating metrics.
Remove the single update as unused anymore.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
Count calls to nextRaw. Update regionserver count in finally block when
scan is done rather than per nextRaw call. Move all metrics updates to
finally.
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Baiqiang Zhao <ZhaoBQ>
2021-03-18 11:40:42 -07:00
Sandeep Pal
72496272aa
HBASE-25627: HBase replication should have a metric to represent if the source is stuck getting initialized ( #3018 )
...
Introduces a new metric that tracks number of replication sources that are stuck in initialization.
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit ff3821814a
)
2021-03-17 10:30:26 -07:00
bitterfox
a61e9b963d
HBASE-25665 Option to use hostname instead of canonical hostname for secure HBase cluster connection ( #3051 )
2021-03-16 21:05:23 -07:00
Baiqiang Zhao
a93b0cd3ca
HBASE-25597 Add row info in Exception when cell size exceeds maxCellSize ( #2976 )
...
Signed-off-by: stack <stack@apache.org>
2021-03-15 15:50:22 -07:00
haxiaolin
6c224b8640
HBASE-25621 Balancer should check region plan source to avoid misplace region groups ( #3002 )
...
Signed-off-by: stack <stack@duboce.net>
2021-03-15 14:48:35 -07:00
haxiaolin
13574de156
HBASE-25595 TestLruBlockCache.testBackgroundEvictionThread is flaky ( #2974 )
...
Signed-off-by: stack <stack@apache.org>
2021-03-15 14:26:06 -07:00
Michael Stack
3bf2ef4259
HBASE-25660 Print split policy in use on Region open (as well as split policy vitals) ( #3044 )
...
Add a toString to all split policy implementations listing name and
vitals. Use this toString in the Region open message. Ditto for flush
policy for the Region.
Signed-off-by: Huaxiang Sun<huaxiangsun@apache.org>
2021-03-15 14:10:37 -07:00
haxiaolin
95e8ee5ca5
HBASE-25635 CandidateGenerator may miss some region balance actions ( #3024 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-03-15 21:33:46 +08:00
Duo Zhang
aafbbc2456
HBASE-25657 Fix spotbugs warnings after upgrading spotbugs to 4.x ( #3041 )
...
Signed-off-by: meiyi <myimeiyi@gmail.com>
Signed-off-by: stack <stack@apache.org>
2021-03-12 14:37:17 +08:00
shahrs87
bb4d8b9cf7
HBASE-25622 Result#compareResults should compare tags. ( #3026 )
...
Signed-off-by: stack <stack@apache.org>
2021-03-11 21:52:11 -08:00
Michael Stack
45b3280497
HBASE-25570 On largish cluster, "CleanerChore: Could not delete dir..." makes master log unreadable ( #2949 )
...
Turn down the amount we log. If you want to see the full exception
enable TRACE-level logging.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: shahrs87
2021-03-11 21:36:31 -08:00
Peter Somogyi
af8dc2e4df
HBASE-25587 [hbck2] Schedule SCP for all unknown servers ( #2978 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-03-11 17:22:08 +01:00
meiyi
b5fc5e17e2
HBASE-25636 Expose HBCK report as metrics ( #3031 )
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2021-03-11 15:15:23 +08:00
Duo Zhang
d546c8e77c
HBASE-25629 Reimplement TestCurrentHourProvider to not depend on unstable TZs ( #3013 )
...
Signed-off-by: XinSun <ddupgs@gmail.com>
2021-03-10 23:53:59 +08:00
Wellington Ramos Chevreuil
7adc00044c
HBASE-25548 Optionally allow snapshots to preserve cluster's max file… ( #2923 )
...
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit 373dc7788d
)
2021-03-10 10:35:58 +00:00
Anoop Sam John
29d771fab9
HBASE-25582 Support setting scan ReadType to be STREAM at cluster level ( #3040 )
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-03-10 13:23:15 +05:30
Narges Shadab
884cb0bd2c
HBASE-25646: Possible Resource Leak in CatalogJanitor #3036
...
Close the stream using a try-with-resources block.
Reviewed-by: Aman Poonia <aman.poonia.29@gmail.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 573daed419
)
2021-03-09 09:11:02 -08:00
Bharath Vissapragada
321b5c7979
HBASE-25547: Thread pools should release unused resources ( #3037 )
...
* HBASE-25547: Thread pools should release unused resources (#2922 )
Plumbs the configuration needed to enable core thread timeout on non-critical thread pools.
Currently only enabled for thread pools with op-codes RS_LOG_REPLAY_OPS, RS_PARALLEL_SEEK, MASTER_SNAPSHOT_OPERATIONS, MASTER_MERGE_OPERATIONS. Others can be added later as
needed.
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 618236dd90
)
* HBASE-25547 (addendum): Roll ExecutorType into ExecutorConfig
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 4c822d7463
)
2021-03-09 09:03:16 -08:00