482ef6904e
HBASE-26488 Memory leak when MemStore retry flushing ( #3899 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-18 12:22:52 +08:00
d072a2e694
HBASE-26340 TableSplit returns false size under 1MB ( #3872 )
...
Signed-off-by: Peter Somogyi <psomogyi@apache.org
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-18 12:21:51 +08:00
37397637ad
HBASE-26542 Apply a `package` to test protobuf files (addendum)
...
RpcServer identifies the services it hosts by unqualified service name. Thus, use `getName()`
instead of `getFullName()`. See also HBASE-26589.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-12-16 16:39:48 -08:00
2bfb743d79
HBASE-26550 Make sure the master is running normally before accepting a balance command ( #3928 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-14 23:32:34 +08:00
49d4c4a4c0
HBASE-26027 The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException ( #3925 )
...
* HBASE-26027 The calling of HTable.batch blocked at AsyncRequestFutureImpl.waitUntilDone caused by ArrayStoreException
2021-12-13 16:07:42 -08:00
9cf0af6e79
HBASE-26537: Make HBASE-15676 backwards compatible, using a flag on the proto ( #3931 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-12-13 13:34:10 -08:00
140b5d8d26
HBASE-24870 Ignore TestAsyncTableRSCrashPublish ( #2248 )
...
Signed-off-by: meiyi <myimeiyi@gmail.com>
2021-12-06 19:07:07 -08:00
ec52519f7c
HBASE-26462 Should persist restoreAcl flag in the procedure state for CloneSnapshotProcedure and RestoreSnapshotProcedure ( #3921 )
...
Signed-off-by: Yu Li <liyu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-06 23:16:15 +08:00
074456e190
HBASE-26533 KeyValueScanner might not be properly closed when using InternalScan.checkOnlyMemStore() ( #3917 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-05 22:35:07 +08:00
19d991d291
HBASE-26517 Add auth method information to AccessChecker audit log ( #3897 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-04 23:14:42 +08:00
d504504875
HBASE-26304 Reflect out of band locality improvements in metrics and balancer ( #3895 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-12-04 12:23:09 +08:00
d1c68a2d5c
HBASE-26471 Move tracing semantic attributes to their own class ( #3896 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
2021-12-01 13:24:08 -08:00
6fc5bbc19a
HBASE-26468 Region Server doesn't exit cleanly incase it crashes. ( #3865 ) ( #3862 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-12-01 11:06:07 +05:30
774484ed44
HBASE-25905 Shutdown of WAL stuck at waitForSafePoint ( #3898 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-12-01 08:34:08 +08:00
799217ee69
HBASE-26455 TestStochasticLoadBalancerRegionReplicaWithRacks fails consistently ( #3879 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2021-11-28 22:23:21 +08:00
5437532910
HBASE-26476 Make DefaultMemStore extensible for HStore.memstore ( #3869 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-27 12:06:35 +08:00
5101f372e4
HBASE-26465 MemStoreLAB may be released early when its SegmentScanner is scanning ( #3859 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Anoop Sam John <anoopsamjohn@apache.org>
2021-11-25 21:35:55 +08:00
6aa2daa2c2
HBASE-26477 Fix MiniMapReduceCluster failure under Java17 ( #3878 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-25 16:27:23 +08:00
fa34cfc7d1
Revert "Backport HBASE-26477 Fix MiniMapReduceCluster failure under Java17 to branch-2 ( #3878 )"
...
This reverts commit ccf150434d
2021-11-25 16:27:11 +08:00
ccf150434d
Backport HBASE-26477 Fix MiniMapReduceCluster failure under Java17 to branch-2 ( #3878 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-25 16:25:44 +08:00
2798ef132b
HBASE-26479 Print too slow/big scan's operation_id in region server log ( #3870 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-25 11:43:01 +08:00
52ce87780a
Revert "HBASE-26479 Print too slow/big scan's operation_id in region server log ( #3870 )"
...
This reverts commit b437a96272
2021-11-24 17:34:01 -05:00
b437a96272
HBASE-26479 Print too slow/big scan's operation_id in region server log ( #3870 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-24 23:38:16 +08:00
0a51d2b5be
HBASE-26475 The flush and compact methods in HTU should skip processing secondary replicas ( #3868 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-11-23 18:51:25 +08:00
f8d03d1227
HBASE-26249 Ameliorate compaction made by bulk-loading files ( #3831 ) ( #3874 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-23 15:52:47 +08:00
61406252a5
Revert "Backport HBASE-26249 Ameliorate compaction made by bulk-loading files ( #3831 ) ( #3874 )"
...
This reverts commit 5843a9a44c
2021-11-23 15:52:34 +08:00
5843a9a44c
Backport HBASE-26249 Ameliorate compaction made by bulk-loading files ( #3831 ) ( #3874 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-23 15:45:21 +08:00
4312f817f3
HBASE-26467 Fix bug for MemStoreLABImpl.forceCopyOfBigCellInto(Cell) ( #3858 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: chenglei <chenglei@apache.org>
2021-11-21 21:02:18 +08:00
e549384437
HBASE-26463 Unreadable table names after HBASE-24605 ( #3853 )
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-21 20:49:13 +08:00
8d3562c6fa
HBASE-26421 Use HFileLink file to replace entire file's reference when splitting ( #3854 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-18 21:37:10 +08:00
4a8da47f74
HBASE-26267 Don't try to recover WALs from a WAL dir which doesn't exist ( #3679 )
...
We currently cause an error to be thrown by trying to list a
non-existent directory. We see that the master region directory exists
on the filesystem, but forget to make sure that the master region's WAL
directory also exists before we try to list it.
2021-11-16 16:27:45 -05:00
e65f28cc57
HBASE-26444 BucketCacheWriter should log only the BucketAllocatorException message, not the full stack trace ( #3840 )
...
Signed-off-by: Anoop <anoopsamjohn@apache.org>
2021-11-15 16:13:36 -08:00
23024edba2
HBASE-26443 Some BaseLoadBalancer log lines should be at DEBUG level ( #3838 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-11-15 16:06:59 -08:00
98a4fb8086
HBASE-26438 Fix flaky test TestHStore.testCompactingMemStoreCellExceedInmemoryFlushSize ( #3834 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-12 23:03:37 +08:00
ad36941932
HBASE-26450 Server configuration will overwrite HStore configuration after using shell command 'update_config' ( #3843 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
2021-11-12 11:43:48 +08:00
9db7630d23
HBASE-26410 Fix HBase TestCanaryTool for Java17 ( #3809 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-09 22:21:01 +08:00
cc2e4aec87
HBASE-26337 Optimization for weighted random generators ( #3828 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-09 21:27:44 +08:00
1302073dfe
HBASE-26430 Increase log level in DefaultHeapMemoryTuner from DEBUG to INFO ( #3820 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-11-08 18:35:39 -08:00
201b3c7fc7
HBASE-26429 HeapMemoryManager fails memstore flushes with NPE if enabled ( #3819 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-11-08 18:32:17 -08:00
2fbb375cd5
HBASE-25322 Redundant Reference file in bottom region of split ( #3814 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-08 14:48:55 +08:00
ebd89b58eb
Revert "HBASE-25322 Redundant Reference file in bottom region of split ( #3814 )"
...
This reverts commit a6edf44175
2021-11-08 11:40:43 +08:00
a6edf44175
HBASE-25322 Redundant Reference file in bottom region of split ( #3814 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-05 18:17:47 +08:00
e06208f3d6
HBASE-26311 Balancer gets stuck in cohosted replica distribution ( #3804 )
...
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-11-03 11:58:38 -07:00
9eb3706021
HBASE-26309 Balancer tends to move regions to the server at the end of list ( #3811 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-02 21:58:50 +08:00
063e0e2e92
HBASE-26406 Can not add peer replicating to non-HBase ( #3806 )
...
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b9b7fec57f
2021-11-02 15:03:33 +08:00
e89f62e611
HBASE-26384 Segment already flushed to hfile may still be remained in CompactingMemStore ( #3779 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-11-01 23:05:57 +08:00
f2c58fcf68
HBASE-26353 Support loadable dictionaries in hbase-compression-zstd ( #3787 )
...
ZStandard supports initialization of compressors and decompressors with a
precomputed dictionary, which can dramatically improve and speed up compression
of tables with small values. For more details, please see
The Case For Small Data Compression
https://github.com/facebook/zstd#the-case-for-small-data-compression
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdCodec.java
2021-10-27 07:57:16 -07:00
b4177577ac
HBASE-26394 Cache in RSRpcServices.executeProcedures does not take effect ( #3794 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-10-26 15:44:17 +08:00
32eb23c8f3
HBASE-26327 Replicas cohosted on a rack shouldn't keep triggering balancer ( #3788 )
...
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-10-25 15:44:11 -07:00
444dec9289
HBASE-26190 High rate logging of BucketAllocatorException: Allocation too big - Addendum ( #3792 )
2021-10-24 09:22:26 +05:30
f299d56934
HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger ( #3783 )
2021-10-22 09:46:58 -07:00
eea2ea8576
HBASE-26368 Fix checkstyle issues for HRegionServer for branch-2 ( #3781 ) ( #3764 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-10-22 12:51:29 +05:30
0eaadd92d8
HBASE-26385 Clear CellScanner when replay ( #3773 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-10-21 21:56:39 +08:00
dce45ec16a
HBASE-26383 HBCK incorrectly reports inconsistencies for recently split regions following a master failover ( #3775 )
...
A regression was introduced by HBASE-25847 which changed regionInfo#isParentSplit
to regionState#isSplit. The region state after restart is CLOSED instead of SPLIT.
We need to check both regionState and regionInfo for split status.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-10-20 15:55:07 -07:00
5c4a053f2a
HBASE-26371 Prioritize meta region move over other region moves in region_mover ( #3771 ) ( #3767 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-10-20 23:06:25 +05:30
b6bb18022f
HBASE-26316 Per-table or per-CF compression codec setting overrides ( #3730 )
...
We get and retain Compressor instances in HFileBlockDefaultEncodingContext,
and could in theory call Compressor#reinit when setting up the context,
to update compression parameters like level and buffer size, but we do
not plumb through the CompoundConfiguration from the Store into the
encoding context. As a consequence we can only update codec parameters
globally in system site conf files.
Fine grained configurability is important for algorithms like ZStandard
(ZSTD), which offers more than 20 compression levels, where at level 1
it is almost as fast as LZ4, and where at higher levels it utilizes
computationally expensive techniques to rival LZMA at compression ratio
but trades off significantly for reduced compresson throughput. The ZSTD
level that should be set for a given column family or table will vary by
use case.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdDecompressor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/compress/HFileTestBase.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestDataBlockEncoders.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestSeekToBlockWithEncoders.java
2021-10-19 13:59:04 -07:00
7e574c9f1d
HBASE-26190 High rate logging of BucketAllocatorException: Allocation too big ( #3752 )
...
Reduce the frequency of allocation failed traces by printing them
preiodically (once per minute). Record the allocation failures in the
Bucket Cache Stats and let the stat thread dump cumulative allocation
failures alongside other traces it dumps.
Also, this change adds trace for the Table name, Column Family and
HFileName for the most recent allocation failure in last 1 minute.
Signed-off-by: Anoop <anoopsamjohn@apache.org>
2021-10-19 13:21:11 +05:30
45b09aa4d7
HBASE-26350 Add a DEBUG when we fail the SASL handshake
...
Closes #3743
Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-10-14 15:27:32 -04:00
3ce9052188
HBASE-24833: Bootstrap should not delete the META table directory if … ( #2237 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
2021-10-12 10:56:00 -07:00
10584d70d2
HBASE-26306 Backport "HBASE-26220 Use P2P communicate between region servers to sync the list for bootstrap node" to branch-2 ( #3727 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-10-10 22:56:12 +08:00
5af02201d2
HBASE-26324 Reuse compressors and decompressors in WAL CompressionContext ( #3728 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-10-07 09:46:38 -07:00
18b9fa8a3c
HBASE-26259 Fallback support to pure Java compression ( #3691 )
...
This change introduces provided compression codecs to HBase as
new Maven modules. Each module provides compression codec support
that formerly required Hadoop native codecs, which in turn relies
on native code integration, which may or may not be available on
a given hardware platform or in an operational environment. We
now provide codecs in the HBase distribution for users whom for
whatever reason cannot or do not wish to deploy the Hadoop native
codecs.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileEncryption.java
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestCompressedWAL.java
2021-10-06 13:48:33 -07:00
706082d513
HBASE-26294 Backport "HBASE-26181 Region server and master could use itself as ConnectionRegistry" to branch-2 ( #3708 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-10-05 22:11:01 +08:00
9a33e234e7
HBASE-24601: Change default Hfile storage policy from HOT to NONE for HDFS ( #1940 )
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-10-02 16:37:39 +08:00
a2c09ae5bc
HBASE-26305 Move NavigableSet add operation to writer thread in BucketCache ( #3705 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: chenglei <chenglei@apache.org>
2021-10-01 21:06:57 +08:00
0748165ab0
HBASE-26295 BucketCache could not free BucketEntry which restored fro… ( #3699 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-29 21:46:57 +08:00
08800e19d6
HBASE-26251 StochasticLoadBalancer metrics should update even if balancer doesn't run ( #3706 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2021-09-29 21:25:39 +08:00
69749ed01e
HBASE-26289 Hbase scan setMaxResultsPerColumnFamily not giving right results ( #3688 )
...
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-29 10:22:24 +02:00
08a2efde16
HBASE-26293 Use reservoir sampling when selecting bootstrap nodes ( #3702 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-09-28 23:41:28 +08:00
786e09a936
HBASE-26238 Short message by Result#compareResults for VerifyReplication to avoid OOME ( #3647 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-28 23:16:20 +08:00
73f42b27bf
HBASE-26297 Balancer run is improperly triggered by accuracy error of double comparison ( #3698 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-26 23:35:50 +08:00
404789adde
HBASE-26178 Improve data structure and algorithm for BalanceClusterState to improve computation speed for large cluster ( #3682 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-25 21:48:51 +08:00
59d133ef95
Amend HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job
...
Compliation fixes for CombinedBlockCache and TestClientSideRegionScanner.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-09-22 12:27:59 -07:00
e0325c40f6
HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job ( #3684 )
...
Introduce `hfile.onheap.block.cache.fixed.size`
and default to disable. when using ClientSideRegionScanner
it will be enabled with a fixed size for caching
INDEX/LEAF_INDEX block when a client, e.g.
snapshot scanner, scans the entire HFile
and does not need to seek/reseek to index
block multiple times.
Signed-off-by: Josh Elser <elserj@apache.org>
2021-09-22 09:38:20 -07:00
84b5511d13
HBASE-26261 Store configuration loss when use update_config ( #3664 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-22 18:35:28 +08:00
8f6c5e1f02
HBASE-26270 Provide getConfiguration method for Region and Store interface ( #3671 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2021-09-22 11:32:32 +08:00
e327fa43c0
HBASE-26288 Revisit the usage of MetaTableLocator when HRegionServer.TEST_SKIP_REPORTING_TRANSITION is true ( #3689 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-09-22 00:27:11 +08:00
ef85204522
HBASE-26285 Remove MetaTableLocator usages in non-migration code ( #3686 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-09-21 23:32:13 +08:00
b21b250570
HBASE-26281 DBB got from BucketCache would be freed unexpectedly before RPC completed ( #3680 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-17 22:28:56 +08:00
f670fc5d4d
HBASE-26229 Limit count and size of L0 files compaction in StripeCompactionPolicy ( #3646 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-09 18:28:45 +08:00
68db7f5ed0
[HBASE-26243]Fix typo for file "hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java" ( #3650 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-09 12:21:54 +08:00
820ba21b2a
HBASE-26106 AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory. ( #3636 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
2021-09-02 10:26:22 -07:00
ccd90269d2
HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky ( #3651 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 345d7256c8
2021-09-02 10:33:41 +08:00
e07928df4c
HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
(Re-application of HBASE-26147 with the correct author metadata)
2021-09-01 21:58:30 -04:00
1e74f08403
Revert "HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed"
...
This reverts commit 5b58d11c91
2021-09-01 21:57:46 -04:00
5b58d11c91
HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org
2021-09-01 21:47:53 -04:00
c639d6f353
HBASE-26136 Backport HBASE-25723 "Temporarily remove the trace support for RegionScanner.next" to branch-2 ( #3623 )
...
13/17 commits of HBASE-22120, original commit 7f90c2201f
2021-09-01 15:29:09 -07:00
d0a53e3f29
HBASE-26133 Backport HBASE-25591 "Upgrade opentelemetry to 0.17.1" to branch-2 ( #3608 )
...
10/17 commits of HBASE-22120, original commit f6ff519dd0
2021-09-01 15:29:09 -07:00
8b7450f5d8
HBASE-26132 Backport HBASE-25535 "Set span kind to CLIENT in AbstractRpcClient" to branch-2 ( #3607 )
...
9/17 commits of HBASE-22120, original commit bb8c4967f8
2021-09-01 15:29:09 -07:00
9cce94a2d0
HBASE-26131 Backport HBASE-25484 "Add trace support for WAL sync" to branch-2 ( #3597 )
...
8/17 commits of HBASE-22120, original commit 2be2c63f0d
2021-09-01 15:29:09 -07:00
321dcd4e77
HBASE-26130 Backport HBASE-25455 "Add trace support for HRegion read/… ( #3594 )
...
7/17 commits of HBASE-22120, original commit 03e12bfa4a
2021-09-01 15:29:09 -07:00
83a27389a0
HBASE-26129 Backport HBASE-25481 "Add host and port attribute when tracing rpc call at client side" to branch-2 ( #3589 )
...
6/17 commits of HBASE-22120, original commit ae2c62ffaa
2021-09-01 15:29:09 -07:00
58b5c007b9
HBASE-26127 Backport HBASE-23898 "Add trace support for simple apis i… ( #3556 )
...
4/17 commits of HBASE-22120, original commit 805b2ae2ad
2021-09-01 15:29:09 -07:00
4dfc59c19e
HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 ( #3543 )
...
2/17 commits of HBASE-22120
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
665305cc3b
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 ( #3529 )
...
1/17 commits of HBASE-22120
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
dbf43dcfbd
HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore ( #3604 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 18:17:13 +08:00
8be2ad2d42
HBASE-25642 Fix or stop warning about already cached block ( #3638 )
...
Our logs have as a fairly common occurrence: 2021-03-05 22:24:31,034 WARN
[StoreFileOpener-foo-1] hfile.BlockCacheUtil: Caching an already cached
block: blah.bub. This is harmless and can happen in rare cases (see HBASE-8547)
Because it is harmless, log at DEBUG level, not WARN.
Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
2021-08-31 09:26:10 -07:00
c08e877d6a
HBASE-26235 We could start RegionServerTracker before becoming active master ( #3645 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-08-30 12:22:41 +08:00
1e49e94051
HBASE-26232 SEEK_NEXT_USING_HINT is ignored on reversed Scans ( #3639 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-29 21:42:48 +08:00
033ff757fd
HBASE-26216 Move HRegionServer.abort(String) to Abortable as a default method ( #3642 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-28 23:06:54 +08:00
3b1482ef60
HBASE-26189 Reduce log level of CompactionProgress notice to DEBUG ( #3579 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-08-27 10:15:28 -07:00
30bb72bdb8
HBASE-26227 Forward port HBASE-26223 test code to branch-2.4+ ( #3632 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-27 16:46:54 +08:00
cd9e45ce8d
HBASE-26219 Negative time is logged while waiting on regionservers ( #3622 )
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-25 16:25:24 +02:00
931a7a5a61
HBASE-26180 Introduce a initial refresh interval for RpcConnectionRegistry ( #3601 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-08-25 18:51:40 +08:00
dacd3404a4
HBASE-26215 The backup master status page should use ActiveMasterManager instead of MasterAddressTracker ( #3621 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-25 16:42:45 +08:00
5dad395869
HBASE-26173 Return only a sub set of region servers as bootstrap nodes ( #3599 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-24 23:17:56 +08:00
6ad7eb88c3
HBASE-26172 Deprecated MasterRegistry ( #3566 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-19 16:30:48 +08:00
101602007a
HBASE-24337 Backport HBASE-23968 to branch-2 ( #3588 )
...
Signed-off-by: stack <stack@apache.org>
2021-08-18 10:19:35 -07:00
f2f99aea34
Amend HBASE-26187 UTs, adding a check for Split procedure completion before proceeding with test logic. ( #3598 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f2e2140b57
2021-08-18 16:41:08 +01:00
30c20d7627
HBASE-26187 Write straight into the store directory when Splitting an… ( #3574 ) ( #3595 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-18 09:42:03 +01:00
4ccd215354
HBASE-26087 JVM crash when displaying RPC params by MonitoredRPCHandler ( #3489 )
...
Signed-off-by: stack <stack@apache.org>
2021-08-18 14:39:28 +08:00
d99ca37307
HBASE-26193 Do not store meta region location as permanent state on zookeeper ( #3583 )
...
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
2021-08-17 18:58:39 +08:00
5eac407944
HBASE-24652 master-status UI make date type fields sortable
2021-08-16 16:03:29 -07:00
63c57d7476
HBASE-26200 Revert "HBASE-25165 Change 'State time' in UI so sorts ( #2508 )"
...
This reverts commit d07d181ea4
2021-08-16 13:49:55 -07:00
4aec407c02
HBASE-25680 Non-idempotent test in TestReplicationHFileCleaner ( #2984 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-14 11:11:34 +08:00
725474bbe3
HBASE-26026 HBase Write may be stuck forever when using CompactingMemStore ( #3421 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-12 23:19:32 +08:00
e8ecafb22b
HBASE-26155 JVM crash when scan ( #3553 )
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-12 17:17:03 +08:00
6b9114d95e
HBASE-26185 Return mutable list in AssignmentManager#getExcludedServersForSystemTable ( #3577 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-08-12 13:47:52 +05:30
243e861985
Revert "HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned ( #3532 )"
...
This reverts commit 8f16e34eb2
2021-08-11 07:41:22 -07:00
8f16e34eb2
HBASE-26122: Implement an optional maximum size for Gets, after which a partial result is returned ( #3532 )
...
Signed-off-by: stack <stack@apache.org>
2021-08-10 20:38:06 -07:00
40a7a75a19
HBASE-26185 Fix TestMaster#testMoveRegionWhenNotInitialized in the presence of hbase.min.version.move.system.tables ( #3573 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-08-10 10:58:31 +05:30
79a6dc6bfd
HBASE-6908: Add pluggable rpc queue implementation ( #3522 )
...
Can pass in a FQCN to load as the call queue implementation.
Standardized arguments to the constructor are the max queue length, the
PriorityFunction, and the Configuration.
`PluggableBlockingQueue` abstract class provided to help guide the
correct constructor signature
Hard fails if the class fails to load as a `BlockingQueue<CallRunner>`
Signed-off-by: stack <stack@apache.org>
2021-08-09 12:55:27 -07:00
611ffa7da9
HBASE-26179 TestRequestTooBigException spends too much time to finish ( #3571 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-08-07 23:05:53 +08:00
61fbe6f480
HBASE-26170 handleTooBigRequest in NettyRpcServer didn't skip enough bytes ( #3564 )
...
Signed-off-by: stack <stack@apache.com>
2021-08-07 23:05:27 +08:00
4f0f519f9f
HBASE-26142 NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero ( #3531 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-08-05 21:04:42 +08:00
effde97a88
HBASE-25469 Add detailed RIT info in JSON format for consumption as metrics ( #3555 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-04 18:23:17 -07:00
8e9253edd8
Revert "Add detailed RIT info in JSON format for consumption as metrics ( #3555 )"
...
Commit is missing JIRA identifier.
This reverts commit 31b5e09ac2
2021-08-04 18:23:08 -07:00
31b5e09ac2
Add detailed RIT info in JSON format for consumption as metrics ( #3555 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-04 18:04:44 -07:00
c11a3e1b39
Revert "HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 ( #3529 )"
...
This reverts commit f049301606
2021-08-04 15:55:13 -07:00
fffdcba5bb
Revert "HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 ( #3543 )"
...
This reverts commit ca096437d7
2021-08-04 15:55:13 -07:00
620806e3fb
HBASE-26150 Let region server also carry ClientMetaService ( #3550 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-08-05 00:15:49 +08:00
04d3c7a9d0
HBASE-26166 Fix a minor bug of table list on master ui ( #3554 )
...
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
2021-08-04 20:42:15 +08:00
ca096437d7
HBASE-26125 Backport HBASE-25401 "Add trace support for async call in rpc client" to branch-2 ( #3543 )
...
2/17 commits of HBASE-22120
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-08-02 11:33:19 -07:00
aa3b07f6bb
HBASE-26154: Adds exception metrics for QuotaExceededException and RpcThrottlingException ( #3544 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2021-08-02 09:51:34 +05:30
f7baa195e3
HBASE-26151 Reimplement MasterAddressTracker to also cache backup master addresses ( #3548 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-31 15:54:06 +08:00
aff880e147
HBASE-26144 The HStore.snapshot method is never called in main code ( #3533 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-31 14:39:13 +08:00
f3c022740a
HBASE-26105 Rectify the expired TODO comment in CombinedBC. ( #3509 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-07-30 21:43:31 +08:00
6df4cb1cf4
HBASE-26114 when “hbase.mob.compaction.threads.max” is set to a negative number, HMaster cannot start normally ( #3541 )
...
Signed-off-by: Anoop <anoopsamjohn@apache.org>
2021-07-30 17:13:50 +05:30
f049301606
HBASE-26124 Backport HBASE-25373 "Remove HTrace completely in code base and try to make use of OpenTelemetry" to branch-2 ( #3529 )
...
1/17 commits of HBASE-22120
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-07-29 09:15:10 -07:00
dd22fecc2b
HBASE-26120 New replication gets stuck or data loss when multiwal groups more than 10 ( #3528 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-07-27 08:14:15 -07:00
20a4aaedcc
HBASE-26118 The HStore.commitFile and HStore.moveFileIntoPlace almost have the same logic ( #3525 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-26 21:47:12 +08:00
37c3f7b446
HBASE-21946 Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable ( #3434 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 4a3c7d73b0
2021-07-26 17:43:10 +08:00
be0a6cea40
HBASE-26119 Polish TestAsyncNonMetaRegionLocator ( #3526 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-07-26 17:34:01 +08:00
59e2825c93
HBASE-26093 Replication is stuck due to zero length wal file in oldWALs directory ( #3504 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-07-23 09:33:43 -07:00
729da5a6d3
HBASE-24734 RegionInfo#containsRange should support check meta table ( #3496 ) ( #3517 )
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2021-07-22 18:04:24 +08:00
8bc180c773
HBASE-24984 WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used with multi operation ( #3512 )
...
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2021-07-22 08:30:25 +05:30
cabf0dc51d
HBASE-25986 set default value of normalization enabled from hbase site ( #3476 ) ( #3372 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-07-19 15:26:52 +05:30
55d93fbf2d
HBASE-25973 Balancer should explain progress in a better way in log -… ( #3483 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-07-18 13:04:33 -07:00
fb01a6c276
HBASE-26083 In branch-2 & branch-1 L1 miss metric is always 0 when using CombinedBlockCache ( #3473 )
...
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by Reid Chan <reidchan@apache.org>
2021-07-16 10:28:27 +08:00
692d384ec6
HBASE-26036 DBB released too early in HRegion.get() and dirty data for some operations ( #3436 ) ( #3486 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-07-14 16:30:46 +08:00
12f2a16280
HBASE-26086 TestHRegionReplayEvents do not pass in branch-2 and throws NullPointerException ( #3480 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-07-14 11:37:25 +08:00
dfc9ac8ffb
HBASE-25739 TableSkewCostFunction need to use aggregated deviation - backport
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@duboce.net>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
2021-07-13 08:37:29 -07:00
3294d32f46
HBASE-25700 Enhance znode parent validation when add_peer ( #3458 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2021-07-11 21:15:57 +08:00
b86d2d4020
HBASE-26069 Remove HStore.compactRecentForTestingAssumingDefaultPolic… ( #3462 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-07-07 21:57:59 +08:00
1ba03cae4b
HBASE-26068 The last assertion in TestHStore.testRefreshStoreFilesNotChanged is wrong ( #3461 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org
2021-07-07 14:39:03 +08:00
bde5d98338
HBASE-26065 StripeStoreFileManager does not need to throw IOException for most methods ( #3459 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-07-05 22:19:17 +08:00
342a478753
HBASE-22923 Consider minVersionToMoveSysTables while moving region and creating regionPlan (ADDENDUM) ( #3455 )
...
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-07-03 22:44:46 +05:30
db75ef265e
HBASE-26050 Remove the reflection used in FSUtils.isInSafeMode ( #3445 )
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 82c44b49f5
2021-07-02 14:21:28 +08:00
1381df4c80
HBASE-22923 min version of RegionServer to move system table regions ( #3439 ) ( #3438 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-07-01 16:24:14 +05:30
75cd25386a
HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade ( #3417 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-07-01 15:20:38 +05:30
b90fbb77c6
HBASE-26028 The view as json page shows exception when using TinyLfuBlockCache ( #3420 )
...
(cherry picked from commit 147b030c1f
2021-07-01 10:27:24 +08:00
748e369629
HBASE-26029 It is not reliable to use nodeDeleted event to track region server's death ( #3430 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2021-06-30 11:46:09 +08:00
f827a4c5f8
HBASE-26039 TestReplicationKillRS is useless after HBASE-23956 ( #3440 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2021-06-30 08:27:17 +08:00
166becfd66
HBASE-25980 Master table.jsp pointed at meta throws 500 when no all r… ( #3373 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:30:46 +08:00
c707a9ffff
HBASE-25914 Provide slow/large logs on RegionServer UI ( #3321 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:27:06 +08:00
c7c5d56084
HBASE-26015 Should implement getRegionServers(boolean) method in Asyn… ( #3406 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-27 22:13:21 +08:00
193a94e523
HBASE-26013 Get operations readRows metrics becomes zero after HBASE-25677 ( #3411 )
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-06-24 15:24:50 +08:00
9fe1781037
HBASE-26020 Split TestWALEntryStream.testDifferentCounts out ( #3409 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-06-23 22:51:23 +08:00
c81a5ab565
HBASE-25934 Add username for RegionScannerHolder ( #3325 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-06-23 12:54:41 +05:30
2ae69a883c
HBASE-25877 Add access check for compactionSwitch ( #3253 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-22 23:45:36 +08:00
bda7afe97d
HBASE-25937: Clarify UnknownRegionException ( #3330 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-22 22:44:05 +08:00
12d707c880
HBASE-25992 Addendum add missing catch WALEntryFilterRetryableException back
2021-06-21 23:58:42 +08:00
41a68325c2
HBASE-25698 Fixing IllegalReferenceCountException when using TinyLfuBlockCache ( #3215 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2021-06-21 12:51:59 +05:30
029dc81b39
HBASE-26001 When turn on access control, the cell level TTL of Increment and Append operations is invalid ( #3403 )
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2021-06-21 15:00:50 +08:00
c800e6c825
HBASE-25992 Polish the ReplicationSourceWALReader code for 2.x after HBASE-25596 ( #3376 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-20 16:38:13 +08:00
3b58919246
HBASE-25998: Redo synchronization in SyncFuture ( #3399 )
...
Currently uses coarse grained synchronized approach that seems to
create a lot of contention. This patch
- Uses a reentrant lock instead of synchronized monitor
- Switches to a condition variable based waiting rather than busy wait
- Removed synchronization for unnecessary fields
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 6bafb59642
2021-06-18 10:37:10 -07:00
836fd3c52a
HBASE-25984: Avoid premature reuse of sync futures in FSHLog ( #3371 ) ( #3392 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 5a19bcfa98
2021-06-17 12:27:23 -07:00
26c4e86ebd
HBASE-25976 Implement a master based ReplicationTracker ( #3390 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-06-17 18:38:43 +08:00
4719a2078e
HBASE-25997 NettyRpcFrameDecoder decode request header wrong when han… ( #3380 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-15 14:39:47 +08:00
6f1e652091
HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update ( #3386 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-15 13:55:21 +09:00
cb5f0dfe1e
HBASE-25995 Change the method name for DoubleArrayCost.setCosts ( #3381 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-15 11:55:54 +08:00
a1d59bf1a5
HBASE-25994 Active WAL tailing fails when WAL value compression is enabled ( #3377 )
...
Depending on which compression codec is used, a short read of the
compressed bytes can cause catastrophic errors that confuse the WAL reader.
This problem can manifest when the reader is actively tailing the WAL for
replication. To avoid these issues when WAL value compression is enabled,
BoundedDelegatingInputStream should assume enough bytes are available to
supply a reader up to its bound. This behavior is valid per the contract
of available(), which provides an _estimate_ of available bytes, and
equivalent to IOUtils.readFully but without requiring an intermediate
buffer.
Added TestReplicationCompressedWAL and TestReplicationValueCompressedWAL.
Without the WALCellCodec change TestReplicationValueCompressedWAL will
fail.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-06-14 17:18:53 -07:00
88f031eeec
HBASE-25967 The readRequestsCount does not calculate when the outResu… ( #3351 )
...
* HBASE-25967 The readRequestsCount does not calculate when the outResults is empty
Co-authored-by: Zheng Wang <wangzheng@apache.org>
(cherry picked from commit 329f0baa98
2021-06-10 10:24:18 +08:00
b6837295f3
HBASE-25981 JVM crash when displaying RegionServer UI ( #3364 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-09 18:16:33 +08:00
e4dd4901a8
HBASE-25963 HBaseCluster should be marked as IA.Public ( #3348 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-06-06 21:27:02 +08:00
1267bdb1a4
HBASE-25929 RegionServer JVM crash when compaction ( #3318 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-06-03 17:20:08 +08:00
a4e8ee183e
HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime ( #3302 )
...
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupBase.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java
hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupSystemTable.java
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncConnectionTracing.java
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionLocatorTracing.java
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestManyRegions.java
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/MoveRegionsOfTableAction.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationCrossDiffHdfs.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestEnableRSGroups.java
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupsAdmin2.java
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/CallRunner.java
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractWALRoller.java
hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
hbase-server/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaTableAccessor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaUpdatesGoToPriorityQueue.java
hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
hbase-server/src/test/java/org/apache/hadoop/hbase/backup/TestHFileArchiving.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/RestoreSnapshotFromClientSimpleTestBase.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin2.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestConnection.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClientCloneLinksAfterDelete.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMutationGetCellBuilder.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotMetadata.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterMetricsWrapper.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaAssignmentWithStopMaster.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionStateStore.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/janitor/TestCatalogJanitor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/MasterProcedureSchedulerPerformanceEvaluation.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestCloneSnapshotProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRestoreSnapshotProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/procedure2/store/region/RegionProcedureStorePerformanceEvaluation.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMajorCompaction.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionOpen.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSimpleTimeRangeTracker.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestWideScanner.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestCloseChecker.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestProtobufLog.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/AbstractTestWALReplay.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCombinedAsyncWriter.java
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/master/TestRecoverStandbyProcedure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestFlushSnapshotFromClient.java
hbase-server/src/test/java/org/apache/hadoop/hbase/wal/TestSyncReplicationWALProvider.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java
hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
2021-06-01 12:41:15 -07:00
d4285be5c1
HBASE-25932: Ensure replication reads the trailer bytes from WAL. ( #3332 ) ( #3345 )
...
This bug was exposed by the test from HBASE-25924. Since this wal
implementations close the wal asynchronously, replication can potentially
miss the trailer bytes. (see jira comment for detailed analysis).
While this is not a correctness problem (since trailer does not have any entry data),
it erroneously bumps a metric that is used to track skipped bytes in WAL resulting
in false alarms which is something we should avoid.
Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
(cherry picked from commit b04c3c7786
2021-06-01 08:04:12 -07:00
1a45b743a0
HBASE-25940 Update Compression/TestCompressionTest: LZ4, SNAPPY, LZO ( #3334 )
...
Undo asserts that LZ4 and SNAPPY fails if their native libs are NOT
loaded; as of hadoop 3.3.1, LZ4 and SNAPPY can work w/o native libs.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-05-29 09:04:08 -07:00
5157521213
HBASE-25927: Fix the log messages by not stringifying the exceptions in log ( #3338 )
...
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 9a2027bf71
2021-05-28 21:34:14 -07:00
2a84fd40d3
HBASE-25938 The SnapshotOfRegionAssignmentFromMeta.initialize call in FavoredNodeLoadBalancer is just a dummy one ( #3329 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-29 11:10:52 +08:00
bbe9558040
HBASE-25745 Deprecate/Rename config `hbase.normalizer.min.region.count` to `hbase.normalizer.merge.min.region.count`
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-28 13:10:20 -07:00
5aa39a975c
HBASE-25910 - Fix port assignment test ( #3308 )
...
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-28 20:53:20 +05:30
cad9ae386a
HBASE-25933 Log trace raw exception, instead of cause message in NettyRpcServerRequestDecoder ( #3323 )
...
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2021-05-27 20:03:17 +01:00
3cf0678aee
HBASE-25926 Cleanup MetaTableAccessor references in FavoredNodeBalancer related code ( #3313 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-05-27 16:25:01 +08:00
e265eccf20
HBASE-25924 Re-compute size of WAL file while removing from WALEntryStream ( #3315 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2021-05-26 10:42:03 -07:00
563ebc214a
HBASE-25534 Honor TableDescriptor settings earlier in normalization ( #2917 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-25 08:24:05 -07:00
2cb6cc8a6c
HBASE-25894 Improve the performance for region load and region count related cost functions ( #3276 )
...
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
2021-05-25 18:07:33 +08:00
8754e88fa7
HBASE-25906 UI of master-status to show recent history of balancer desicion ( #3303 ) ( #3296 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2021-05-25 12:39:47 +05:30
073f23e2bd
HBASE-25513 When the table is turned on normalize, the first region may not be merged even the size is 0 ( #2887 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-05-24 17:27:06 -07:00
chenglei
Norbert Kalmar
Nick Dimiduk
Josh Elser
bsglz
Bryan Beaudreault
Guanghao Zhang
LiangJun He
RamanChodzka
Tomu Tsuruhara
Rushabh Shah
Duo Zhang
Yutong Xiao
liangxs
Andrew Purtell
Xiaolin Ha
zhengzhuobinzzb
ss77892
Zhuoyue Huang
clarax
Xiaolin Ha
XinSun
Anoop Sam John
Sourabh Dhanotia
binlijin
Viraj Jasani
KulwantSingh011
Tak Lon (Stephen) Wu
imbajin
GeorryHuang
richardantal
bitterfox
meiyi
hapihu
Josh Elser
keyserbrian1
Pankaj
lpalasthy
Michael Stack
Wellington Ramos Chevreuil
eomiks
stack
lzx404243
Richard Marscher
caroliney14
Ruanhui
liever18
Wei-Chiu Chuang
gkanade
Aman Poonia
mokai
YutSean
litao
lujiefsi
belugabehr
Bharath Vissapragada
Toshihiro Suzuki
Sandeep Pal
Baiqiang Zhao
Victor