Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,054 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

9048 Commits

Author SHA1 Message Date
imbajin f65b769e8c
HBASE-24601: Change default Hfile storage policy from HOT to NONE for HDFS (#1940) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-10-02 16:35:14 +08:00
Yutong Xiao d3c86f4f90
HBASE-26305 Move NavigableSet add operation to writer thread in BucketCache (#3705) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: chenglei <chenglei@apache.org>
 2021-10-01 20:57:51 +08:00
chenglei 121bdea230
HBASE-26295 BucketCache could not free BucketEntry which restored fro… (#3699) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-29 21:24:12 +08:00
Duo Zhang 1152a61b5b
HBASE-26220 Use P2P communicate between region servers to sync the list for bootstrap node (#3697) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-09-29 21:12:58 +08:00
richardantal f000b77532
HBASE-26289 Hbase scan setMaxResultsPerColumnFamily not giving right results (#3688) 
Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-09-29 10:03:53 +02:00
Duo Zhang 113cc2a591
HBASE-26293 Use reservoir sampling when selecting bootstrap nodes (#3702) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-09-28 23:27:47 +08:00
bitterfox 748a6827c4
HBASE-26238 Short message by Result#compareResults for VerifyReplication to avoid OOME (#3647) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-28 21:48:32 +08:00
GeorryHuang 63306942b1
HBASE-26251 StochasticLoadBalancer metrics should update even if balancer doesn't run (#3678) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
 2021-09-28 21:20:48 +08:00
Tak Lon (Stephen) Wu d956828aac HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job (#3684) 
Introduce `hfile.onheap.block.cache.fixed.size`
and default to disable. when using ClientSideRegionScanner
it will be enabled with a fixed size for caching
INDEX/LEAF_INDEX block when a client, e.g.
snapshot scanner, scans the entire HFile
and does not need to seek/reseek to index
block multiple times.

Signed-off-by: Josh Elser <elserj@apache.org>
 2021-09-22 09:26:57 -07:00
Tak Lon (Stephen) Wu ecd35f77ae
Revert "HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job (#3684)" (#3695) 
This reverts commit 6556a5ee91.
 2021-09-22 09:19:05 -07:00
Tak Lon (Stephen) Wu 6556a5ee91
HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job (#3684) 
Introduce `hfile.onheap.block.cache.fixed.size`
and default to disable. when using ClientSideRegionScanner
it will be enabled with a fixed size for caching 
INDEX/LEAF_INDEX block when a client, e.g. 
snapshot scanner, scans the entire HFile
and does not need to seek/reseek to index
block multiple times.
 2021-09-22 09:17:18 -07:00
meiyi 96fa015043
HBASE-26261 Store configuration loss when use update_config (#3664) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-22 18:22:30 +08:00
meiyi e6508f6776 HBASE-26270 Provide getConfiguration method for Region and Store interface (#3671) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
 2021-09-22 11:10:04 +08:00
meiyi 2599bd8eab Revert "Provide getConfiguration method for Region and Store interface (#3671)" 
This reverts commit 4f6e0cf601.
 2021-09-22 11:08:51 +08:00
meiyi 4f6e0cf601
Provide getConfiguration method for Region and Store interface (#3671) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
 2021-09-22 10:23:25 +08:00
Duo Zhang a9abdf4936
HBASE-26288 Revisit the usage of MetaTableLocator when HRegionServer.TEST_SKIP_REPORTING_TRANSITION is true (#3689) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-09-22 00:00:35 +08:00
Duo Zhang dd58fdb4ba HBASE-26285 Addendum fix typo in table.jsp 2021-09-21 23:58:53 +08:00
Duo Zhang 4ea23c3baf
HBASE-26285 Remove MetaTableLocator usages in non-migration code (#3686) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-09-21 22:22:00 +08:00
chenglei 4614c7bc57
HBASE-26281 DBB got from BucketCache would be freed unexpectedly before RPC completed (#3680) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-17 22:15:23 +08:00
Duo Zhang d26bcaaa91
HBASE-25288 Make MasterRpcServices not extends RSRpcServices and also HMaster not extends HRegionServer (#3612) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-09-13 09:56:13 +08:00
Xiaolin Ha 8d2b995352
HBASE-26229 Limit count and size of L0 files compaction in StripeCompactionPolicy (#3646) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-09 18:26:37 +08:00
hapihu 922d0666a3
[HBASE-26243]Fix typo for file "hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java" (#3650) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-09 12:11:49 +08:00
caroliney14 03b6f2d3f2
HBASE-26163 Better logging in RSGroupInfoManagerImpl (#3610) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bharath Vissapragada <bharathv@apache.org>
 2021-09-07 22:20:09 +08:00
Pankaj b7c6a0b637
HBASE-26228 updateRSGroupConfig operation should be authorized by AccessController (#3633) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
 2021-09-07 22:17:26 +08:00
Duo Zhang dd293c8ba9
HBASE-26230 Start an in process HRegionServer in maintenance mode (#3659) 
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
 2021-09-03 17:07:30 +08:00
Rushabh Shah ec747bcb29
HBASE-26106 AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory. (#3636) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-02 09:16:54 -07:00
Bryan Beaudreault a15e94a47f HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed 
Closes #3630

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org
 2021-09-01 18:01:13 -04:00
XinSun 345d7256c8
HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky (#3651) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-01 18:43:25 +08:00
chenglei 36884c3dff
HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore (#3604) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-01 15:50:51 +08:00
Andrew Purtell d19728b80c
HBASE-25642 Fix or stop warning about already cached block (#3638) 
Our logs have as a fairly common occurrence: 2021-03-05 22:24:31,034 WARN
[StoreFileOpener-foo-1] hfile.BlockCacheUtil: Caching an already cached
block: blah.bub. This is harmless and can happen in rare cases (see HBASE-8547)

Because it is harmless, log at DEBUG level, not WARN.

Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
 2021-08-31 09:25:31 -07:00
Duo Zhang 889049eab6
HBASE-26235 We could start RegionServerTracker before becoming active master (#3645) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-08-30 12:14:10 +08:00
keyserbrian1 c4daabd9c5
HBASE-26232 SEEK_NEXT_USING_HINT is ignored on reversed Scans (#3639) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-29 21:13:17 +08:00
GeorryHuang 00981643d2
HBASE-26216 Move HRegionServer.abort(String) to Abortable as a default method (#3642) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-28 22:23:23 +08:00
Andrew Purtell 07c0eeebb3 HBASE-26189 Reduce log level of CompactionProgress notice to DEBUG (#3579) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-08-27 10:13:36 -07:00
Andrew Purtell 7cc876de52 Revert "HBASE-26189 Reduce log level of CompactionProgress notice to DEBUG (#3579)" 
This reverts commit c770b1de0d.

GH workflow error caused commit with incorrect message.
 2021-08-27 10:13:34 -07:00
Andrew Purtell c770b1de0d
HBASE-26189 Reduce log level of CompactionProgress notice to DEBUG (#3579) 2021-08-27 10:11:16 -07:00
Pankaj 11b32dbfd3
HBASE-26227 Forward port HBASE-26223 test code to branch-2.4+ (#3632) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-27 16:42:27 +08:00
lpalasthy 735bcf85e9
HBASE-26219 Negative time is logged while waiting on regionservers (#3622) 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-25 16:24:32 +02:00
Duo Zhang 137c7dcd3d
HBASE-26214 Introduce a ConnectionRegistryEndpoint interface (#3613) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-25 19:20:10 +08:00
Duo Zhang 91db10a8bc
HBASE-26180 Introduce a initial refresh interval for RpcConnectionRegistry (#3601) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2021-08-25 18:42:04 +08:00
Duo Zhang 1fb78a3302
HBASE-26213 Refactor AnnotationReadingPriorityFunction (#3614) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2021-08-25 18:36:29 +08:00
GeorryHuang 4b0a64033a
HBASE-26215 The backup master status page should use ActiveMasterManager instead of MasterAddressTracker (#3621) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-25 16:41:42 +08:00
Duo Zhang 2ce2f9336f
HBASE-26173 Return only a sub set of region servers as bootstrap nodes (#3599) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-24 23:04:34 +08:00
Wellington Ramos Chevreuil 6bb5701783
HBASE-26184 TestTableSnapshotScanner.testMergeRegion error message is… (#3572) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-20 10:45:16 +01:00
Duo Zhang d4593ce8c5
HBASE-26181 Region server and master could use itself as ConnectionRegistry (#3603) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-20 15:35:28 +08:00
Duo Zhang c8d9d4df80
HBASE-26172 Deprecated MasterRegistry (#3566) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-19 16:16:35 +08:00
Wellington Ramos Chevreuil f2e2140b57
Amend HBASE-26187 UTs, adding a check for Split procedure completion before proceeding with test logic. (#3598) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-18 15:39:00 +01:00
Xiaolin Ha 33e15a261d HBASE-26087 JVM crash when displaying RPC params by MonitoredRPCHandler (#3489) 
Signed-off-by: stack <stack@apache.org>
 2021-08-18 14:52:01 +08:00
Xiaolin Ha 10c837f0e4
HBASE-26087 JVM crash when displaying RPC params by MonitoredRPCHandler (#3489) 
Signed-off-by: stack <stack@apache.org>
 2021-08-18 13:28:46 +08:00
Wellington Ramos Chevreuil 721cb96f8c
HBASE-26187 Write straight into the store directory when Splitting an… (#3574) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-17 09:46:00 +01:00
Duo Zhang cacf4a86a6
HBASE-26193 Do not store meta region location as permanent state on zookeeper (#3583) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
 2021-08-17 15:31:50 +08:00
eomiks 356cfe9351 HBASE-24652 master-status UI make date type fields sortable 2021-08-16 16:03:50 -07:00
stack 19f5ad5e30 HBASE-26200 Revert "HBASE-25165 Change 'State time' in UI so sorts (#2508)" 
This reverts commit 8a995ae81b.
 2021-08-16 13:50:37 -07:00
lzx404243 44d5624908
HBASE-25680 Non-idempotent test in TestReplicationHFileCleaner (#2984) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-14 11:00:52 +08:00
chenglei 11222fc4df
HBASE-26026 HBase Write may be stuck forever when using CompactingMemStore (#3421) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-12 23:11:19 +08:00
Xiaolin Ha 238c9b40bf
HBASE-26155 JVM crash when scan (#3553) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-12 17:09:18 +08:00
Rushabh Shah d307a8f2e8
HBASE-26185 Return mutable list in AssignmentManager#getExcludedServersForSystemTable (#3577) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-08-12 13:46:22 +05:30
Viraj Jasani 5e8a269b1a
HBASE-26185 Fix TestMaster#testMoveRegionWhenNotInitialized in the presence of hbase.min.version.move.system.tables (#3573) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-08-10 10:57:04 +05:30
Richard Marscher 7f614ce77e
HBASE-6908: Add pluggable rpc queue implementation (#3522) 
Can pass in a FQCN to load as the call queue implementation.

Standardized arguments to the constructor are the max queue length, the
PriorityFunction, and the Configuration.

`PluggableBlockingQueue` abstract class provided to help guide the
correct constructor signature

Hard fails if the class fails to load as a `BlockingQueue<CallRunner>`

Signed-off-by: stack <stack@apache.org>
 2021-08-09 12:54:18 -07:00
Duo Zhang a79a9cc53e
HBASE-26179 TestRequestTooBigException spends too much time to finish (#3571) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-08-07 23:01:55 +08:00
Xiaolin Ha c9383f2c89
HBASE-26170 handleTooBigRequest in NettyRpcServer didn't skip enough bytes (#3564) 
Signed-off-by: stack <stack@apache.com>
 2021-08-05 08:05:26 -07:00
chenglei d4aed4d59e
HBASE-26142 NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero (#3531) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-05 20:48:18 +08:00
caroliney14 83661c5636
HBASE-25469 Add detailed RIT info in JSON format for consumption as metrics (#3535) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-04 18:04:13 -07:00
Duo Zhang 63d4970de4
HBASE-26150 Let region server also carry ClientMetaService (#3550) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-04 23:44:10 +08:00
Ruanhui 73a0411bb9
HBASE-26166 Fix a minor bug of table list on master ui (#3554) 
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
 2021-08-04 20:27:44 +08:00
Bryan Beaudreault 2e9ab3ce94
HBASE-26154: Adds exception metrics for QuotaExceededException and RpcThrottlingException (#3545) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-08-02 09:50:17 +05:30
Duo Zhang b248730126
HBASE-26151 Reimplement MasterAddressTracker to also cache backup master addresses (#3548) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-31 15:39:08 +08:00
Duo Zhang 8fbc2d2400
HBASE-26144 The HStore.snapshot method is never called in main code (#3533) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-31 14:33:46 +08:00
YutSean ed4b51a6b7
HBASE-26105 Rectify the expired TODO comment in CombinedBC. (#3509) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-30 21:34:14 +08:00
Duo Zhang 5f0950558f
HBASE-26096 Cleanup the deprecated methods in HBTU related classes and format code (#3503) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-29 10:18:38 +08:00
Duo Zhang 02d263e7dd
HBASE-26118 The HStore.commitFile and HStore.moveFileIntoPlace almost have the same logic (#3525) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-26 20:58:16 +08:00
Wei-Chiu Chuang 4a3c7d73b0
HBASE-21946 Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable (#3434) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-26 02:31:39 -07:00
Wei-Chiu Chuang c74366c498
HBASE-26049 Remove DfsBuilderUtility (#3444) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-26 02:30:22 -07:00
Duo Zhang f0324a7516
HBASE-26119 Polish TestAsyncNonMetaRegionLocator (#3526) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2021-07-26 16:57:49 +08:00
GeorryHuang 8ae394285a
HBASE-26091 Remove FirstKeyValueMatchingQualifiersFilter (#3497) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-25 21:12:20 +08:00
Rushabh Shah 0294c73f1f
HBASE-26093 Replication is stuck due to zero length wal file in oldWALs directory (#3504) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-23 09:32:55 -07:00
Peter Somogyi 3c70bc1f31
HBASE-26107 MOB compaction with missing files catches incorrect exception (#3511) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2021-07-22 10:16:03 +02:00
meiyi 9e27de6aed
HBASE-24734 RegionInfo#containsRange should support check meta table (#3496) 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2021-07-22 16:07:52 +08:00
gkanade 21c4578f22
HBASE-24984 WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used with multi operation (#3495) 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-07-21 12:18:54 +05:30
Duo Zhang 16721239e7
HBASE-26100 Set version as 3.0.0-alpha-2-SNAPSHOT in master (#3508) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-20 23:04:08 +08:00
Aman Poonia 0f313176be
HBASE-25986 set default value of normalization enabled from hbase site (#3372) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-07-19 15:25:36 +05:30
Duo Zhang d30cc27097
HBASE-26081 Copy HBTU to hbase-testing-util, rename the HBTU related classes in hbase-server and mark them as IA.LimitedPrivate (#3478) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-19 09:29:08 +08:00
Duo Zhang 83d1bf1667
HBASE-26090 Remove the deprecated methods in Scan which should be removed in 3.0.0 (#3493) 
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
 2021-07-16 22:27:22 +08:00
YutSean 3666357ba0
HBASE-26083 L1 miss metric is incorrect when using CombinedBlockCache (#3491) 
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by Reid Chan <reidchan@apache.org>
 2021-07-16 10:25:56 +08:00
Xiaolin Ha 0836695459
HBASE-26036 DBB released too early in HRegion.get() and dirty data for some operations (#3436) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-14 11:26:38 +08:00
Duo Zhang 9f21eec777
HBASe-26082 Mark LocalHBaseCluster as IA.Private (#3471) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2021-07-13 15:14:50 +08:00
clarax 1e763d521f
HBASE-25973 Balancer should explain progress in a better way in log (#3356) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2021-07-12 10:10:24 -07:00
mokai bb84892550
HBASE-25700 Enhance znode parent validation when add_peer (#3458) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2021-07-11 20:45:25 +08:00
Duo Zhang e65fc9226f
HBASE-26069 Remove HStore.compactRecentForTestingAssumingDefaultPolic… (#3462) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-07-07 21:34:52 +08:00
Wei-Chiu Chuang 29cd782d25
HBASE-25516 [JDK17] reflective access Field.class.getDeclaredField("modifiers") not supported (#3443) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-07 16:00:57 +08:00
Duo Zhang 7e5a0dbebf
HBASE-26068 The last assertion in TestHStore.testRefreshStoreFilesNotChanged is wrong (#3461) 
Signed-off-by: Yulin Niu <niuyulin@apache.org
 2021-07-07 14:34:42 +08:00
Duo Zhang a3ad97fbfc
HBASE-26065 StripeStoreFileManager does not need to throw IOException for most methods (#3459) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-07-05 22:16:00 +08:00
Viraj Jasani 1883889e26
HBASE-22923 Consider minVersionToMoveSysTables while moving region and creating regionPlan (ADDENDUM) (#3455) 
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-07-03 22:41:28 +05:30
Wei-Chiu Chuang 82c44b49f5
HBASE-26050 Remove the reflection used in FSUtils.isInSafeMode (#3445) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-07-01 23:19:03 -07:00
Wei-Chiu Chuang fab0505257
HBASE-26051 Remove reflections used to access HDFS EC APIs (#3446) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-02 11:10:52 +08:00
Duo Zhang 5118321ec9
HBASE-26059 Set version as 3.0.0-alpha-1 in master in prep for first RC of 3.0.0-alpha-1 (#3453) 
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2021-07-02 07:50:41 +08:00
Viraj Jasani 84f9900c99
HBASE-22923 min version of RegionServer to move system table regions (#3439) (#3438) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-07-01 16:02:37 +05:30
Viraj Jasani 4c7da496ad
HBASE-25902 Add missing CFs in meta during HBase 1 to 2 Upgrade (#3441) (#3417) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-01 15:13:25 +05:30
bsglz 147b030c1f
HBASE-26028 The view as json page shows exception when using TinyLfuBlockCache (#3420) 2021-06-30 11:36:00 +08:00
Duo Zhang 51893b9ba3
HBASE-26029 It is not reliable to use nodeDeleted event to track region server's death (#3430) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2021-06-30 08:44:19 +08:00
Duo Zhang 64d4915ca8
HBASE-26039 TestReplicationKillRS is useless after HBASE-23956 (#3440) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-06-30 08:00:17 +08:00
GeorryHuang 22ec681ad9
HBASE-25980 Master table.jsp pointed at meta throws 500 when no all r… (#3374) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-27 22:30:28 +08:00
GeorryHuang fb4af2a8bf
HBASE-25914 Provide slow/large logs on RegionServer UI (#3319) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2021-06-27 22:26:35 +08:00
GeorryHuang e6eb65733a
HBASE-26015 Should implement getRegionServers(boolean) method in Asyn… (#3406) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-27 21:58:18 +08:00
Duo Zhang 39d143f290
HBASE-26020 Split TestWALEntryStream.testDifferentCounts out (#3409) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-06-23 22:46:07 +08:00
litao fa2d127b7f
HBASE-25934 Add username for RegionScannerHolder (#3325) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-06-23 12:34:43 +05:30
lujiefsi d9bd29603b
HBASE-25877 Add access check for compactionSwitch (#3253) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-22 23:37:36 +08:00
belugabehr d44292ac1a
HBASE-25937: Clarify UnknownRegionException (#3330) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-22 22:36:30 +08:00
YutSean f640eef924
HBASE-26013 Get operations readRows metrics becomes zero after HBASE-25677 (#3404) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-06-22 11:38:42 +08:00
Duo Zhang c5461aaa5b HBASE-25992 Addendum add missing catch WALEntryFilterRetryableException back 2021-06-21 23:58:08 +08:00
Viraj Jasani 9f4177f7b4
HBASE-25698 Fixing IllegalReferenceCountException when using TinyLfuBlockCache (#3215) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-06-21 11:50:33 +05:30
Duo Zhang d2923755ec
HBASE-25992 Polish the ReplicationSourceWALReader code for 2.x after HBASE-25596 (#3376) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-20 16:32:42 +08:00
YutSean 53f61ef8d5
HBASE-26001 When turn on access control, the cell level TTL of Increment and Append operations is invalid (#3385) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-06-18 11:15:45 +08:00
Bharath Vissapragada 336d8464cc
HBASE-25998: Redo synchronization in SyncFuture (#3382) 
Currently uses coarse grained synchronized approach that seems to
create a lot of contention. This patch

- Uses a reentrant lock instead of synchronized monitor
- Switches to a condition variable based waiting rather than busy wait
- Removed synchronization for unnecessary fields

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-06-17 12:21:05 -07:00
Duo Zhang eb242be674
HBASE-25976 Implement a master based ReplicationTracker (#3390) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-06-17 18:24:49 +08:00
Bharath Vissapragada 5a19bcfa98
HBASE-25984: Avoid premature reuse of sync futures in FSHLog (#3371) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-06-16 14:30:15 -07:00
binlijin 8f618a0846
HBASE-25997 NettyRpcFrameDecoder decode request header wrong when han… (#3380) 
* HBASE-25997 NettyRpcFrameDecoder decode request header wrong when handleTooBigRequest
 2021-06-15 14:25:18 +08:00
Toshihiro Suzuki 4262887432 HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update (addendum) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-15 09:42:49 +09:00
Andrew Purtell 97f90e0be2
HBASE-25994 Active WAL tailing fails when WAL value compression is enabled (#3377) 
Depending on which compression codec is used, a short read of the
compressed bytes can cause catastrophic errors that confuse the WAL reader.
This problem can manifest when the reader is actively tailing the WAL for
replication. To avoid these issues when WAL value compression is enabled,
BoundedDelegatingInputStream should assume enough bytes are available to
supply a reader up to its bound. This behavior is valid per the contract
of available(), which provides an _estimate_ of available bytes, and
equivalent to IOUtils.readFully but without requiring an intermediate
buffer.

Added TestReplicationCompressedWAL and TestReplicationValueCompressedWAL.
Without the WALCellCodec change TestReplicationValueCompressedWAL will
fail.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-06-14 17:16:31 -07:00
Toshihiro Suzuki a35ec994b9 HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update (addendum) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-15 08:03:30 +09:00
Toshihiro Suzuki ec31818574
HBASE-26002 MultiRowMutationEndpoint should return the result of the conditional update (#3384) 2021-06-15 07:43:27 +09:00
bsglz 329f0baa98
HBASE-25967 The readRequestsCount does not calculate when the outResu… (#3351) 
* HBASE-25967 The readRequestsCount does not calculate when the outResults is empty

Co-authored-by: Zheng Wang <wangzheng@apache.org>
 2021-06-10 09:37:31 +08:00
Xiaolin Ha 471e8159f0
HBASE-25981 JVM crash when displaying RegionServer UI (#3364) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-09 18:11:18 +08:00
GeorryHuang 40a3d57628
HBASE-22708 Remove the deprecated methods in Hbck interface (#3362) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-08 09:33:08 +08:00
Duo Zhang eddf4cc3a1
HBASE-25963 HBaseCluster should be marked as IA.Public (#3348) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-06-06 21:20:59 +08:00
Peter Somogyi de06e20e0a
HBASE-25970 MOB data loss - incorrect concatenation of MOB_FILE_REFS (#3355) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-06-05 08:57:26 +02:00
meiyi 4671cb1801
HBASE-25929 RegionServer JVM crash when compaction (#3318) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-06-03 17:17:17 +08:00
xijiawen 426c3c16f3
HBASE-25799 add clusterReadRequests and clusterWriteRequests jmx (#3188) 
* HBASE-25799 add clusterReadRequests and clusterWriteRequests jmx

Co-authored-by: stevenxi <stevenxi@tencent.com>
 2021-06-03 15:48:03 +08:00
Andrew Purtell 335305e0cf
HBASE-25911 Replace calls to System.currentTimeMillis with EnvironmentEdgeManager.currentTime (#3302) 
We introduced EnvironmentEdgeManager as a way to inject alternate clocks
for unit tests. In order for this to be effective, all callers that would
otherwise use System.currentTimeMillis() must call
EnvironmentEdgeManager.currentTime() instead, except the implementers of
EnvironmentEdge.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-06-01 09:57:48 -07:00
Bharath Vissapragada 4fb0861214
HBASE-25932 addendum: Add test comments. (#3344) 
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
 2021-06-01 08:03:25 -07:00
Bharath Vissapragada b04c3c7786
HBASE-25932: Ensure replication reads the trailer bytes from WAL. (#3332) 
This bug was exposed by the test from HBASE-25924. Since this wal
implementations close the wal asynchronously, replication can potentially
miss the trailer bytes. (see jira comment for detailed analysis).

While this is not a correctness problem (since trailer does not have any entry data),
it erroneously bumps a metric that is used to track skipped bytes in WAL resulting
in false alarms which is something we should avoid.

Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
 2021-05-31 22:12:47 -07:00
Duo Zhang 06c6e06803
HBASE-25916 Move FavoredNodeLoadBalancer to hbase-balancer module (#3327) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-31 22:55:04 +08:00
Duo Zhang f2ff816532
HBASE-25939 Move more tests code for StochasticLoadBalancer to hbase-balancer module (#3331) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-30 22:00:18 +08:00
Michael Stack f119a865cf
HBASE-25940 Update Compression/TestCompressionTest: LZ4, SNAPPY, LZO (#3334) 
Undo asserts that LZ4 and SNAPPY fails if their native libs are NOT
loaded; as of hadoop 3.3.1, LZ4 and SNAPPY can work w/o native libs.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-29 09:02:52 -07:00
Sandeep Pal 9a2027bf71
HBASE-25927: Fix the log messages by not stringifying the exceptions in log (#3338) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-05-28 21:28:19 -07:00
Duo Zhang c1d299fc1d
HBASE-25938 The SnapshotOfRegionAssignmentFromMeta.initialize call in FavoredNodeLoadBalancer is just a dummy one (#3329) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-29 11:04:41 +08:00
Victor 3f7d2897a1
HBASE-25910 - Fix port assignment test (#3308) 
Signed-off-by: David Manning <david.manning@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-28 20:52:07 +05:30
Duo Zhang 7218c83f81
HBASE-25931 Move FavoredNodeManager to hbase-balancer module (#3324) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-05-28 15:57:59 +08:00
Duo Zhang ed8df5eded
HBASE-25758 Move MetaTableAccessor out of hbase-balancer module (#3309) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-28 09:19:07 +08:00
Wellington Ramos Chevreuil feb89d988b
HBASE-25933 Log trace raw exception, instead of cause message in NettyRpcServerRequestDecoder (#3323) 
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2021-05-27 19:54:25 +01:00
Duo Zhang 63141bf576
HBASE-25926 Cleanup MetaTableAccessor references in FavoredNodeBalancer related code (#3313) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-27 16:05:14 +08:00
Rushabh Shah a22e418cf6
HBASE-25924 Re-compute size of WAL file while removing from WALEntryStream (#3314) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-05-26 10:40:44 -07:00
Duo Zhang 76fbb8b965
HBASE-25818 Move StochasticLoadBalancer to hbase-balancer module (#3206) 
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
 2021-05-25 23:24:35 +08:00
Duo Zhang 6a77872879
HBASE-25894 Improve the performance for region load and region count related cost functions (#3276) 
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
 2021-05-25 18:04:06 +08:00
GeorryHuang 36affdaa8e
HBASE-25906 UI of master-status to show recent history of balancer desicion (#3296) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-25 11:08:48 +05:30
Baiqiang Zhao 21aa553bc1
HBASE-25745 Deprecate/Rename config `hbase.normalizer.min.region.count` to `hbase.normalizer.merge.min.region.count` 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-05-24 13:03:27 -07:00
Anoop Sam John f53ceeecb0
HBASE-25898 RS getting aborted due to NPE in Replication WALEntryStream (#3292) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
 2021-05-24 23:41:45 +05:30
Xiaolin Ha b02c8102b7
HBASE-25899 Improve efficiency of SnapshotHFileCleaner (#3280) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-24 22:15:47 +08:00
Duo Zhang f94f4e29fe
HBASE-25873 Refactor and cleanup the code for CostFunction (#3274) 
Signed-off-by: Yi Mei <myimeiyi@gmail.com>
 2021-05-24 18:14:55 +08:00
Xiaolin Ha 7f6b778c14
HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky (#3160) 2021-05-22 21:56:17 +08:00
caoliqing edde01c605
HBASE-25892: 'False' should be 'True' in auditlog of listLabels (#3273) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-22 11:22:51 +08:00
Andrew Purtell 8ec6fd9459
HBASE-25869 WAL value compression (#3244) 
WAL storage can be expensive, especially if the cell values
represented in the edits are large, consisting of blobs or
significant lengths of text. Such WALs might need to be kept around
for a fairly long time to satisfy replication constraints on a space
limited (or space-contended) filesystem.

We have a custom dictionary compression scheme for cell metadata that
is engaged when WAL compression is enabled in site configuration.
This is fine for that application, where we can expect the universe
of values and their lengths in the custom dictionaries to be
constrained. For arbitrary cell values it is better to use one of the
available compression codecs, which are suitable for arbitrary albeit
compressible data.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-05-21 11:05:52 -07:00
Rushabh Shah dfa88e1ffe
HBASE-25827 Per Cell TTL tags get duplicated with increments causing tags length overflow (#3210) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Aman Poonia <apoonia@salesforce.com>
 2021-05-21 22:56:11 +05:30
Baiqiang Zhao a1177b3e91
HBASE-25682 Add a new command to update the configuration of all RSs in a RSGroup (#3080) 
* HBASE-25682 Add a new command to update the configuration of all RSs in a RSGroup

Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-05-21 22:49:25 +05:30
Sandeep Pal 15e861169f
HBASE-25848: Add flexibility to backup replication in case replication filter throws an exception (#3283) 
* HBASE-25848: Add flexibility to backup replication in case replication filter throws an exception
 2021-05-20 13:31:44 -07:00
Duo Zhang 7c24ed4f45
HBASE-25897 TestRetainAssignmentOnRestart is flaky after HBASE-25032 (#3281) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-05-20 20:58:53 +08:00
GeorryHuang 5b9940907e
HBASE-25791 UI of master-status to show a recent history of that why balancer was rejected to run (#3275) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-19 12:05:29 +08:00
Duo Zhang 741b4b4674
HBASE-25032 Do not assign regions to region server which has not called regionServerReport yet (#3268) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-05-18 08:08:03 +08:00
Pankaj 2126ec94f0
HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager (#3250) 
* HBASE-25875 RegionServer failed to start with IllegalThreadStateException due to race condition in AuthenticationTokenSecretManager's start & retrievePassword method

Signed-off-by: stack <stack@apache.com>
 2021-05-17 12:17:24 +05:30
Baiqiang Zhao d69d5c24b1
HBASE-25861 Correct the usage of Configuration#addDeprecation (#3249) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-05-14 09:31:06 -07:00
Viraj Jasani 0955a7a22e
HBASE-25884 Return empty records for disabled balancer in-memory queue (#3263) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Aman Poonia <apoonia@salesforce.com>
 2021-05-14 12:54:07 +05:30
Michael Stack 630c73fda4 HBASE-25867 Extra doc around ITBLL (#3242) 
* HBASE-25867 Extra doc around ITBLL

Minor edits to a few log messages.
Explain how the '-c' option works when passed to ChaosMonkeyRunner.
Some added notes on ITBLL.
Fix whacky 'R' and 'Not r' thing in Master (shows when you run ITBLL).
In HRS, report hostname and port when it checks in (was debugging issue
where Master and HRS had different notions of its hostname).
Spare a dirty FNFException on startup if base dir not yet in place.

* Address Review by Sean

Signed-off-by: Sean Busbey <busbey@apache.org>
 2021-05-11 19:26:57 +01:00
Duo Zhang 29bd3dd586
HBASE-25852 Move all the intialization work of LoadBalancer implementation to initialize method (#3248) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-11 22:03:33 +08:00
Catalin Luca 2b6a91a1da
HBASE-25859 Reference class incorrectly parses the protobuf magic marker (#3236) 
Co-authored-by: Catalin Luca <luca@adobe.com>
Signed-off-by: stack <stack@apache.org>
 2021-05-10 12:45:23 -07:00
Rushabh Shah 8c2332d465
HBASE-25860 Add metric for successful wal roll requests. (#3238) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-08 12:58:29 +05:30
Duo Zhang c2a1d31270 HBASE-25774 Addendum fix compile error 2021-05-08 13:56:57 +08:00
Andrew Purtell 02b018cf1a HBASE-25774 ServerManager.getOnlineServer may miss some region servers when refreshing state in some procedure implementations 
Revert "HBASE-25032 Wait for region server to become online before adding it to online servers in Master (#2769)"

This reverts commit 1e4639d2eb.

Conflicts:

	hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
 2021-05-07 18:21:46 -07:00
niuyulin 6cfff27465
HBASE-25837 TestRollingRestart is flaky (#3220) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-07 18:58:45 +08:00
Andrew Purtell 6309c090b5
HBASE-25854 Remove redundant AM in-memory state changes in CatalogJanitor (#3234) 
In CatalogJanitor we schedule GCRegionProcedure to clean up both
filesystem and in-memory state after a split, and
GCMultipleMergedRegionsProcedure to do the same for merges. Both of these
procedures clean up in-memory state, but CatalogJanitor also does this
redundantly just after scheduling the procedures. The cleanup should be
done in only one place. Presumably we are using the procedures to do it in
a principled way. Remove the redundancy in CatalogJanitor and fix any
follow on issues, like test failures.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-06 09:13:33 -07:00
Duo Zhang ba4cb91211
HBASE-25851 Make LoadBalancer not extend Configurable interface (#3233) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-06 16:11:46 +08:00
Andrew Purtell cc88cf0ecf
HBASE-25847 More DEBUG and TRACE level logging in CatalogJanitor and HbckChore (#3230) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-05-05 17:01:00 -07:00
Nick Dimiduk eb9b54304e HBASE-25843 move master http-related code into o.a.h.h.master.http 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-05 08:34:37 -07:00
Duo Zhang 90f986497b
HBASE-25834 Remove balanceTable method from LoadBalancer interface (#3217) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-05 15:48:01 +08:00
Nick Dimiduk 17193dae58 HBASE-25842 move regionserver http-related code into o.a.h.h.regionserver.http 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-04 15:40:17 -07:00
Andrew Purtell 432d141474
HBASE-25835 Ignore duplicate split requests from regionserver reports (#3218) 
Processing of the RS report happens asynchronously from other activities
which can mutate region state. For example, a split procedure may already
be running. A split procedure cannot succeed if the parent region is no
longer open, so we can ignore it in that case.

Note that submitting more than one split procedure for a given region is
harmless -- the split is fenced in the procedure handling -- but it would
be noisy in the logs. Only one procedure can succeed. The other
procedure(s) would abort during initialization and report failure with
WARN level logging.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
 2021-05-04 10:05:29 -07:00
Andrew Purtell fda324b116
HBASE-25836 RegionStates#getAssignmentsForBalancer should only care about OPEN or OPENING regions (#3219) 
RegionStates#getAssignmentsForBalancer is used by the HMaster to
collect all regions of interest to the balancer for the next chore
iteration. We check if a table is in disabled state to exclude
regions that will not be of interest (because disabled regions are
or will be offline) or are in a state where they shouldn't be
mutated (like SPLITTING). The current checks are not actually
comprehensive.

Filter out regions not in OPEN or OPENING state when building the
set of interesting regions for the balancer to consider. Only
regions open (or opening) on the cluster are of interest to
balancing calculations for the current iteration. Regions in all
other states can be expected to not be of interest – either offline
(OFFLINE, or FAILED_*), not subject to balancer decisions now
(SPLITTING, SPLITTING_NEW, MERGING, MERGING_NEW), or will be
offline shortly (CLOSING) – until at least the next chore
iteration.

Add TRACE level logging.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-03 18:23:07 -07:00
Andrew Purtell e44592a37d
HBASE-25840 CatalogJanitor warns about skipping gc of regions during RIT, but does not actually skip (#3223) 
We claim in a WARN level log line to be "Playing-it-safe skipping merge/
split gc'ing of regions from hbase:meta while regions-in-transition (RIT)"
but do not actually skip because of a missing return. Remove the warning.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-05-03 18:14:38 -07:00
Duo Zhang 762abe3bea
HBASE-25838 Use double instead of Double in StochasticLoadBalancer (#3221) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-05-04 09:04:47 +08:00
Duo Zhang 7640134e3e
HBASE-25774 Added more detailed logs about the restarting of region servers (#3213) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-03 20:33:33 +08:00
GeorryHuang 00fec24c90
HBASE-25790 NamedQueue 'BalancerRejection' for recent history of balancer skipping (#3182) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-05-02 21:30:48 +05:30
Duo Zhang 73a82bd7c6
HBASE-25825 RSGroupBasedLoadBalancer.onConfigurationChange should chain the request to internal balancer (#3209) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-30 22:45:33 +08:00
Duo Zhang 6c65314cdf
HBASE-25819 Fix style issues for StochasticLoadBalancer (#3207) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-29 11:03:55 +08:00
Nick Dimiduk b061b0c4ed HBASE-25779 HRegionServer#compactSplitThread should be private 
Minor refactor. Make the `compactSplitThread` member field of `HRegionServer` private, and gate
all access through the getter method.

Signed-off-by: Yulin Niu <niuyulin@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2021-04-28 16:46:36 -07:00
Michael Stack 2382f68b23
HBASE-25792 Filter out o.a.hadoop.thirdparty building shaded jars (#3184) 
Need to add to allowed-licenses list too....

Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-04-27 08:37:25 -07:00
Duo Zhang 8856f61986 HBASE-25757 Addendum remove CandidateGenerator classes under hbase-server module 2021-04-27 23:25:51 +08:00
Duo Zhang a4d954e606
HBASE-25757 Move BaseLoadBalancer to hbase-balancer module (#3191) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-26 12:03:25 +08:00
Duo Zhang 7f90c2201f HBASE-25723 Temporarily remove the trace support for RegionScanner.next (#3119) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang f6ff519dd0 HBASE-25591 Upgrade opentelemetry to 0.17.1 (#2971) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang bb8c4967f8 HBASE-25535 Set span kind to CLIENT in AbstractRpcClient (#2907) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 2be2c63f0d HBASE-25484 Add trace support for WAL sync (#2892) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 03e12bfa4a HBASE-25455 Add trace support for HRegion read/write operation (#2861) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang ae2c62ffaa HBASE-25481 Add host and port attribute when tracing rpc call at client side (#2857) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 805b2ae2ad HBASE-23898 Add trace support for simple apis in async client (#2813) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 2420286715 HBASE-25401 Add trace support for async call in rpc client (#2790) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 302d9ea8b8 HBASE-25373 Remove HTrace completely in code base and try to make use of OpenTelemetry 
Signed-off-by: stack <stack@apache.org>
 2021-04-25 09:23:23 +08:00
Andrew Purtell 9895b2dfdf
HBASE-25756 Support alternate compression for major and minor compactions (#3142) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-04-23 15:45:26 -07:00
Duo Zhang 96fefce9c3
HBASE-25802 Miscellaneous style improvements for load balancer related classes (#3192) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-23 15:20:27 +08:00
haxiaolin 996862c1cc
HBASE-25754 StripeCompactionPolicy should support compacting cold regions (#3152) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-04-23 14:58:53 +08:00
Toshihiro Suzuki 5f4e2e111b
HBASE-25766 Introduce RegionSplitRestriction that restricts the pattern of the split point (#3150) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-04-22 13:53:36 +09:00
Duo Zhang 50920ee306
HBASE-25774 TestSyncReplicationStandbyKillRS#testStandbyKillRegionServer is flaky (#3189) 
Wait for the restarter thread to finish before checking the state
Add more detailed logs

Signed-off-by: meiyi <myimeiyi@gmail.com>
 2021-04-22 10:10:15 +08:00