Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,739 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

9439 Commits

Author SHA1 Message Date
Duo Zhang 0c4263a18b
HBASE-27185 Rewrite NettyRpcServer to decode rpc request with netty handler (#4624) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-07-27 09:00:42 +08:00
Andrew Purtell 9ba868d2ad HBASE-27087 TestQuotaThrottle times out 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-26 14:09:48 -07:00
Bryan Beaudreault 277058e18d
HBASE-27229 BucketCache statistics should not count evictions by hfile (#4639) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-26 16:59:57 -04:00
Duo Zhang 8b091c4061
HBASE-27222 Purge FutureReturnValueIgnored warnings from error prone (#4634) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-26 23:42:37 +08:00
Bryan Beaudreault ebd189157e
HBASE-27225 Add BucketAllocator bucket size statistic logging (#4637) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-07-25 20:54:40 -04:00
Wellington Ramos Chevreuil d5ed8f59e5
HBASE-27232 Fix checking for encoded block size when deciding if bloc… (#4640) 
Signed-off-by: Andor Molnár <andor@cloudera.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Ankit Singhal <ankit@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-25 11:17:16 +01:00
chenglei 0ae42ddaee
HBASE-27230 RegionServer should be aborted when WAL.sync throws Timeo… (#4641) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-24 15:08:03 +08:00
Duo Zhang 31fc97edee
HBASE-27075 TestUpdateRSGroupConfiguration.testCustomOnlineConfigChangeInRSGroup is flaky (#4636) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-07-21 14:26:52 +08:00
Duo Zhang 75a05b5cd0 Revert "HBASE-27152 Under compaction mark may leak (#4568)" 
This reverts commit da27a67a1e.
 2022-07-21 09:02:40 +08:00
Xiaolin Ha da27a67a1e
HBASE-27152 Under compaction mark may leak (#4568) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-20 08:49:10 +08:00
Duo Zhang acf144717b HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-19 10:00:43 +08:00
Andrew Purtell 745142adb1
HBASE-27195 Clean up netty worker/thread pool configuration (#4619) 
The configuration settings "hbase.netty.worker.count" and
"hbase.netty.eventloop.rpcserver.thread.count" appear to duplicate each
other.

Also, formalizes another setting found in NettyEventLoopGroupConfig,
"hbase.netty.nativetransport".

Also, native epoll is not limited to amd64. aarch64 supports it too.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-18 15:48:40 -07:00
Duo Zhang 9ab0b1504f
HBASE-27211 Data race in MonitoredTaskImpl could cause split wal failure (#4630) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-07-18 19:25:06 +08:00
SiCheng-Zheng ff8eb59709
HBASE-27144 Add special rpc handlers for bulkload operations (#4558) 
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-17 21:21:58 +08:00
Bryan Beaudreault 70a2ee1716
HBASE-27053 IOException during caching of uncompressed block to the block cache (#4610) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: wenwj0 <wenweijian2@huawei.com>
 2022-07-16 17:30:55 -04:00
Andrew Purtell 01b45e285d Amend HBASE-27194 Add test coverage for SimpleRpcServer (#4616) 
When backporting to branch-2 it was discovered that the order of tear
down operations requires the KDC to be torn down first, although on
master branch the original code did not produce the same errors (a NPE
in KDC#stop). Forward port the difference for maintainability.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-13 11:23:07 -07:00
Andrew Purtell 84f0d145d7
HBASE-27194 Add test coverage for SimpleRpcServer (#4616) 
Add test coverage for SimpleRpcServer.

Improve the way we test both SimpleRpcServer and NettyRpcServer. Use
LoadTestKVGenerator to generate random values with varying sizes between
1000 bytes and 1M bytes, and also to verify them when reading the values
back.

Add secure test coverage for both SimpleRpcServer and NettyRpcServer.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-13 09:28:30 -07:00
Duo Zhang 2662607b71
HBASE-27193 TestZooKeeper is flaky (#4615) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2022-07-13 10:32:01 +08:00
Andrew Purtell 7cb7100de5
HBASE-27161 Improve TestMultiRespectsLimits (#4586) 
TestMultiRespectLimits submits a put with an empty value. This appears
to not be what was intended. Instead, use a single byte.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-12 11:19:35 -07:00
Andrew Purtell ed32043da0
HBASE-27097 SimpleRpcServer is broken (#4613) 
Replace BufferChain#write(channel,int) with a simpler #write(channel)
implementation that does not attempt to "chunk" data to be written. This
method was used exclusively by SimpleRpcServer. The code was unnecessarily
complex and caused short writes when values were large, so was corrected
and simplified. Any difference in performance from this change will be
limited to SimpleRpcServer. Testing under load confirms the fix and does
not show significant regression.

SimpleRpcServer and its related code is now also marked as @Deprecated.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-12 11:07:23 -07:00
Duo Zhang 648bb6d0eb
HBASE-27192 The retry number for TestSeparateClientZKCluster is too small (#4614) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2022-07-12 19:08:41 +08:00
Bryan Beaudreault 886e31952f
HBASE-27149 Server should close scanner if client times out before results are ready (#4604) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-11 22:22:00 -04:00
Bryan Beaudreault 6e66e78177
HBASE-27188 Report maxStoreFileCount in jmx (#4609) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-11 22:11:58 -04:00
Bryan Beaudreault 779c4e2316
HBASE-27186 Report block cache size metrics separately for L1 and L2 (#4608) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-11 22:00:04 -04:00
Duo Zhang 5bc8670322
HBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel is closed (#4611) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-12 09:40:19 +08:00
Bryan Beaudreault 6886d02a91
HBASE-27078 Allow configuring a separate timeout for meta scans (#4557) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-07 10:02:14 -04:00
Duo Zhang 816e919e95
HBASE-26708 Netty leak detected and OutOfDirectMemoryError due to direct memory buffering with SASL implementation (#4596) 
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-07 15:55:26 +08:00
Duo Zhang 1643c45770 HBASE-27157 Addendum fix compile error after revert 2022-07-07 10:26:13 +08:00
Duo Zhang 0116aff9c0 Revert "HBASE-27157 Potential race condition in WorkerAssigner (#4577)" 
This reverts commit f76d8554ca.
 2022-07-07 08:36:21 +08:00
Ujjawal a31cc931fd
HBASE-27175 - Failure to cleanup WAL split dir log should be at INFO level (#4593) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-06 16:32:52 -07:00
Bryan Beaudreault b9706916cc
HBASE-27048 Server side scanner time limit should account for time in queue (#4562) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-06 11:57:37 -04:00
Duo Zhang 6f4a0476f2 HBASE-27148 Addendum uncomment the shutdown code in TestVerifyReplicationCrossDiffHdfs 2022-07-06 14:48:08 +08:00
Duo Zhang 41972cb460
HBASE-27148 Move minimum hadoop 3 support version to 3.2.3 (#4561) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-07-06 12:49:47 +08:00
Ruanhui f76d8554ca
HBASE-27157 Potential race condition in WorkerAssigner (#4577) 
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
 2022-07-06 10:59:13 +08:00
tianhang 6031a3a8d4
HBASE-27171 Fix Annotation Error in HRegionFileSystem (#4588) 
Co-authored-by: Tang Tianhang <tianhang.tang@shopee.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-07-06 10:58:34 +08:00
Bryan Beaudreault f3f292fad4
HBASE-27170 ByteBuffAllocator leak when decompressing blocks near minSizeForReservoirUse (#4592) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-07-04 16:18:45 -04:00
Duo Zhang 03c23beb57
HBASE-27169 TestSeparateClientZKCluster is flaky (#4587) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-07-01 22:09:01 +08:00
Ishika acfbc3ff7d
HBASE-26218 Add logs in Canary tool (#4501) 
Co-authored-by: Ishika Soni <isoni@isoni-ltmag9r.internal.salesforce.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-30 21:53:20 +08:00
Duo Zhang 9bcb7493e0
HBASE-27165 Set version as 3.0.0-alpha-4-SNAPSHOT in master (#4582) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-06-29 11:04:21 +08:00
Ruanhui 9c2bc55350
HBASE-27158 Add namespace column family to UNDELETABLE_META_COLUMNFAMILIES (#4578) 
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-29 10:27:53 +08:00
Andrew Purtell ca34bdccc3
HBASE-27166 WAL value compression minor improvements (#4584) 
A larger IO buffer for absorbing WALCodec writes can improve the compression
ratio of larger values, because the compressor will be given a larger internal
buffer over which there will be more match opportunities. Does not impact the
ability to read existing written files.

Also, reset the BAOS internal buffer on the way out of compress() so potential
large-ish buffers do not linger on the heap longer than necessary.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-28 12:49:25 -07:00
Duo Zhang d7f6861e8d
HBASE-27160 ClientZKSyncer.deleteDataForClientZkUntilSuccess should break from the loop when deletion is succeeded (#4579) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2022-06-26 15:22:25 +08:00
Andrew Purtell 4f88a3c4bb
HBASE-27111 Make Netty channel bytebuf allocator configurable. (#4525) 
Support site configuration of the bytebuf allocator that Netty will use for
NettyRpcServer channels. Property name is 'hbase.netty.rpcserver.allocator'.
Default is no value, which is equivalent to "pooled". Valid values are:
- "pooled": use PooledByteBufAllocator
- "unpooled": use UnpooledByteBufAllocator
- "heap": use HeapByteBufAllocator, which is a PooledByteBufAllocator that
   preferentially allocates buffers on heap wherever possible
- <class>: If the value is none of the recognized labels, treat it as a class
  name implementing org.apache.hbase.thirdparty.io.netty.buffer.ByteBufAllocator.
  This allows the user to add a custom implementation, perhaps for debugging.

Also updates ReflectionUtils with a new helper method.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-06-23 15:13:29 -07:00
Kerasone b8558d30d0
HBASE-27001 The deleted variable cannot be printed out (#4479) 
* The deleted variable cannot be printed out, add it

Co-authored-by: selina.yan <selina.yan@huolala.cn>


Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2022-06-23 22:41:32 +05:30
Bryan Beaudreault e84cd3dd6a
HBASE-26945 Quotas causes too much load on meta for large clusters (#4567) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-06-23 10:34:29 -04:00
Bryan Beaudreault 59e5c610e3 HBASE-26790 Addendum ensure test is not flaky due to async caching 2022-06-23 10:22:55 -04:00
Bryan Beaudreault 50f11151fd
HBASE-26790 getAllRegionLocations can cache locations with null hostname (#4565) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-06-23 08:24:49 -04:00
Pankaj 342183e596
HBASE-27105 HBaseInterClusterReplicationEndpoint should honor replication adaptive timeout (#4515) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-06-23 12:51:48 +05:30
chenglei de804938b9
HBASE-27062 Remove ThreadPool in HBaseInterClusterReplicationEndpoint when use AsyncClusterConnection (#4560) 2022-06-22 20:01:56 +08:00
chenglei 902bc3ed70
Revert "HBASE-27062 ThreadPool is unnecessary in HBaseInterClusterReplication… (#4463)" (#4559) 2022-06-22 19:38:48 +08:00
chenglei 34ba2c51cf
HBASE-27062 ThreadPool is unnecessary in HBaseInterClusterReplication… (#4463) 
* HBASE-27062 ThreadPool is unnecessary in HBaseInterClusterReplicationEndpoint when use AsyncClusterConnection
 2022-06-22 19:33:05 +08:00
litao 9da2a3fead
HBASE-27098 Fix link for field comments (#4499) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-06-21 18:29:34 -07:00
BukrosSzabolcs 5cf728da5a
HBASE-26969:Eliminate MOB renames when SFT is enabled (#4418) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-06-21 10:18:55 +01:00
xiaozhang0319 b498efdbc1
HBASE-27099 The log printing fspread/fsread cost time unit should be milliseconds (#4500) 
Co-authored-by: xiaozhang <issac.zhang@huolala.cn>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-21 00:13:33 +08:00
SiCheng-Zheng a0a018d6c3 HBASE-27128 when open archiveRetries totalLogSize calculation mistake (#4546) 
Co-authored-by: zhengsicheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 5833756e5e)
 2022-06-20 23:48:33 +08:00
Xiaolin Ha 89bda2c322
HBASE-27125 The batch size of cleaning expired mob files should have an upper bound (#4541) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-17 23:35:26 +08:00
litao 5e5d18bf21
HBASE-27117 Update the method comments for RegionServerAccounting (#4532) 2022-06-16 17:18:18 -07:00
Duo Zhang 02990894ab
HBASE-27121 Set version as 3.0.0-alpha-3 in master in prep for first RC of 3.0.0-alpha-3 (#4535) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-06-15 10:51:31 +08:00
LiangJun He c6298c709a HBASE-27091 Speed up the loading of table descriptor from filesystem (#4493) 
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
Signed-off-by: Yu Li <liyu@apache.org>
 2022-06-15 00:48:25 +08:00
Yu Li 86b7b027b7 Revert "HBASE-27091 Speed up the loading of table descriptor from filesystem (#4493)" 
This reverts commit 5d0f4dc729 to correct signed-off info.
 2022-06-15 00:47:28 +08:00
LiangJun He 5d0f4dc729 HBASE-27091 Speed up the loading of table descriptor from filesystem (#4493) 
Signed-off-by: Yu Li <liyu@apache.org>
 2022-06-15 00:41:24 +08:00
LiangJun He 002c92cd7a
HBASE-27028 Add a shell command for flushing master local region (#4457) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-13 11:09:05 +08:00
Viraj Jasani 5e34cdf1ef
Revert "HBASE-25709 Close region may stuck when region is compacting and skipped most cells read (#3117)" (#4524) 
This reverts commit f3a48d1910.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-06-12 10:22:42 -07:00
Nick Dimiduk ed7e15dc91
HBASE-26366 Provide meaningful parent spans to ZK interactions 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-06-10 16:25:55 +02:00
Nick Dimiduk e191fe9219
HBASE-27095 HbckChore should produce a report 
In #4470 for HBASE-26192, it was noted that the HbckChore is kind of a pain to use and test
because it maintains a bunch of local state. By contract, the CatalogJanitorChore makes a nice
self-contained report. Let's update HbckChore to do the same.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-06-10 15:46:24 +02:00
Nick Dimiduk 414cfb30f6 HBASE-27066 The Region Visualizer display failed 
Reviewed-by: litao <tomleescut@gmail.com>
 2022-06-09 12:56:38 +02:00
Nick Dimiduk 02cc63e524 Revert "HBASE-27066 The Region Visualizer display failed (#4472)" 
This reverts commit 5c116d239f.
 2022-06-09 12:56:38 +02:00
litao 9342653691
HBASE-27092 Regionserver table on Master UI is broken (#4495) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-06-07 17:37:48 -07:00
liuzhuang2017 12a9e6e5ec
HBASE-27080 Optimize debug output log of ConstantSizeRegionSplitPolicy class. (#4481) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-06 13:37:58 +08:00
BukrosSzabolcs 7184949cea
HBASE-27017: MOB snapshot is broken when FileBased SFT is used (#4466) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-06-05 19:51:28 +01:00
chenglei d57159f31c
HBASE-26993 Make the new framework for region replication could work for SKIP_WAL (#4392) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-03 16:39:17 +08:00
skysiders c0e8243c8b
HBASE-26985 check permission for SecureBulkLoadManager (#4379) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-06-02 15:28:22 -07:00
Duo Zhang 313a37f7ac
HBASE-27046 The filenum in AbstractFSWAL should be monotone increasing (#4449) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2022-06-02 23:55:23 +08:00
liangxs d03004060d
HBASE-26962: Add mob info in web UI (#4359) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-02 23:53:03 +08:00
Xiaolin Ha 80f410ec3f
HBASE-26680 Close and do not write trailer for the broken WAL writer(addendum) (#4405) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-02 23:52:14 +08:00
litao ac8b51718f
HBASE-27036 Displays the number of decommissioned region server for status command (#4431) 
add decommissionedServers to cluster metric

Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org
 2022-06-02 23:51:39 +08:00
LiangJun He 15002fccb4
HBASE-27039 Some methods of MasterRegion should be annotated for testing only (#4433) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-02 21:27:25 +08:00
litao db992083af
HBASE-27037 Display the region server state on the Web UI (#4432) 
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-06-02 21:19:45 +08:00
Duo Zhang 99e4e3ea3e
HBASE-27082 Change the return value of RSGroupInfo.getServers from SortedSet to Set to keep compatibility (#4480) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-06-01 08:30:52 -07:00
Andrew Purtell 540fe8bbfa
HBASE-27079 Lower some DEBUG level logs in ReplicationSourceWALReader to TRACE (#4476) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-05-31 12:43:04 -07:00
litao d532135a26
HBASE-27068 NPE occurs when the active master has not yet been elected (#4474) 
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
Signed-off-by: Viraj Jasani<virajjasani@apache.org
 2022-05-31 11:49:49 -07:00
Andrew Purtell 88d9c4bef4
HBASE-27064 CME in TestRegionNormalizerWorkQueue (#4468) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-05-31 11:33:06 -07:00
Istvan Toth 4ce85aa39f
HBASE-27069 Hbase SecureBulkload permission regression (#4475) 
Signed-off-by: Rajeshbabu Chintaguntla <rajeshbabu@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-05-31 09:52:35 +01:00
litao 5c116d239f
HBASE-27066 The Region Visualizer display failed (#4472) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-05-29 11:53:50 -07:00
Sergey Soldatov fdf054de2c
HBASE-27061 two phase bulkload is broken when SFT is in use. (#4465) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-05-26 11:01:21 +01:00
Duo Zhang 286f6c5af4 HBASE-26649 Addendum fix spotless error 2022-05-25 21:28:47 +08:00
Xiaolin Ha bc21967946
HBASE-26342 Support custom paths of independent configuration and pool for hfile cleaner (#4403) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-05-24 09:53:48 +08:00
huaxiangsun 623f8affe2
HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations() (#4442) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-05-23 08:54:00 -07:00
Xiaolin Ha 1aa07d5e6b
HBASE-27043 Let lock wait timeout to improve performance of SnapshotHFileCleaner (#4437) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-05-20 15:32:58 -07:00
Nick Dimiduk bf5f0c7e7f
HBASE-27052 TestAsyncTableScanner.testScanWrongColumnFamily is flaky (#4451) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-05-20 15:13:23 -07:00
Andrew Purtell 42a64f2a4d
HBASE-27051 TestReplicationSource.testReplicationSourceInitializingMetric is flaky (#4448) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-05-20 15:10:42 -07:00
Duo Zhang 487cf301a4
HBASE-27045 Disable TestClusterScopeQuotaThrottle (#4440) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-05-18 20:29:06 +08:00
LiangJun He b787648847
HBASE-27040 Optimize the log display of the ZKProcedureUtil.java (#4434) 
Signed-off-by: Yu Li <liyu@apache.org>
 2022-05-17 15:29:43 +08:00
litao ed1665355a
HBASE-27032 The draining region servers metric description is incorrect (#4428) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2022-05-17 10:35:35 +05:30
Nick Dimiduk ad64a9baae HBASE-26986 Trace a one-shot execution of a Master procedure 
This one adds a tracing context to the Master's ProcedureV2 execution. This ties all the spans
that result from a procedure execution back to a procedure by name. Maybe in a follow-on issue, we
can serialize span context along with procedures and display procedure start/stop/retry events and
parent-child hierarchy in tracing visualization.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-05-16 11:39:21 +02:00
Rushabh Shah 27ced7074a
HBASE-26905 ReplicationPeerManager#checkPeerExists should throw ReplicationPeerNotFoundException if peer doesn't exists (#4422) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-05-13 18:24:19 -07:00
Tak Lon (Stephen) Wu 0eb1a4e5ed
HBASE-27013 Introduce read all bytes when using pread for prefetch (#4414) 
- introduce optional flag `hfile.pread.all.bytes.enabled` for pread that must read full bytes with the next block header
 2022-05-13 12:16:35 -07:00
Wellington Chevreuil 3fbf66729d HBASE-27021 StoreFileInfo should set its initialPath in a consistent way 
(Amending spotless errors after revert by Duo)
 2022-05-12 10:34:29 +01:00
Duo Zhang 141b827f47 Revert "HBASE-27021 StoreFileInfo should set its initialPath in a consistent way (#4419)" 
Revert due to failure spotless:check

This reverts commit aa31b0f695.
 2022-05-12 16:42:30 +08:00
Wellington Ramos Chevreuil aa31b0f695
HBASE-27021 StoreFileInfo should set its initialPath in a consistent way (#4419) 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-05-12 09:36:46 +01:00
litao a712f9fa34
HBASE-27020 Fix spotless warn for master branch (#4417) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed by: Rushabh Shah <shahrs87@gmail.com>
 2022-05-11 14:36:37 +08:00
Wellington Ramos Chevreuil f6e9d3e1dd
HBASE-26999 HStore should try write WAL compaction marker before repl… (#4407) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-05-09 10:24:47 +01:00
litao e8b44d948c
HBASE-27015. Fix log format for ServerManager (#4415) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-05-08 13:33:17 -07:00
skysiders 5d6c0a9ca5
HBASE-26994 MasterFileSystem create directory without permission check (#4391) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-05-08 13:26:53 -07:00
Rushabh Shah 2d1e3c9268
HBASE-26963 ReplicationSource#removePeer hangs if we try to remove bad peer. (#4361) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-05-07 10:37:05 -07:00
litao e9046a64f4
HBASE-27000 Block cache stats (Misses Caching) display error in RS web UI (#4406) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-05-05 23:32:28 -07:00
Nick Dimiduk 404e1d6870 HBASE-26648 Improve fidelity of RegionLocator spans (addendum) 
Addendum extends the test case to cover both `RpcConnectionRegistry` and `ZKConnectionRegistry`.
 2022-05-03 14:00:53 +02:00
Duo Zhang 9c8c9e7fbf HBASE-26899 Run spotless:apply 
Closes #4312
 2022-05-01 22:15:09 +08:00
chenglei f5a566ea1b
HBASE-26960 Another case for unnecessary replication suspending in Re… (#4355) 2022-04-30 11:37:02 +08:00
LiangJun He b0c2832b6e
HBASE-26976 Update related comments after HMaster can load the live RS infos from local region (#4375) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Yu Li <liyu@apache.org>
 2022-04-30 10:53:13 +08:00
Bryan Beaudreault da37896588
HBASE-26886 Add unit test for custom RpcControllerFactory ability to overwrite priorities (#4277) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-04-29 19:01:08 -07:00
Nick Dimiduk ebfac2164a HBASE-26648 Improve fidelity of RegionLocator spans 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-04-29 10:53:06 +02:00
LiangJun He 7d5bf1c4f2 HBASE-26943 HMaster page style display confusion (#4369) 
Signed-off-by: Yu Li <liyu@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-04-29 15:56:42 +08:00
LiangJun He 23971b2861 Revert "HBASE-26943 HMaster page style display confusion" and "HBASE-26943 Formatting bootstrap.min.css" (#4369) 
This reverts commits 242a194b3c and d0318732b7.

Signed-off-by: Yu Li <liyu@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-04-29 15:56:21 +08:00
d-c-manning f848c61e3d
HBASE-22349 slop in StochasticLoadBalancer (#4371) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-04-28 12:28:15 -07:00
liangxs cdf81ea5cc
HBASE-26975 Add on heap and off heap memstore info in rs web UI (#4368) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-04-28 23:08:53 +08:00
liangxs c02e7553d4
HBASE-26980 Update javadoc of BucketCache.java (#4374) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-04-28 23:08:40 +08:00
Andrew Purtell 90d42f2733
HBASE-26979 StoreFileListFile logs frequent stacktraces at INFO level (#4373) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-04-27 18:02:00 -07:00
Bri Augenreich ee0c921258
HBASE-26581 Add metrics for failed replication edits (#4347) 
Co-authored-by: Briana Augenreich <baugenreich@hubspot.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2022-04-26 17:42:54 -04:00
LiangJun He ba713ac379 HBASE-26951 HMaster should exit gracefully, when stopped via hbase-daemon.sh (#4358) 
Signed-off-by: Yu Li <liyu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-04-27 00:21:57 +08:00
Wellington Ramos Chevreuil af13c6d4c6
HBASE-26971 SnapshotInfo --snapshot param is marked as required even when trying to list all snapshots (#4366) 
Signed-off-by: Josh Elser <elserj@apache.org>
 2022-04-25 12:30:07 +01:00
Ruanhui f2f2a89592
HBASE-26942 cache region locations when getAllRegionLocations (#4335) 
HBASE-26942 cache region locations when getAllRegionLocations

Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2022-04-22 07:48:27 -04:00
Duo Zhang 35aa57e445
HBASE-26941 LocalHBaseCluster.waitOnRegionServer should not call join while interrupted (#4352) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-04-17 22:56:52 +08:00
Duo Zhang 48c4a4626e
HBASE-26938 Compaction failures after StoreFileTracker integration (#4350) 
Introduce a StoreFileWriterCreationTracker to track the store files being written

Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-04-17 21:58:12 +08:00
Duo Zhang 2622fa05a6
HBASE-26946 TestRetainAssignmentOnRestartSplitWithoutZk infinite loop (#4344) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-04-16 12:24:44 +08:00
Pankaj 62e3efccef
HBASE-26944 Possible resource leak while creating new region scanner (#4339) 
* HBASE-26944 Possible resource leak while creating new region scanner

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-04-15 11:34:25 +05:30
LiangJun He 242a194b3c HBASE-26943 HMaster page style display confusion 
Signed-off-by: Yu Li <liyu@apache.org>
 2022-04-13 17:18:18 +08:00
LiangJun He d0318732b7 HBASE-26943 Formatting bootstrap.min.css 
Signed-off-by: Yu Li <liyu@apache.org>
 2022-04-13 17:18:18 +08:00
Yu Li 58d40dbcb5 Revert "HBASE-26943 Formatting bootstrap.min.css" to correct author information 
This reverts commit c5158aef93.
 2022-04-13 17:18:07 +08:00
Yu Li 463a92da65 Revert "HBASE-26943 HMaster page style display confusion" to correct author information 
This reverts commit 67ac16c8da.
 2022-04-13 17:17:35 +08:00
liangjunhe.hlj 67ac16c8da HBASE-26943 HMaster page style display confusion 
Signed-off-by: Yu Li <liyu@apache.org>
 2022-04-13 17:09:00 +08:00
liangjunhe.hlj c5158aef93 HBASE-26943 Formatting bootstrap.min.css 
Signed-off-by: Yu Li <liyu@apache.org>
 2022-04-13 17:08:57 +08:00
Duo Zhang b7ffb56927 Revert "HBASE-26941 LocalHBaseCluster.waitOnRegionServer should quit while thread is interrupted (#4333)" 
This reverts commit 8247b7c722.
 2022-04-13 11:54:50 +08:00
Duo Zhang 8247b7c722
HBASE-26941 LocalHBaseCluster.waitOnRegionServer should quit while thread is interrupted (#4333) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-04-13 10:49:40 +08:00
eomiks 7ac9e0be27
HBASE-26901 delete with null columnQualifier occurs NullPointerException when NewVersionBehavior is on (#4295) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-04-12 20:26:36 +08:00
Duo Zhang 3ae0d9012c
HBASE-26922 Fix LineLength warnings as much as possible if it can not be fixed by spotless (#4324) 
Signed-off-by: Yulin Niu <niuyulin@apache.org
 2022-04-09 21:38:41 +08:00
Bryan Beaudreault eb4c2ae4b4
HBASE-26807 Unify CallQueueTooBigException special pause with CallDroppedException (#4180) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-04-07 08:01:44 -07:00
huaxiangsun 0b4ff98614
HBASE-26618 Involving primary meta region in meta scan with CatalogRe… (#4321) 
HBASE-26618 Involving primary meta region in meta scan with CatalogReplicaLoadBalanceSimpleSelector

Signed-off-by: Michael Stack <stack@apache.org>
 2022-04-06 12:11:10 -07:00
Duo Zhang e68c61dd54
HBASE-26928 Fix several indentation problems (#4323) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-04-06 15:09:00 +08:00
Wellington Ramos Chevreuil ae3718be0f
HBASE-26927 Add snapshot scanner UT with SFT and some cleanups to Tes… (#4322) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2022-04-05 16:55:34 +01:00
bsglz 7f9d1f2259
HBASE-26885 Addendum throw exception instead of return in TRSP to let the procedure retry (#4299) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-04-04 20:25:42 +08:00
weimingdiit 003c572246
HBASE-26924 Fix log parameter error and spelling error (#4318) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2022-04-04 20:20:30 +08:00
Duo Zhang 3e09e87fad
HBASE-26921 Rewrite the counting cells part in TestMultiVersions (#4316) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-04-03 22:54:00 +08:00
Duo Zhang 78676bb85e
HBASE-26920 Fix missing braces warnings in TestProcedureMember (#4315) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-04-03 22:53:50 +08:00
Duo Zhang 4e12eee531
HBASE-26919 Rewrite the counting rows part in TestFromClientSide4 (#4314) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-04-03 22:42:49 +08:00
chenglei f51e57908f
HBASE-26811 (addendum)Secondary replica may be disabled for read incorrectly forever (#4311) 2022-04-01 14:08:19 +08:00
chenglei 805c346e54
HBASE-26811 Secondary replica may be disabled for read forever (#4182) 2022-03-31 21:46:20 +08:00
Duo Zhang 27111421d9 HBASE-26245 Addendum fix a naming issue 2022-03-31 21:40:44 +08:00
Duo Zhang bb1bbddf74
HBASE-26245 Store region server list in master local region (#4136) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-31 21:22:01 +08:00
bsglz 1efd8fe53c
HBASE-26885 The TRSP should not go on when it get a bogus server name… (#4276) 
* HBASE-26885 The TRSP should not go on when it get a bogus server name from AM
 2022-03-29 10:05:24 +08:00
d-c-manning a4db023bc5
HBASE-26718 HFileArchiver can remove referenced StoreFiles from the archive (#4274) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-28 11:45:10 -07:00
Nick Dimiduk 620e5c6d0a HBASE-26545 Implement tracing of scan 
* on `AsyncTable`, both `scan` and `scanAll` methods should result in `SCAN` table operations.
* the span of the `SCAN` table operation should have children representing all the RPC calls
  involved in servicing the scan.
* when a user provides custom implementation of `AdvancedScanResultConsumer`, any spans emitted
  from the callback methods should also be tied to the span that represents the `SCAN` table
  operation. This is easily done because these callbacks are executed on the RPC thread.
* when a user provides a custom implementation of `ScanResultConsumer`, any spans emitted from the
  callback methods should be also be tied to the span that represents the `SCAN` table
  operation. This accomplished by carefully passing the span instance around after it is created.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-28 12:46:59 +02:00
Duo Zhang 4f491fd5e4
HBASE-26832 Avoid repeated releasing of flushed wal entries in AsyncFSWAL#syncCompleted (#4281) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-28 07:53:28 +08:00
huaxiangsun 698937d32b
HBASE-26864 SplitTableRegionProcedure calls openParentRegions() at a … (#4261) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-26 12:02:16 -07:00
Tak Lon (Stephen) Wu 4ac600b04b
HBASE-26810 Add dynamic configuration support for system coprocessors (#4181) 
Signed-off-by: Ankit Singhal <ankit@apache.org>
 2022-03-24 17:08:50 -07:00
Xiaolin Ha ec7141720f
HBASE-26175 MetricsHBaseServer should record all kinds of Exceptions (#4248) 
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2022-03-24 19:03:22 +08:00
chenglei ab4351a155
HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallCo… (#4249) 2022-03-23 17:14:07 +08:00
Duo Zhang b67c16a763
HBASE-26866 Shutdown WAL may abort region server (#4254) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-23 14:53:58 +08:00
Nick Dimiduk f78f232b28
HBASE-26834 Adapt ConnectionRule for both sync and async connections 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-21 12:41:22 +01:00
Duo Zhang e56ed404cb
HBASE-26791 Memstore flush fencing issue for SFT (#4202) 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-03-21 19:33:07 +08:00
Duo Zhang 520ad638d9
HBASE-26861 Fix flaky TestSnapshotFromMaster.testSnapshotHFileArchiving (#4242) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-18 23:28:30 +08:00
Duo Zhang 31db8e3b3b
HBASE-26858 Refactor TestMasterRegionOnTwoFileSystems to avoid dead loop (#4238) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-18 23:09:09 +08:00
huiruan 84acdf98ce HBASE-26859 Split TestSnapshotProcedure to several smaller tests (#4239) 
Closes #4239

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-18 17:52:15 +08:00
Duo Zhang bd65ad42a2 HBASE-26833 Addendum fix NPE in TestReplicationSource.testTerminateClearsBuffer 2022-03-17 23:13:41 +08:00
Xiaolin Ha c7773adeed
HBASE-26680 Close and do not write trailer for the broken WAL writer (#4174) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-16 19:58:45 +08:00
Duo Zhang 5b28d76652
HBASE-26837 Set SFT config when creating TableDescriptor in TestCloneSnapshotProcedure (#4226) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-16 19:34:30 +08:00
Nick Dimiduk c4ec5230f2
HBASE-25896 Implement a Region Visualization on Master WebUI (#4228) 
This is a demonstration of visualization of regions on the cluster. The visualization is a stacked
bar chart showing total storefile size per table per region server, with the x-axis being server
names, the y-axis being storfile size, and the bars stacked per table. The visualization is
generated entirely on the fly from within the browser, implemented using Vega Lite. So far, Vega
appears to handle rendering this visualization for a cluster of over 700 region servers with
approximately 300,000 regions.

Per [0], include an update to the top-level LICENSE.txt. Also update LICENSE files in all binary
distributions (i.e., jars), by way of LICENSE.vm. Vega uses a BSD 3-clause variant without
advertising clause, and as such is a "Category A" license, per [1].

No changes are made to the NOTICE files, as per the existing example of bundling the minified
JQuery, which is also a Category A license.

[0]: https://infra.apache.org/licensing-howto.html
[1]: https://www.apache.org/legal/resolved.html#category-a

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-15 21:01:57 +01:00
Nick Dimiduk 2507dc1b01 Revert "HBASE-25896 Implement a Region Visualization on Master WebUI" 
This reverts commit d76e636da0.
 2022-03-15 20:43:46 +01:00
Nick Dimiduk d76e636da0 HBASE-25896 Implement a Region Visualization on Master WebUI 
This is a demonstration of visualization of regions on the cluster. The visualization is a stacked
bar chart showing total storefile size per table per region server, with the x-axis being server
names, the y-axis being storfile size, and the bars stacked per table. The visualization is
generated entirely on the fly from within the browser, implemented using Vega Lite. So far, Vega
appears to handle rendering this visualization for a cluster of over 700 region servers with
approximately 300,000 regions.

Per [0], include an update to the top-level LICENSE.txt. Also update LICENSE files in all binary
distributions (i.e., jars), by way of LICENSE.vm. Vega uses a BSD 3-clause variant without
advertising clause, and as such is a "Category A" license, per [1].

No changes are made to the NOTICE files, as per the existing example of bundling the minified
JQuery, which is also a Category A license.

[0]: https://infra.apache.org/licensing-howto.html
[1]: https://www.apache.org/legal/resolved.html#category-a

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-15 20:31:14 +01:00
Xiaolin Ha d050217a3d
HBASE-26833 Avoid waiting to clear buffer usage of ReplicationSourceShipper when aborting the RS (#4218) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-15 22:04:40 +08:00
Xiaolin Ha fb09cb9e7b
HBASE-26840 Fix NPE in the retry of logroller (#4224) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-15 21:59:09 +08:00
Duo Zhang a816204793
HBASE-26611 Changing SFT implementation on disabled table is dangerous (#4082) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-15 20:13:46 +08:00
Duo Zhang 5ad51e8b2b
HBASE-26835 Rewrite TestLruAdaptiveBlockCache to make it more stable (#4219) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-15 20:06:05 +08:00
Nick Dimiduk f640b7fd27 HBASE-25895 Implement a Cluster Metrics JSON endpoint (addendum) 2022-03-14 17:27:53 +01:00
Nick Dimiduk 36a5f86c50 HBASE-26531 Trace coprocessor exec endpoints 
Trace table ExecService invocations as table operations. Ensure span relationships for both table
and master invocations.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-14 11:27:30 +01:00
meiyi d542f471d1
HBASE-26670 HFileLinkCleaner should be added even if snapshot is disabled (#4032) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-14 11:47:23 +08:00
Duo Zhang c3d0cc7040
HBASE-26830 Rewrite TestLruBlockCache to make it more stable (#4212) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-14 11:04:15 +08:00
Ruanhui fd301ad55f
HBASE-26323 Introduce a Snapshot Procedure (#4115) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-12 23:21:22 +08:00
Duo Zhang c2485215de
HBASE-25465 Use javac --release option for supporting cross version compilation (#4164) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-12 16:51:49 +08:00
Andrew Purtell 71ff3ec323
HBASE-26761 TestMobStoreScanner (testGetMassive) can OOME (#4204) 
Change the timing of the test but not the objective by storing three large
MOB values into the row with three separate puts, each randomizing the
data. Increases running time of the testGetMassive case 2x but avoids OOME
in the test environment where the OOME was consistently reproductable.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2022-03-11 14:44:42 +08:00
Andrew Purtell f3faa26047
HBASE-26824 TestHBaseTestingUtil.testResolvePortConflict failing after HBASE-26582 (#4203) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-11 14:43:00 +08:00
Xiaolin Ha dbf56819c3
HBASE-26816 Fix CME in ReplicationSourceManager (#4187) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-11 11:06:14 +08:00
Andrew Purtell e5dbbd20ac
HBASE-26715 Blocked on SyncFuture in AsyncProtobufLogWriter#write (#4184) 
Co-authored-by: Bryan Beaudreault <bbeaudreault@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-10 11:55:38 -08:00
Bryan Beaudreault 4da53efcf9
HBASE-26814 Addendum: Fix spotbugs warning and clarify intent (#4201) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-10 10:09:51 -08:00
Nick Dimiduk be0afbf23a HBASE-25895 Implement a Cluster Metrics JSON endpoint 
Publishes a set of JSON endpoints following a RESTful structure, which expose a subset of the
`o.a.h.h.ClusterMetrics` object tree. The URI structure is as follows

    /api/v1/admin/cluster_metrics
    /api/v1/admin/cluster_metrics/live_servers
    /api/v1/admin/cluster_metrics/dead_servers

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-10 14:58:05 +01:00
Nick Dimiduk 5851400a46 HBASE-26819 Minor code cleanup in and around RpcScheduler 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-10 14:45:04 +01:00
Nick Dimiduk 068031ea82 HBASE-26817 Mark RpcExecutor as IA.LimitedPrivate COPROC and PHOENIX 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-10 14:36:39 +01:00
chenglei 1057da8be6
HBASE-26768 Avoid unnecessary replication suspending in RegionReplica… (#4127) 2022-03-10 10:58:47 +08:00
Bryan Beaudreault 1dd29dba5c
HBASE-26814 Default StoreHotnessProtector to off, with logs to guide when to turn it on (#4194) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-09 14:43:58 -08:00
Andrew Purtell 10471944bd
HBASE-26582 Prune use of Random and SecureRandom objects (#4118) 
Avoid the pattern where a Random object is allocated, used once or twice, and
then left for GC. This pattern triggers warnings from some static analysis tools
because this pattern leads to poor effective randomness. In a few cases we were
legitimately suffering from this issue; in others a change is still good to
reduce noise in analysis results.

Use ThreadLocalRandom where there is no requirement to set the seed to gain
good reuse.

Where useful relax use of SecureRandom to simply Random or ThreadLocalRandom,
which are unlikely to block if the system entropy pool is low, if we don't need
crypographically strong randomness for the use case. The exception to this is
normalization of use of Bytes#random to fill byte arrays with randomness.
Because Bytes#random may be used to generate key material it must be backed by
SecureRandom.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-08 13:49:02 -08:00
Bryan Beaudreault 39ecaa1975
HBASE-26784 Use HIGH_QOS for ResultScanner.close requests (#4146) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-08 10:48:57 +08:00
Nick Dimiduk f3f2aa9631 HBASE-26782 Minor code cleanup in and around RpcExecutor 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-07 11:55:36 +01:00
Xiaolin Ha 591f7819f3
HBASE-26552 Introduce retry to logroller to avoid abort (#4038) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-07 12:20:01 +08:00
Xiaolin Ha f3a48d1910
HBASE-25709 Close region may stuck when region is compacting and skipped most cells read (#3117) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-07 11:21:39 +08:00
Rushabh Shah 2b205d2230
HBASE-26792 Implement ScanInfo#toString (#4153) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-05 17:45:37 +08:00
Nick Dimiduk 2b9ab90a03
HBASE-26759 Fix trace continuity through CallRunner (#4126) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-04 15:23:25 -08:00
Bryan Beaudreault be59eb7e8d
HBASE-26731 Add metrics for active and expired scanners (#4145) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-04 14:03:32 -08:00
Istvan Toth 7d2457e075
HBASE-26777 BufferedDataBlockEncoder$OffheapDecodedExtendedCell.deepC… (#4139) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-04 13:49:19 -08:00
Xiaolin Ha f4b05a8364
HBASE-26242 Allow split when store file count larger than the configed blocking file count (#3652) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-04 20:18:38 +08:00
Nick Dimiduk 9c037b1be7 HBASE-26764 Implement generic exception support for TraceUtil methods over Callables and Runnables 
For the `TraceUtil` methods that accept `Callable` and `Runnable` types, make them generic over a
child of `Throwable`. This allows us to consolidate the two method signatures into a single more
flexible definition.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-03 12:18:33 +01:00
chenglei 1b994a179b
HBASE-26787 TestRegionReplicaReplicationError should inject error in replicateToReplica method (#4148) 2022-03-03 12:00:45 +08:00
Mike Drob 06f06cbd86 HBASE-26622 Update error-prone to 2.10 
Author:    Mike Drob <mdrob@apache.org>
Co-authored-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-02 16:07:23 +01:00
Niels Basjes f6b74b3083
HBASE-26762 Un-Deprecate and improve documentation for Scan#setRowPrefixFilter (#4119) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-02 20:19:13 +08:00
Nick Dimiduk e85bf414ab
HBASE-26776 RpcServer failure to SASL handshake always logs user "unknown" to audit log (#4138) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2022-03-01 14:33:02 +01:00
Duo Zhang c8662ed575
HBASE-26640 Reimplement master local region initialization to better work with SFT (#4111) 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-02-24 10:10:02 +08:00
Bryan Beaudreault a49c758f58
HBASE-26727 Fix CallDroppedException reporting (#4088) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-02-18 17:27:09 -08:00
Bryan Beaudreault 8fb3d4666a
HBASE-26703 Allow configuration of IPC queue balancer (#4063) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-02-18 17:21:40 -08:00
Richard Marscher b3b4405d61
HBASE-26576 Allow pluggable queue to be used with the fast path executor or normal balanced executor (#3944) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-02-18 10:00:10 -08:00
Yutong Xiao e100198ab5
HBASE-26659 The ByteBuffer of metadata in RAMQueueEntry in BucketCache could be reused. (#4026) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-02-18 22:24:51 +08:00
BukrosSzabolcs 4829806220
HBASE-26707: Reduce number of renames during bulkload (#4066) 
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
 2022-02-17 19:34:48 +00:00
chenglei 8ddd1d0f9e
HBASE-26712 Balancer encounters NPE in rare case (#4092) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-02-16 12:12:18 +05:30
Xiaolin Ha 1b47575214
HBASE-26742 Comparator of NOT_EQUAL NULL is invalid for checkAndMutate (#4105) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-02-16 11:08:33 +08:00
Andrew Purtell 407c6e7b22
HBASE-26730 Extend hbase shell 'status' command to support an option 'tasks' (#4095) 
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2022-02-15 18:38:18 -08:00
Xiaolin Ha 7c52895e8c
HBASE-26434 Do compact when all L0 files are expired (#3830) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-02-15 22:08:52 +08:00
Yutong Xiao 1af22625e9
HBASE-26688 Threads shared EMPTY_RESULT may lead to unexpected client job down (#4073) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-02-15 20:28:00 +08:00
chenglei 8bb36d6317
HBASE-26753 RegionReplicationSink.send set timeout for replicating meta edits error (#4110) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-02-14 17:12:29 +08:00
Nick Dimiduk 6c3c53a81c
HBASE-26521 Name RPC spans as `$package.$service/$method` (#4024) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-02-09 15:22:31 +01:00
Tak Lon (Stephen) Wu e848d3b9d9
HBASE-26714 Introduce path configuration for system coprocessors (#4069) 
Signed-off-by: Ankit Singhal <ankit@apache.org>
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
 2022-02-03 20:19:38 -08:00
Andrew Purtell 3da23c22c5
HBASE-26726 Allow disable of region warmup before graceful move (#4085) 
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
 2022-02-01 15:38:23 -08:00
Huaxiang Sun 351caa5bcc Revert "HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations() (#4044)" 
This reverts commit 8c607397c1.
 2022-01-31 09:48:46 -08:00
huaxiangsun 8c607397c1
HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations() (#4044) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-28 10:18:48 -08:00
Duo Zhang f278a4c98d
HBASE-26587 Introduce a new Admin API to change SFT implementation (#4030) 
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
Reviewed-by: Josh Elser <elserj@apache.org>
 2022-01-28 21:07:55 +08:00
Duo Zhang f6348d4100
HBASE-26709 Ban the usage of junit 3 TestCase (#4065) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-01-27 13:41:48 +08:00
Duo Zhang e1c2c16214
HBASE-26690 Modify FSTableDescriptors to not rely on renaming when writing TableDescriptor (#4054) 
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
 2022-01-27 13:28:56 +08:00
Duo Zhang 8251bd566a
HBASE-26700 The way we bypass broken track file is not enough in StoreFileListFile (#4055) 
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
 2022-01-25 14:51:11 +08:00
Nick Dimiduk 625d610bcc
HBASE-26614 Refactor code related to "dump"ing ZK nodes (#3969) 
The code starting at `ZKUtil.dump(ZKWatcher)` is a small mess – it has cyclic dependencies woven
through itself, `ZKWatcher` and `RecoverableZooKeeper`. It also initializes a static variable in
`ZKUtil` through the factory for `RecoverableZooKeeper` instances. Let's decouple and clean it
up.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2022-01-24 11:33:18 -08:00
Duo Zhang 97f3c1cf7f
HBASE-26675 Data race on Compactor.writer (#4035) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2022-01-24 19:45:50 +08:00
sunyuting1 1d2b8a033d
HBASE-26661 Remove deprecated methods in MasterObserver (#4031) 
Signrd-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-19 19:25:06 +08:00
Duo Zhang d63ca4febe
HBASE-26674 Should modify filesCompacting under storeWriteLock (#4040) 
Signed-off-by: Josh Elser <elserj@apache.org>
 2022-01-19 13:59:35 +08:00
Duo Zhang c9bcd87b34
HBASE-26654 ModifyTableDescriptorProcedure shoud load TableDescriptor while executing (#4034) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2022-01-16 16:25:28 +08:00
xianjingfeng bc5a5af1bc
HBASE-26671 Misspellings of hbck usage (#4033) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-16 16:23:23 +08:00
Yutong Xiao 3a14cfc6f9
HBASE-26551 Add FastPath feature to HBase RWQueueRpcExecutor (#3929) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2022-01-13 11:11:12 +08:00
Xiaolin Ha 803afee777
HBASE-26397 Display the excluded datanodes on regionserver UI (#3990) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-10 12:53:05 +08:00
Duo Zhang 24b9876edf HBASE-26643 Addendum align the test code with branch-2 2022-01-08 00:39:06 +08:00
Duo Zhang 69520ad199 HBASE-26643 Addendum align the test code with branch-2 2022-01-08 00:37:25 +08:00
Duo Zhang 443bf30a5d
HBASE-26643 LoadBalancer should not return empty map (#4005) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2022-01-07 23:50:37 +08:00
Duo Zhang 436fb7875c
HBASE-26586 Should not rely on the global config when setting SFT implementation for a table while upgrading (#4006) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
 2022-01-07 23:39:18 +08:00
Duo Zhang 47983cf790
HBASE-26639 The implementation of TestMergesSplitsAddToTracker is problematic (#4010) 
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
 2022-01-07 23:05:47 +08:00
Nick Dimiduk 5e263dac38
HBASE-26616 Refactor code related to ZooKeeper authentication (#3973) 
This refactor reduces the size and scope of the `ZKUtil` class. The core of this refactor is
moving the `login*` methods from `ZKUtil` into their own class, `ZKAuthentication`. The class
`JaasConfiguration` is also moved along with them.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-06 17:04:15 -08:00
Nick Dimiduk 5cb7ed8005
HBASE-26474 Implement connection-level attributes (#3952) 
Add support for `db.system`, `db.connection_string`, `db.user`.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-05 15:32:12 -08:00
Wellington Ramos Chevreuil baeb51ff8a
HBASE-26585 Add SFT configuration to META table descriptor when creating META (#3998) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2022-01-05 12:20:07 +00:00
Xiaolin Ha ce2bbaf30a
HBASE-26578 CompactedHFilesDischarger should be shutdown when aborting regionserver (#3946) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-05 11:10:16 +08:00
chenglei 70cb9b0ba9
HBASE-26567 Remove IndexType from ChunkCreator (#3947) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-05 00:05:54 +08:00
Duo Zhang e96df8bf40
HBASE-26641 Split TestMasterFailoverWithProcedures (#3994) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2022-01-03 00:35:14 +08:00
chenglei 1b8e577cc6
HBASE-26494 Using RefCnt to fix the flawed MemStoreLABImpl (#3983) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-01-01 20:50:52 +08:00
Duo Zhang 9e469e4a2a HBASE-26540 Implement a tool to verify the lag of region replication (#3950) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2021-12-31 12:29:48 +08:00
Duo Zhang 5ac76c1c00 HBASE-26539 The default rpc timeout 200ms is too small for replicating meta edits (#3919) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-12-31 12:29:48 +08:00
Duo Zhang 1fdd0a4cfd HBASE-26538 Should find a way to clear the replication queue for a legacy region_replica_replication peer (#3918) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2021-12-31 12:29:48 +08:00
Duo Zhang fa37aed8f6 HBASE-26481 Consider rolling upgrading from old region replication framework (#3880) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
Reviewed-by: GeorryHuang <huangzhuoyue@apache.org>
 2021-12-31 12:29:48 +08:00
林尚泉 9f6dfe6d3f HBASE-26417 Remove NoopReplicationQueueStorage (#3882) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-12-31 12:29:48 +08:00