11b327fddc
HBASE-27368 Do not need to throw IllegalStateException when peer is not active in ReplicationSource.initialize ( #4779 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 460bb47c04
2022-09-15 15:35:33 +08:00
a303a9fd71
HBASE-27317 Rectifying the option for columnfamily as mandatory ( #4773 )
...
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2022-09-14 13:43:49 +02:00
7ae66a1d2e
HBASE-27360 The trace related assertions are flaky for async client tests ( #4767 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit f3f88ff6c1
2022-09-09 23:03:05 +08:00
a1fc483ba8
HBASE-27313: Persist list of Hfiles names for which prefetch is done ( #4771 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-09-08 20:52:50 +01:00
88d1bd0942
HBASE-27362 CompactSplit.requestCompactionInternal may bypass compactionsEnabled check ( #4768 )
...
Co-authored-by: comnetwork <comnetwork@163.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit a26cbf1057
2022-09-08 16:43:52 +08:00
bfd8501e23
HBASE-27314 Make index block be customized and configured ( #4763 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-08 09:27:53 +08:00
b103685a1b
HBASE-27224 HFile tool statistic sampling produces misleading results ( #4638 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Clay Baenziger <cwb@clayb.net>
2022-09-06 15:23:18 -04:00
18d03058aa
HBASE-27332 Remove RejectedExecutionHandler for long/short compaction thread pools ( #4731 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-05 21:14:08 +08:00
0f29423cc5
HBASE-27340 Artifacts with resolved profiles ( #4740 )
...
Make it so our published poms carry the minimum needed to run
an hbase; the published pom has no profiles -- the profiles
specified at build time are resolved, their dependencies inlined,
and then they are stripped -- and no build-time, or plugins
dependencies or properties, etc. Resultant poms have explicit
hadoop lib versions baked in -- no more being able to choose
hbase with hadoop2 or haddop3 at downstream build time by setting
a '-Dhadoop.profile=X.0'.
Pattern is to add profiles when none in sub-modules when
the flatten plugin complains it can't resolve an hadoop
dependency's 'version' (e.g. hadoop-common, hadoop-hdfs).
Adding the hadoop-2.0 and hadoop-3.0 profiles in the sub-module
make it so the flatten plugin can figure 'hadoop.version'
definitively.
Another spin on the above happens when profiles already exist
in submodule but the flatten plugin is complaining it can't
figure figure version on an hadoop dependency NOT under
profiles. Below, we move the delinquent hadoop dependency under
existing profiles (minikdc was the usual dependency outside
profiles in sub-modules that flatten complained about).
Sometimes, moving an hadoop dependency under a profile, there
would be excludes on the local dependency. If the parent pom
excludes section was missing the local excludes, we added them
up to the parent module so all excluding is done up there in
the parent profile dependencyManagement section.
* hbase-asyncfs/pom.xml
* hbase-endpoint/pom.xml
* hbase-examples/pom.xml
* hbase-http/pom.xml
* hbase-rest/pom.xml
* hbase-server/pom.xml
Move the minikdc under profiles so it picks up appropriate hadoop version
when the flatten plugin runs.
* hbase-hadoop2-compat/pom.xml
Add hadoop2 and hadoop3 profiles and move hadoop-common, etc.
under them so we pick up appropriate hadoop version when flatten
plugin runs.
* hbase-mapreduce/pom.xml
Move hadoop dependencies under profiles so right version is
available when the flatten plugin runs.
* hbase-shaded/hbase-shaded-testing-util/pom.xml
Add profiles for hadoop-2.0 and hadoop-3.0 and move the
hadoop dependencies under them.
pom.xml
Add the flatten plugin with the flatten profiles enabled.
Add a few excludes on hadoop profiles picked up from sub-modules.
E.g. exclude bouncycastle bcprov-jdk15 when we include minikdc.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-02 16:44:27 -07:00
9838c070aa
HBASE-27342 Use Hadoop Credentials API to retrieve passwords of TLS key/trust stores ( #4751 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2022-09-01 10:06:15 +02:00
a7dab37b7f
HBASE-22939 SpaceQuotas - Bulkload from different hdfs failed when space quotas are turned on. ( #4748 )
...
Signed-off-by: Sakthi <sakthi@apache.org>
2022-08-31 15:47:52 +02:00
bd032460b8
HBASE-27152 Under compaction mark may leak ( #4725 ) ( #4744 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-08-30 01:19:09 +08:00
2e53a517a2
HBASE-27336 The region visualizer shows 'undefined' region server ( #4736 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit fa02350a27
2022-08-28 22:52:49 +08:00
935b0d2277
HBASE-27321 The ReplicationLogCleaner is not thread safe but can be called from different threads at the same time ( #4730 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 37651ee1b0
2022-08-28 00:30:34 +08:00
c71e984a1f
HBASE-25922 - Disabled sanity checks ignored on snapshot restore ( #4533 )
...
Co-authored-by: Ujjawal Kumar <u.kumar@ukumar-ltmit1s.internal.salesforce.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
2022-08-26 08:57:12 -04:00
d92d533924
HBASE-27325 the bulkload max call queue size can be update to a wrong value ( #4728 )
...
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 92cf962fee
2022-08-25 11:00:28 +08:00
f66d67bbdf
HBASE-27305 add an option to skip file splitting when bulkload hfiles ( #4709 )
...
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 00a719e76f
2022-08-22 22:45:30 +08:00
1bd0b581cb
HBASE-27104 Add a tool command list_unknownservers ( #4714 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-08-22 21:15:47 +08:00
fc93fbac9a
HBASE-27129 Add a config that allows us to configure region-level storage policies ( #4547 )
...
Co-authored-by: Tang Tianhang <tianhang.tang@shopee.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f238a92b6d
2022-08-22 10:07:27 +08:00
f705e3e7d8
HBASE-27206 Clean up error-prone findings in hbase-common ( #4645 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit fbe3b90e0c
2022-08-21 21:36:01 +08:00
8a10dbf6c4
HBASE-27268 In trace log mode, the client does not print callId/startTime and the server does not print receiveTime ( #4710 )
...
Co-authored-by: jay.zhu <jay.zhu@huolala.cn>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 32c135d8a5
2022-08-20 21:09:43 +08:00
d073404706
HBASE-26969:Eliminate MOB renames when SFT is enabled
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
closes #4712
2022-08-19 11:33:48 +02:00
1176b484d1
HBASE-25970 MOB data loss - incorrect concatenation of MOB_FILE_REFS ( #3355 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2022-08-19 11:20:53 +02:00
1986a6ec31
HBASE-24163 MOB compactor implementations should use format specifiers when calling String.format
...
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2022-08-17 13:46:18 +02:00
c2dff398d3
HBASE-27279 Make SslHandler work with SaslWrapHandler/SaslUnwrapHandler ( #4705 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Reviewed-by: Andor Molnár <andor@cloudera.com>
(cherry picked from commit 2b9d36869f
2022-08-16 21:37:47 +08:00
3acf920e1f
HBASE-27264 Add options to consider compressed size when delimiting blocks during hfile writes ( #4675 )
...
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Ankit Singhal <ankit@apache.org>
2022-08-15 22:36:44 +01:00
aaad3a7077
HBASE-26982 Add index and bloom filter statistics of LruBlockCache on… ( #4376 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: stack <stack@apache.org>
2022-08-15 11:12:05 -07:00
b25ded026b
HBASE-27296 Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose ( #4695 ) ( #4701 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-15 09:16:29 -07:00
f36b34984f
HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone ( #4617 )
...
* HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#1446 )
* Reorganize MOB compaction tests for more reuse.
* Add tests for mob compaction after snapshot clone operations
* note the original table used to write a given mob hfile and use that to find it later.
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
* spotless:apply to fix HBaseTestingUtility
* Fix error-prone errors
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
Co-authored-by: Sean Busbey <busbey@apache.org>
2022-08-12 14:29:26 -07:00
b820188c68
HBASE-27278 Improve TestTlsIPC to reuse existing IPC test code ( #4682 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit 3309108ca7
2022-08-12 13:21:11 +08:00
25e192b807
HBASE-27248 WALPrettyPrinter add print timestamp ( #4659 )
...
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6b2e9d14a7
2022-08-10 21:30:55 +08:00
e99faad83e
HBASE-27241 Add metrics for evaluating cost and effectiveness of bloom filters ( #4669 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-09 15:25:47 -04:00
beb7c6d490
HBASE-27283 Use readTO instead of hard coded RpcClient.DEFAULT_SOCKET_TIMEOUT_READ when creating ReadTimeoutHandler in NettyRpcConnection ( #4685 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 5919b30b6d
2022-08-09 16:22:57 +08:00
30f6a17056
HBASE-27282 CME in AuthManager causes region server crash ( #4684 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit d6641626a9
2022-08-09 09:56:26 +08:00
a85de838ab
HBASE-27265 : Tool to read StoreFileTrackerFile ( #4673 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-08-08 15:31:14 +01:00
c81517638c
HBASE-27270 Add bulkloadExecutor.resizeQueues ( #4677 )
...
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 4ebf71982c
2022-08-08 18:15:25 +08:00
2821cc518a
HBASE-26666 Add native TLS encryption support to RPC server/client ( #4666 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit f8dcf07055
2022-08-06 19:43:17 -04:00
2e7a983923
HBASE-27273 Should stop autoRead and skip all the bytes when rpc request too big ( #4679 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 486d19e99f
2022-08-06 20:31:27 +08:00
c5b22d973b
HBASE-27269 The implementation of TestReplicationStatus.waitOnMetricsReport is incorrect ( #4678 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 2dc26082be
2022-08-04 23:14:24 +08:00
bcec871959
HBASE-27153 Improvements to read-path tracing
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-08-02 14:09:39 +02:00
dbad3d1175
HBASE-27257 Remove unnecessary usage of CachedBlocksByFile from RS UI ( #4667 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-07-29 15:28:54 -04:00
5f231d3fb3
HBASE-27185 Rewrite NettyRpcServer to decode rpc request with netty handler ( #4624 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 0c4263a18b
2022-07-27 09:10:32 +08:00
4c08a8a423
HBASE-27087 TestQuotaThrottle times out
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-26 14:10:49 -07:00
827c1ab5dd
HBASE-27229 BucketCache statistics should not count evictions by hfile ( #4639 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-26 17:00:46 -04:00
84e7cdb2c0
HBASE-27222 Purge FutureReturnValueIgnored warnings from error prone ( #4634 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 8b091c4061
2022-07-26 23:45:15 +08:00
1fd772c3b1
HBASE-27225 Add BucketAllocator bucket size statistic logging ( #4637 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-07-25 20:56:05 -04:00
80c33e0d46
HBASE-27232 Fix checking for encoded block size when deciding if bloc… ( #4640 )
...
Signed-off-by: Andor Molnár <andor@cloudera.com>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Signed-off-by: Ankit Singhal <ankit@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit d5ed8f59e5
2022-07-25 12:08:09 +01:00
db98fab114
HBASE-27075 TestUpdateRSGroupConfiguration.testCustomOnlineConfigChangeInRSGroup is flaky ( #4636 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 31fc97edee
2022-07-21 14:52:39 +08:00
468715bff3
Revert "HBASE-27152 Under compaction mark may leak ( #4568 )"
...
This reverts commit 5c5ceb1ef2
2022-07-21 09:04:17 +08:00
5c5ceb1ef2
HBASE-27152 Under compaction mark may leak ( #4568 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit da27a67a1e
2022-07-20 08:51:58 +08:00
eb73492eac
HBASE-20499 Replication/Priority executors can use specific max queue length as default value instead of general maxQueueLength
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
(cherry picked from commit 6d080762ef
2022-07-19 13:48:07 +08:00
642e933cda
HBASE-27144 Addendum fix spotbug warning
2022-07-19 13:41:07 +08:00
99f2ab5aa8
HBASE-27220 Apply the spotless format change in HBASE-27208 to our code base
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-19 10:00:31 +08:00
d19805a7bc
HBASE-27195 Clean up netty worker/thread pool configuration ( #4619 )
...
The configuration settings "hbase.netty.worker.count" and
"hbase.netty.eventloop.rpcserver.thread.count" appear to duplicate each
other.
Also, formalizes another setting found in NettyEventLoopGroupConfig,
"hbase.netty.nativetransport".
Also, native epoll is not limited to amd64. aarch64 supports it too.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
2022-07-18 17:42:37 -07:00
f76c0d9021
HBASE-27202 Clean up error-prone findings in hbase-balancer ( #4623 )
...
On branch-2, the affected code is in hbase-server.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/RegionLocationFinder.java
2022-07-18 16:00:26 -07:00
824785eeeb
HBASE-27211 Data race in MonitoredTaskImpl could cause split wal failure ( #4630 )
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 9ab0b1504f
2022-07-18 20:08:38 +08:00
325f90bb40
HBASE-27144 Add special rpc handlers for bulkload operations ( #4558 )
...
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit ff8eb59709
2022-07-17 21:41:27 +08:00
ca15d39026
HBASE-27053 IOException during caching of uncompressed block to the block cache ( #4610 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: wenwj0 <wenweijian2@huawei.com>
2022-07-16 17:33:27 -04:00
8d3693a018
HBASE-27192 The retry number for TestSeparateClientZKCluster is too small ( #4614 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 648bb6d0eb
2022-07-14 08:52:12 +08:00
70a561eb63
HBASE-27194 Add test coverage for SimpleRpcServer ( #4616 )
...
Add test coverage for SimpleRpcServer.
Improve the way we test both SimpleRpcServer and NettyRpcServer. Use
LoadTestKVGenerator to generate random values with varying sizes between
1000 bytes and 1M bytes, and also to verify them when reading the values
back.
Add secure test coverage for both SimpleRpcServer and NettyRpcServer.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-07-13 11:22:29 -07:00
47230c4727
HBASE-27193 TestZooKeeper is flaky ( #4615 )
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
(cherry picked from commit 2662607b71
2022-07-13 10:34:15 +08:00
fbfe7b3880
HBASE-27161 Improve TestMultiRespectsLimits ( #4586 )
...
TestMultiRespectLimits submits a put with an empty value. This appears
to not be what was intended. Instead, use a single byte.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-07-12 11:20:56 -07:00
0018cbec58
HBASE-27097 SimpleRpcServer is broken ( #4613 )
...
Replace BufferChain#write(channel,int) with a simpler #write(channel)
implementation that does not attempt to "chunk" data to be written. This
method was used exclusively by SimpleRpcServer. The code was unnecessarily
complex and caused short writes when values were large, so was corrected
and simplified. Any difference in performance from this change will be
limited to SimpleRpcServer. Testing under load confirms the fix and does
not show significant regression.
SimpleRpcServer and its related code is now also marked as @Deprecated.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
2022-07-12 11:09:05 -07:00
b2a145661f
HBASE-27149 Server should close scanner if client times out before results are ready ( #4604 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-11 22:36:42 -04:00
98ec7586b4
HBASE-27188 Report maxStoreFileCount in jmx ( #4609 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-11 22:14:01 -04:00
9ccb0f96b9
HBASE-27186 Report block cache size metrics separately for L1 and L2 ( #4608 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-11 22:01:06 -04:00
68f539ef94
HBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel is closed ( #4611 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 5bc8670322
2022-07-12 09:44:18 +08:00
e7551c1359
HBASE-26950 Use AsyncConnection in ReplicationSink ( #4595 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2022-07-09 10:25:17 +08:00
6dd1b58481
HBASE-27148 Move minimum hadoop 3 support version to 3.2.3 ( #4561 ) ( #4599 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 41972cb460
2022-07-08 15:59:25 +08:00
11283cac96
HBASE-27078 Allow configuring a separate timeout for meta scans ( #4585 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-07 16:33:16 -04:00
b543da974a
HBASE-26708 Netty leak detected and OutOfDirectMemoryError due to direct memory buffering with SASL implementation ( #4596 )
...
Co-authored-by: Norman Maurer <norman_maurer@apple.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 816e919e95
2022-07-07 16:04:21 +08:00
1c90b4344d
Revert "HBASE-27157 Potential race condition in WorkerAssigner ( #4577 )"
...
This reverts commit d79d812dd0
2022-07-07 08:38:43 +08:00
fadf48c40f
HBASE-27175 - Failure to cleanup WAL split dir log should be at INFO level ( #4593 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-07-06 16:34:21 -07:00
f213c52d96
HBASE-27048 Addendum spotless fix
2022-07-06 13:53:35 -04:00
8fcb94ae8a
HBASE-27048 Server side scanner time limit should account for time in queue ( #4562 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-07-06 13:46:37 -04:00
d79d812dd0
HBASE-27157 Potential race condition in WorkerAssigner ( #4577 )
...
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
(cherry picked from commit f76d8554ca
2022-07-06 12:39:48 +08:00
b59c8b8dcb
HBASE-27171 Fix Annotation Error in HRegionFileSystem ( #4588 )
...
Co-authored-by: Tang Tianhang <tianhang.tang@shopee.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6031a3a8d4
2022-07-06 12:39:06 +08:00
e047ef8c68
HBASE-23330: Fix delegation token fetch with MasterRegistry ( #1084 ) ( #4598 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit d8b3f55518
2022-07-06 08:10:50 +08:00
57a3781160
HBASE-27170 ByteBuffAllocator leak when decompressing blocks near minSizeForReservoirUse ( #4592 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-07-04 16:37:29 -04:00
20f247934a
HBASE-27169 TestSeparateClientZKCluster is flaky ( #4587 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 03c23beb57
2022-07-01 22:13:21 +08:00
0727015bf5
HBASE-22749 Distributed MOB compactions ( #4581 )
...
* HBASE-22749 Distributed MOB compactions
- MOB compaction is now handled in-line with per-region compaction on region
servers
- regions with mob data store per-hfile metadata about which mob hfiles are
referenced
- admin requested major compaction will also rewrite MOB files; periodic RS
initiated major compaction will not
- periodically a chore in the master will initiate a major compaction that
will rewrite MOB values to ensure it happens. controlled by
'hbase.mob.compaction.chore.period'. default is weekly
- control how many RS the chore requests major compaction on in parallel
with 'hbase.mob.major.compaction.region.batch.size'. default is as
parallel as possible.
- periodic chore in master will scan backing hfiles from regions to get the
set of referenced mob hfiles and archive those that are no longer
referenced. control period with 'hbase.master.mob.cleaner.period'
- Optionally, RS that are compacting mob files can limit write
amplification by not rewriting values from mob hfiles over a certain size
limit. opt-in by setting 'hbase.mob.compaction.type' to 'optimized'.
control threshold by 'hbase.mob.compactions.max.file.size'.
default is 1GiB
- Should smoothly integrate with existing MOB users via rolling upgrade.
will delay old MOB file cleanup until per-region compaction has managed
to compact each region at least once so that used mob hfile metadata can
be gathered.
* HBASE-22749 Distributed MOB compactions
fix RestrictedApi
Co-authored-by: Vladimir Rodionov <vrodionov@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-06-30 20:44:45 +01:00
d2509d709b
HBASE-26218 Add logs in Canary tool ( #4501 )
...
Co-authored-by: Ishika Soni <isoni@isoni-ltmag9r.internal.salesforce.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit acfbc3ff7d
2022-06-30 23:28:26 +08:00
e4d2895cad
HBASE-27166 WAL value compression minor improvements ( #4584 )
...
A larger IO buffer for absorbing WALCodec writes can improve the compression
ratio of larger values, because the compressor will be given a larger internal
buffer over which there will be more match opportunities. Does not impact the
ability to read existing written files.
Also, reset the BAOS internal buffer on the way out of compress() so potential
large-ish buffers do not linger on the heap longer than necessary.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-06-28 12:53:30 -07:00
bd08ebb21c
HBASE-27160 ClientZKSyncer.deleteDataForClientZkUntilSuccess should break from the loop when deletion is succeeded ( #4579 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit d7f6861e8d
2022-06-26 15:27:28 +08:00
f06ac9192b
HBASE-26790 getAllRegionLocations can cache locations with null hostname ( #4575 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-24 07:40:42 -04:00
da76941b0e
HBASE-27111 Make Netty channel bytebuf allocator configurable. ( #4525 )
...
Support site configuration of the bytebuf allocator that Netty will use for
NettyRpcServer channels. Property name is 'hbase.netty.rpcserver.allocator'.
Default is no value, which is equivalent to "pooled". Valid values are:
- "pooled": use PooledByteBufAllocator
- "unpooled": use UnpooledByteBufAllocator
- "heap": use HeapByteBufAllocator, which is a PooledByteBufAllocator that
preferentially allocates buffers on heap wherever possible
- <class>: If the value is none of the recognized labels, treat it as a class
name implementing org.apache.hbase.thirdparty.io.netty.buffer.ByteBufAllocator.
This allows the user to add a custom implementation, perhaps for debugging.
Also updates ReflectionUtils with a new helper method.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestNettyRpcServer.java
2022-06-23 15:18:32 -07:00
409a7a44e8
HBASE-26945 Quotas causes too much load on meta for large clusters ( #4576 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-06-23 16:47:09 -04:00
d129b56f26
HBASE-27001 The deleted variable cannot be printed out ( #4479 )
...
* The deleted variable cannot be printed out, add it
Co-authored-by: selina.yan <selina.yan@huolala.cn>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
(cherry picked from commit b8558d30d0
2022-06-23 22:43:05 +05:30
c74bf8b6e2
HBASE-27105 HBaseInterClusterReplicationEndpoint should honor replication adaptive timeout ( #4569 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-06-23 22:22:57 +05:30
ed96746747
HBASE-27098 Fix link for field comments ( #4499 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-06-21 18:30:50 -07:00
666aa064e7
HBASE-27028 Add a shell command for flushing master local region ( #4539 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-06-21 13:25:35 +08:00
18400ac3bd
HBASE-27099 The log printing fspread/fsread cost time unit should be milliseconds ( #4500 )
...
Co-authored-by: xiaozhang <issac.zhang@huolala.cn>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b498efdbc1
2022-06-21 00:14:23 +08:00
e1de804ba1
HBASE-27128 when open archiveRetries totalLogSize calculation mistake ( #4546 )
...
Co-authored-by: zhengsicheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 5833756e5e
2022-06-20 23:48:45 +08:00
f7d2d12ac1
HBASE-27125 The batch size of cleaning expired mob files should have an upper bound ( #4541 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-06-18 02:07:38 +08:00
ef0ff2729c
HBASE-27117 Update the method comments for RegionServerAccounting ( #4532 )
2022-06-16 17:19:38 -07:00
b8f0c61ed7
Revert "HBASE-25709 Close region may stuck when region is compacting and skipped most cells read ( #3117 )" ( #4524 )
...
This reverts commit f3a48d1910
2022-06-12 10:24:02 -07:00
9b8bb580b9
HBASE-26366 Provide meaningful parent spans to ZK interactions
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-10 16:26:37 +02:00
1dcee76e07
HBASE-27095 HbckChore should produce a report
...
In #4470 for HBASE-26192, it was noted that the HbckChore is kind of a pain to use and test
because it maintains a bunch of local state. By contract, the CatalogJanitorChore makes a nice
self-contained report. Let's update HbckChore to do the same.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-06-10 15:46:48 +02:00
b3c9ef34b7
HBASE-27066 The Region Visualizer display failed
...
Reviewed-by: litao <tomleescut@gmail.com>
2022-06-09 13:09:25 +02:00
33f633a0b1
Revert "HBASE-27066 The Region Visualizer display failed ( #4472 )"
...
This reverts commit 5e54c6daf6
2022-06-09 13:09:25 +02:00
63a35facad
HBASE-27092 Regionserver table on Master UI is broken ( #4495 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-06-07 17:39:27 -07:00
fd6e6e0cbb
HBASE-27080 Optimize debug output log of ConstantSizeRegionSplitPolicy class. ( #4481 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 12a9e6e5ec
2022-06-06 13:41:57 +08:00
ef221846e5
HBASE-27017: MOB snapshot is broken when FileBased SFT is used ( #4466 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-06-05 20:01:57 +01:00
286c9c3327
HBASE-26985 check permission for SecureBulkLoadManager ( #4379 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-06-02 15:29:43 -07:00
58aaed19b0
Revert "HBASE-26962: Add mob info in web UI ( #4359 )"
...
This reverts commit 7a02cbfd7e
2022-06-02 14:04:27 -07:00
672b0a5f5c
HBASE-26649 Support meta replica LoadBalance mode for RegionLocator#getAllRegionLocations() ( #4442 ) ( #4464 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-06-02 12:35:11 -07:00
f97af48d30
HBASE-27046 The filenum in AbstractFSWAL should be monotone increasing ( #4449 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit 313a37f7ac
2022-06-03 00:10:53 +08:00
f61ac22f02
HBASE-26680 Close and do not write trailer for the broken WAL writer(addendum) ( #4405 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 80f410ec3f
2022-06-03 00:08:30 +08:00
7a02cbfd7e
HBASE-26962: Add mob info in web UI ( #4359 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit d03004060d
2022-06-03 00:05:22 +08:00
8270b5df5e
HBASE-27039 Some methods of MasterRegion should be annotated for testing only ( #4433 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 15002fccb4
2022-06-02 21:38:59 +08:00
b21930f3fa
HBASE-27037 Display the region server state on the Web UI ( #4432 )
...
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit db992083af
2022-06-02 21:33:07 +08:00
59700babd0
HBASE-27079 Lower some DEBUG level logs in ReplicationSourceWALReader to TRACE ( #4476 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-05-31 12:43:41 -07:00
f4f3e63f9f
HBASE-27068 NPE occurs when the active master has not yet been elected ( #4474 )
...
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
Signed-off-by: Viraj Jasani<virajjasani@apache.org
2022-05-31 11:50:33 -07:00
cfba2f2b14
HBASE-27064 CME in TestRegionNormalizerWorkQueue ( #4468 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-05-31 11:33:38 -07:00
7fbfb10649
HBASE-27069 Hbase SecureBulkload permission regression ( #4475 )
...
Signed-off-by: Rajeshbabu Chintaguntla <rajeshbabu@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 4ce85aa39f
2022-05-31 10:07:23 +01:00
5e54c6daf6
HBASE-27066 The Region Visualizer display failed ( #4472 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-29 11:56:32 -07:00
50dd9619b0
HBASE-27061 two phase bulkload is broken when SFT is in use. ( #4465 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-05-26 11:11:18 +01:00
56b81b42d8
HBASE-26342 Support custom paths of independent configuration and pool for hfile cleaner ( #4403 ) ( #4461 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-24 15:51:52 +08:00
200cbcc34f
HBASE-26320 Implement a separate thread pool for the LogCleaner ( #3712 ) ( #4460 )
...
This avoids starvation when the archive directory is large and takes a long time
to iterate through.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
2022-05-24 14:26:09 +08:00
b00e3032a6
HBASE-27054 TestStochasticLoadBalancerRegionReplicaLargeCluster.testRegionReplicasOnLargeCluster is flaky ( #4454 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplicaLargeCluster.java
2022-05-21 09:23:05 -07:00
8bc55f8308
HBASE-27043 Let lock wait timeout to improve performance of SnapshotHFileCleaner ( #4437 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-20 15:34:17 -07:00
2cae259f36
HBASE-27052 TestAsyncTableScanner.testScanWrongColumnFamily is flaky ( #4455 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-20 15:13:57 -07:00
7af82cf164
HBASE-27051 TestReplicationSource.testReplicationSourceInitializingMetric is flaky ( #4448 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-05-20 15:11:19 -07:00
b9b5302929
HBASE-27045 Disable TestClusterScopeQuotaThrottle ( #4440 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
(cherry picked from commit 487cf301a4
2022-05-18 20:47:20 +08:00
1e6d93a789
HBASE-26986 Trace a one-shot execution of a Master procedure
...
This one adds a tracing context to the Master's ProcedureV2 execution. This ties all the spans
that result from a procedure execution back to a procedure by name. Maybe in a follow-on issue, we
can serialize span context along with procedures and display procedure start/stop/retry events and
parent-child hierarchy in tracing visualization.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-05-18 13:08:27 +02:00
31f4b422ab
HBASE-27032 The draining region servers metric description is incorrect ( #4428 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
(cherry picked from commit ed1665355a
2022-05-17 10:38:15 +05:30
7bc7eb5f48
HBASE-26905 ReplicationPeerManager#checkPeerExists should throw ReplicationPeerNotFoundException if peer doesn't exists ( #4422 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-13 18:25:19 -07:00
43e8c11989
HBASE-27013 Introduce read all bytes when using pread for prefetch ( #4414 )
...
- introduce optional flag `hfile.pread.all.bytes.enabled` for pread that must read full bytes with the next block header
Signed-off-by: Josh Elser <elserj@apache.org>
2022-05-13 13:14:17 -07:00
62dbae2914
HBASE-26999 Addendum fix spotless errors
2022-05-13 09:57:26 +08:00
930bc5368c
HBASE-27021 StoreFileInfo should set its initialPath in a consistent way
...
(Amending spotless errors after revert by Duo)
2022-05-12 10:53:19 +01:00
e3359efe7a
Revert "HBASE-27021 StoreFileInfo should set its initialPath in a consistent way ( #4419 )"
...
Revert due to failure spotless:check
This reverts commit aa31b0f695
2022-05-12 10:53:10 +01:00
b7647d8d5f
HBASE-27021 StoreFileInfo should set its initialPath in a consistent way ( #4419 )
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-05-12 10:52:47 +01:00
7b7f57619b
HBASE-26999 HStore should try write WAL compaction marker before repl… ( #4407 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-09 11:00:26 +01:00
ad2180b75d
HBASE-26994 MasterFileSystem create directory without permission check ( #4391 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-05-08 13:28:40 -07:00
f5a08063b6
HBASE-26963 ReplicationSource#removePeer hangs if we try to remove bad peer. ( #4413 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-05-07 10:38:17 -07:00
91eab0b657
HBASE-27000 Block cache stats (Misses Caching) display error in RS web UI ( #4406 )
...
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-05-05 23:35:09 -07:00
cb61d6a4bd
HBASE-26977 HMaster's ShutdownHook does not take effect, if tablesOnMaster is false ( #4397 )
...
Signed-off-by: Yu Li <liyu@apache.org>
2022-05-05 21:55:50 +08:00
1b5c881ffb
HBASE-26648 Improve fidelity of RegionLocator spans (addendum)
...
Addendum extends the test case to cover both `RpcConnectionRegistry` and `ZKConnectionRegistry`.
2022-05-03 13:40:42 +02:00
a38716ca2a
HBASE-26648 Improve fidelity of RegionLocator spans
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-05-03 13:40:42 +02:00
be8a9a5920
HBASE-26989 TestStochasticLoadBalancer fixes for performance and consistency ( #4385 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Reviewed by: Rushabh Shah <shahrs87@gmail.com>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
2022-05-02 10:51:37 -07:00
1a5b1b266c
HBASE-26899 Run spotless:apply
2022-05-01 22:41:49 +08:00
afb8afd26c
HBASE-26988 dynamic configuration of loadbalance.bytable ( #4384 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
2022-04-29 15:55:16 -07:00
e9c9418d1d
HBASE-26943 HMaster page style display confusion ( #4369 )
...
Signed-off-by: Yu Li <liyu@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-04-29 15:59:53 +08:00
fd2e7205ac
HBASE-22349 slop in StochasticLoadBalancer ( #4371 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/BalancerTestBase.java
2022-04-28 12:38:50 -07:00
895e0f474a
HBASE-26975 Add on heap and off heap memstore info in rs web UI ( #4368 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit cdf81ea5cc
2022-04-28 23:15:10 +08:00
ea5c5da1b3
HBASE-26980 Update javadoc of BucketCache.java ( #4374 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit c02e7553d4
2022-04-28 23:11:51 +08:00
6813a5c502
HBASE-26979 StoreFileListFile logs frequent stacktraces at INFO level ( #4373 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-04-27 18:02:18 -07:00
c2baa1fb7b
HBASE-26581 Add metrics for failed replication edits ( #4347 )
...
Co-authored-by: Briana Augenreich <baugenreich@hubspot.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2022-04-26 17:44:01 -04:00
c9d833ed5e
HBASE-26971 SnapshotInfo --snapshot param is marked as required even when trying to list all snapshots ( #4366 )
...
Signed-off-by: Josh Elser <elserj@apache.org>
(cherry picked from commit af13c6d4c6
2022-04-25 14:18:57 +01:00
8d17241ce0
HBASE-26942 cache region locations when getAllRegionLocations ( #4364 )
...
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2022-04-22 15:30:26 -04:00
2240025349
HBASE-26807 Unify CallQueueTooBigException special pause with CallDroppedException ( #4273 )
...
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-04-22 08:56:52 -04:00
085c65107e
HBASE-26941 LocalHBaseCluster.waitOnRegionServer should not call join while interrupted ( #4352 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 35aa57e445
2022-04-17 23:26:56 +08:00
ae8eccd5c0
HBASE-26938 Compaction failures after StoreFileTracker integration ( #4350 )
...
Introduce a StoreFileWriterCreationTracker to track the store files being written
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 48c4a4626e
2022-04-17 22:44:44 +08:00
bbe048c1e5
HBASE-26946 TestRetainAssignmentOnRestartSplitWithoutZk infinite loop ( #4344 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 2622fa05a6
2022-04-16 13:23:57 +08:00
c76daa1cc8
HBASE-26944 Possible resource leak while creating new region scanner ( #4339 )
...
* HBASE-26944 Possible resource leak while creating new region scanner
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 62e3efccef
2022-04-15 11:37:58 +05:30
b405a6f119
Revert "HBASE-26941 LocalHBaseCluster.waitOnRegionServer should quit while thread is interrupted ( #4333 )"
...
This reverts commit f8b2ac0518
2022-04-13 11:53:19 +08:00
f8b2ac0518
HBASE-26941 LocalHBaseCluster.waitOnRegionServer should quit while thread is interrupted ( #4333 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 8247b7c722
2022-04-13 11:10:25 +08:00
52ca440ad6
HBASE-26901 delete with null columnQualifier occurs NullPointerException when NewVersionBehavior is on ( #4295 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 7ac9e0be27
2022-04-12 20:31:48 +08:00
f9abe3d8f1
HBASE-26618 Involving primary meta region in meta scan with CatalogRe… ( #4321 ) ( #4326 )
...
Signed-off-by: Michael Stack <stack@apache.org>
2022-04-11 10:27:15 -07:00
e7eb628025
HBASE-26922 Fix LineLength warnings as much as possible if it can not be fixed by spotless ( #4324 )
...
Signed-off-by: Yulin Niu <niuyulin@apache.org
(cherry picked from commit 3ae0d9012c
2022-04-09 23:13:49 +08:00
197b8d48ec
HBASE-26928 Fix several indentation problems ( #4323 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit e68c61dd54
2022-04-06 15:14:03 +08:00
87555093cf
HBASE-26882 Backport "HBASE-26810 Add dynamic configuration support f… ( #4278 )
...
- Include HBASE-25288 that has the configuration manager registration in HMaster
Signed-off-by: Ankit Singhal <ankit@apache.org>
2022-04-05 10:06:13 -07:00
1c14046ee0
HBASE-26927 Add snapshot scanner UT with SFT and some cleanups to Tes… ( #4322 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
(cherry picked from commit ae3718be0f
2022-04-05 17:52:30 +01:00
8ef1ae32e6
HBASE-26885 Addendum throw exception instead of return in TRSP to let the procedure retry ( #4299 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 7f9d1f2259
2022-04-04 20:26:48 +08:00
3b383806be
HBASE-26924 Fix log parameter error and spelling error ( #4318 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
(cherry picked from commit 003c572246
2022-04-04 20:26:41 +08:00
ab9ccabe8f
HBASE-26921 Rewrite the counting cells part in TestMultiVersions ( #4316 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 3e09e87fad
2022-04-03 23:04:51 +08:00
1560b4149c
HBASE-26920 Fix missing braces warnings in TestProcedureMember ( #4315 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 78676bb85e
2022-04-03 23:04:43 +08:00
0b87621a38
HBASE-26919 Rewrite the counting rows part in TestFromClientSide4 ( #4314 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 4e12eee531
2022-04-03 23:04:36 +08:00
4311974758
HBASE-26811 Secondary replica may be disabled for read incorrectly forever ( #4308 )
2022-04-01 11:20:48 +08:00
489af42792
HBASE-26545 Implement tracing of scan
...
* on `AsyncTable`, both `scan` and `scanAll` methods should result in `SCAN` table operations.
* the span of the `SCAN` table operation should have children representing all the RPC calls
involved in servicing the scan.
* when a user provides custom implementation of `AdvancedScanResultConsumer`, any spans emitted
from the callback methods should also be tied to the span that represents the `SCAN` table
operation. This is easily done because these callbacks are executed on the RPC thread.
* when a user provides a custom implementation of `ScanResultConsumer`, any spans emitted from the
callback methods should be also be tied to the span that represents the `SCAN` table
operation. This accomplished by carefully passing the span instance around after it is created.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-31 17:29:51 +02:00
98db15acd2
HBASE-26245 Store region server list in master local region ( #4136 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit bb1bbddf7427111421d9
2022-03-31 22:26:40 +08:00
1a85588265
HBASE-26812 ShortCircuitingClusterConnection fails to close RegionSca… ( #4243 )
2022-03-31 20:04:01 +08:00
af5624bb8e
HBASE-26885 The TRSP should not go on when it get a bogus server name… ( #4276 )
...
* HBASE-26885 The TRSP should not go on when it get a bogus server name from AM
(cherry picked from commit 1efd8fe53c
2022-03-29 10:18:15 +08:00
0dbcf8f263
HBASE-26718 HFileArchiver can remove referenced StoreFiles from the archive ( #4274 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-28 11:47:11 -07:00
170570b799
HBASE-26872 Load rate calculator for cost functions should be more precise ( #4253 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
2022-03-28 23:38:30 +08:00
762390045b
Revert "HBASE-26872 Load rate calculator for cost functions should be more precise ( #4253 )"
...
This reverts commit 7ae6be7c25
2022-03-28 18:16:38 +08:00
fd621a7c1b
HBASE-26832 Avoid repeated releasing of flushed wal entries in AsyncFSWAL#syncCompleted ( #4281 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 4f491fd5e4
2022-03-28 07:59:58 +08:00
7ae6be7c25
HBASE-26872 Load rate calculator for cost functions should be more precise ( #4253 )
...
Signed-off-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
2022-03-27 11:13:55 +08:00
9ecd4e7c56
HBASE-26864 SplitTableRegionProcedure calls openParentRegions() at a … ( #4261 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-03-26 12:04:44 -07:00
66508074cc
HBASE-26881 Backport HBASE-25368 to branch-2 ( #4267 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-25 10:04:13 +00:00
b7cfc1d0bd
HBASE-26175 MetricsHBaseServer should record all kinds of Exceptions ( #4248 )
...
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2022-03-24 19:04:59 +08:00
1caa28869b
HBASE-21065 Try ROW_INDEX_V1 encoding on meta table ( #4268 )
...
Set encoding and blooms on meta as default. Also shutdown access to the
initial meta schema creating method; get from TableDescriptors if you
need access to schema or edit it as you would any other table if you
want to edit it.
Co-authored-by: Michael Stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-23 16:30:08 -07:00
5affa20359
HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallContext is null ( #4264 )
2022-03-23 18:31:47 +08:00
c55c289e99
Revert "HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallContext is null ( #4262 )" ( #4263 )
...
This reverts commit d889746bc6
2022-03-23 17:52:11 +08:00
d889746bc6
HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallContext is null ( #4262 )
2022-03-23 17:48:20 +08:00
68f3fbe035
HBASE-26791 Memstore flush fencing issue for SFT ( #4202 )
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit e56ed404cb
2022-03-22 12:35:09 +08:00
0d968af513
HBASE-26531 Trace coprocessor exec endpoints
...
Trace table ExecService invocations as table operations. Ensure span relationships for both table
and master invocations.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-21 19:29:37 +01:00
321c35a6ef
HBASE-26834 Adapt ConnectionRule for both sync and async connections
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-21 12:42:18 +01:00
7af08ce614
HBASE-26861 Fix flaky TestSnapshotFromMaster.testSnapshotHFileArchiving ( #4242 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 520ad638d9
2022-03-19 00:46:31 +08:00
ac5651c1b2
HBASE-26858 Refactor TestMasterRegionOnTwoFileSystems to avoid dead loop ( #4238 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 31db8e3b3b
2022-03-19 00:46:03 +08:00
4a6ba116bf
HBASE-26842 TestSnapshotProcedure fails in branch-2 ( #4225 )
...
Closes #4225
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-18 16:21:29 +08:00
81398f9064
HBASE-26833 Addendum fix NPE in TestReplicationSource.testTerminateClearsBuffer
...
(cherry picked from commit bd65ad42a2
2022-03-17 23:23:10 +08:00
db5e954068
HBASE-26680 Close and do not write trailer for the broken WAL writer ( #4174 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-16 20:29:27 +08:00
d3629bbf16
HBASE-26837 Set SFT config when creating TableDescriptor in TestCloneSnapshotProcedure ( #4226 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 5b28d76652
2022-03-16 20:16:26 +08:00
3c968a1bb8
HBASE-25896 Implement a Region Visualization on Master WebUI ( #4228 )
...
This is a demonstration of visualization of regions on the cluster. The visualization is a stacked
bar chart showing total storefile size per table per region server, with the x-axis being server
names, the y-axis being storfile size, and the bars stacked per table. The visualization is
generated entirely on the fly from within the browser, implemented using Vega Lite. So far, Vega
appears to handle rendering this visualization for a cluster of over 700 region servers with
approximately 300,000 regions.
Per [0], include an update to the top-level LICENSE.txt. Also update LICENSE files in all binary
distributions (i.e., jars), by way of LICENSE.vm. Vega uses a BSD 3-clause variant without
advertising clause, and as such is a "Category A" license, per [1].
No changes are made to the NOTICE files, as per the existing example of bundling the minified
JQuery, which is also a Category A license.
[0]: https://infra.apache.org/licensing-howto.html
[1]: https://www.apache.org/legal/resolved.html#category-a
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-15 21:10:23 +01:00
c1bf2ffcd9
HBASE-26833 Avoid waiting to clear buffer usage of ReplicationSourceShipper when aborting the RS ( #4218 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-15 22:12:02 +08:00
46af73b5c0
HBASE-26840 Fix NPE in the retry of logroller ( #4224 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-15 22:08:41 +08:00
a4b192e33d
HBASE-26611 Changing SFT implementation on disabled table is dangerous ( #4082 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-03-15 20:26:24 +08:00
350db514c7
HBASE-26835 Rewrite TestLruAdaptiveBlockCache to make it more stable ( #4219 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 5ad51e8b2b
2022-03-15 20:19:32 +08:00
e6879b8fd7
HBASE-25895 Implement a Cluster Metrics JSON endpoint (addendum)
2022-03-14 17:26:20 +01:00
Duo Zhang
Abhradeep Kundu
Kota-SH
chenglei
binlijin
Bryan Beaudreault
Xiaolin Ha
Michael Stack
Andor Molnár
Istvan Toth
LiangJun He
Ujjawal
Ruanhui
tianhang
Andrew Purtell
Fantasy-Jay
BukrosSzabolcs
Peter Somogyi
Sean Busbey
Wellington Ramos Chevreuil
liangxs
huaxiangsun
SiCheng-Zheng
chaijunjie0101
Nick Dimiduk
Nihal Jain
Ishika
Kerasone
Pankaj
litao
xiaozhang0319
Viraj Jasani
liuzhuang2017
skysiders
Huaxiang Sun
Sergey Soldatov
d-c-manning
Rushabh Shah
Tak Lon (Stephen) Wu
Bri Augenreich
eomiks
bsglz
weimingdiit
huiruan