Commit Graph

18000 Commits

Author SHA1 Message Date
Duo Zhang 706082d513
HBASE-26294 Backport "HBASE-26181 Region server and master could use itself as ConnectionRegistry" to branch-2 (#3708)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-10-05 22:11:01 +08:00
imbajin 9a33e234e7 HBASE-24601: Change default Hfile storage policy from HOT to NONE for HDFS (#1940)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-10-02 16:37:39 +08:00
Yutong Xiao a2c09ae5bc HBASE-26305 Move NavigableSet add operation to writer thread in BucketCache (#3705)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: chenglei <chenglei@apache.org>
2021-10-01 21:06:57 +08:00
chenglei 0748165ab0 HBASE-26295 BucketCache could not free BucketEntry which restored fro… (#3699)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-29 21:46:57 +08:00
GeorryHuang 08800e19d6
HBASE-26251 StochasticLoadBalancer metrics should update even if balancer doesn't run (#3706)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
2021-09-29 21:25:39 +08:00
richardantal 69749ed01e HBASE-26289 Hbase scan setMaxResultsPerColumnFamily not giving right results (#3688)
Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-29 10:22:24 +02:00
Tak Lon (Stephen) Wu d4d22f8da4
HBASE-26299 Fix TestHTableTracing.testTableClose for nightly build of branch-2 (#3703)
For unit testing in sync connection with OpenTelemetry tracing,
there is a race condition between ConnectionImplementation#finalize
(object GC) and the test method before Rule triggered by
OpenTelemetryRule.create. such that a error message of
`GlobalOpenTelemetry.set has already been called`.

This change fixed it test by moving up the TRACE_RULE creation
to class-level such that the GlobalOpenTelemetry is being set
and reuse it for all methods.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2021-09-28 09:26:07 -07:00
Duo Zhang 08a2efde16 HBASE-26293 Use reservoir sampling when selecting bootstrap nodes (#3702)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2021-09-28 23:41:28 +08:00
bitterfox 786e09a936 HBASE-26238 Short message by Result#compareResults for VerifyReplication to avoid OOME (#3647)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-28 23:16:20 +08:00
clarax 73f42b27bf HBASE-26297 Balancer run is improperly triggered by accuracy error of double comparison (#3698)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-26 23:35:50 +08:00
Tak Lon (Stephen) Wu 49ae675275
HBASE-26141 Add tracing support for HTable and sync connection on branch-2 (#3696)
* RPC and IPC tracing supported by HBASE-26125
* WAL tracing supported by HBASE-26131

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-25 16:44:34 -07:00
clarax 404789adde HBASE-26178 Improve data structure and algorithm for BalanceClusterState to improve computation speed for large cluster (#3682)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-25 21:48:51 +08:00
Andrew Purtell 59d133ef95 Amend HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job
Compliation fixes for CombinedBlockCache and TestClientSideRegionScanner.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2021-09-22 12:27:59 -07:00
Tak Lon (Stephen) Wu e0325c40f6 HBASE-26274 Create an option to reintroduce BlockCache to mapreduce job (#3684)
Introduce `hfile.onheap.block.cache.fixed.size`
and default to disable. when using ClientSideRegionScanner
it will be enabled with a fixed size for caching
INDEX/LEAF_INDEX block when a client, e.g.
snapshot scanner, scans the entire HFile
and does not need to seek/reseek to index
block multiple times.

Signed-off-by: Josh Elser <elserj@apache.org>
2021-09-22 09:38:20 -07:00
meiyi 84b5511d13 HBASE-26261 Store configuration loss when use update_config (#3664)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-22 18:35:28 +08:00
meiyi 8f6c5e1f02 HBASE-26270 Provide getConfiguration method for Region and Store interface (#3671)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2021-09-22 11:32:32 +08:00
Duo Zhang e327fa43c0 HBASE-26288 Revisit the usage of MetaTableLocator when HRegionServer.TEST_SKIP_REPORTING_TRANSITION is true (#3689)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-09-22 00:27:11 +08:00
Duo Zhang ef85204522 HBASE-26285 Remove MetaTableLocator usages in non-migration code (#3686)
Signed-off-by: Yulin Niu <niuyulin@apache.org>
2021-09-21 23:32:13 +08:00
chenglei b21b250570 HBASE-26281 DBB got from BucketCache would be freed unexpectedly before RPC completed (#3680)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-17 22:28:56 +08:00
Josh Elser e8d62139d5 HBASE-26273 Force ReadType.STREAM when the user does not explicitly set a ReadType on the Scan for a Snapshot-based Job
HBase 2 moved over Scans to use PREAD by default instead of STREAM like
HBase 1. In the context of a MapReduce job, we can generally expect that
clients using the InputFormat (batch job) would be reading most of the
data for a job. Cater to them, but still give users who want PREAD the
ability to do so.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
2021-09-13 19:10:25 -04:00
bitterfox e139d31612
HBASE-26272 Fix TestTableMapReduceUntil failure in branch-2 (#3676)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-13 10:35:22 +08:00
Wellington Ramos Chevreuil 126f6bc520 HBASE-26276 Allow HashTable/SyncTable to perform rawScan when comparing cells (#3673)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry-picked from commit ee632bdcae)
2021-09-10 16:46:27 +01:00
Bryan Beaudreault 1664f52066 HBASE-26277: Fix InterfaceAudience for BalanceRequest$Builder and BalanceResponse$Builder
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2021-09-10 10:40:06 -04:00
Bryan Beaudreault bb059019ac HBASE-26240 Set InterfaceAudience to Private for BalanceRequest$Builder (#3663)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2021-09-09 12:16:32 -07:00
huaxiangsun 21d9741adb
HBASE-26255 Add an option to use region location from meta table in TableSnapshotInputFormat (#3661) (#3668)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2021-09-09 10:43:36 -07:00
Xiaolin Ha f670fc5d4d HBASE-26229 Limit count and size of L0 files compaction in StripeCompactionPolicy (#3646)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-09 18:28:45 +08:00
hapihu 68db7f5ed0 [HBASE-26243]Fix typo for file "hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java" (#3650)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-09 12:21:54 +08:00
chenglei 62c58fc11a HBASE-26197 Fix some obvious bugs in MultiByteBuff.put (#3586)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-07 23:48:58 +08:00
Rushabh Shah 820ba21b2a HBASE-26106 AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory. (#3636)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
2021-09-02 10:26:22 -07:00
Andrew Purtell 284103b8a3 HBASE-25588 Excessive logging of "hbase.zookeeper.useMulti is deprecated. Default to true always." (#3640)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Duo Zhang <zhangduo@apache.org>
2021-09-02 10:18:42 -07:00
Duo Zhang 49a3e15aa9 HBASE-26157 Expose some IA.LimitedPrivate interface in TestingHBaseCluster (#3643)
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
2021-09-02 17:15:54 +08:00
XinSun ccd90269d2 HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky (#3651)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 345d7256c8)
2021-09-02 10:33:41 +08:00
Bryan Beaudreault e07928df4c HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>

(Re-application of HBASE-26147 with the correct author metadata)
2021-09-01 21:58:30 -04:00
Josh Elser 1e74f08403 Revert "HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed"
This reverts commit 5b58d11c91.

I accidentally applied this as myself instead of as Bryan. Reverting it
to apply the patch properly.
2021-09-01 21:57:46 -04:00
Josh Elser 5b58d11c91 HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org
2021-09-01 21:47:53 -04:00
Tak Lon (Stephen) Wu 625093f907 HBASE-26168 Backport HBASE-25811 "The client integration test is failing after HBASE-22120 merged" into branch-2 (#3634)
move opentelemetry jars to client-facing-thirdparty
add opentelemetry jars when init map reduce job dependencies

original commit 8d2a0efb7a

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu ab431fc8a9 HBASE-26140 Backport HBASE-25778 "The tracinig implementation for AsyncConnectionImpl.getHbck is incorrect" to branch-2 (#3631)
17/17 commits of HBASE-22120, original commit f36e153964

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 65b5b9b2a8 HBASE-26139 Backport HBASE-23762 "Add documentation on how to enable and view tracing with OpenTelemetry" to branch-2 (#3629)
16/17 commits of HBASE-22120, original commit  be4503d9f8

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu a746e19c3b HBASE-26138 Backport HBASE-25733 "Upgrade opentelemetry to 1.0.1" to branch-2 (#3628)
15/17 commits of HBASE-22120, original commit b714889989

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu dd1bdc8aff HBASE-26137 Backport HBASE-25732 "Change the command line argument for tracing after upgrading opentelemtry to 1.0.0" to branch-2 (#3624)
14/17 commits of HBASE-22120, original commit 8df9bebdd3

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu c639d6f353 HBASE-26136 Backport HBASE-25723 "Temporarily remove the trace support for RegionScanner.next" to branch-2 (#3623)
13/17 commits of HBASE-22120, original commit 7f90c2201f

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 8f160109cd HBASE-26135 Backport HBASE-25616 "Upgrade opentelemetry to 1.0.0" to branch-2 (#3620)
12/17 commits of HBASE-22120, original commits 8399293e21

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu a2b8f7a3be HBASE-26134 Backport HBASE-25617 "Revisit the span names" to branch-2 (#3618)
11/17 commits of HBASE-22120, original commit 8d68f8cd1c

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu d0a53e3f29 HBASE-26133 Backport HBASE-25591 "Upgrade opentelemetry to 0.17.1" to branch-2 (#3608)
10/17 commits of HBASE-22120, original commit f6ff519dd0

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 8b7450f5d8 HBASE-26132 Backport HBASE-25535 "Set span kind to CLIENT in AbstractRpcClient" to branch-2 (#3607)
9/17 commits of HBASE-22120, original commit bb8c4967f8

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 9cce94a2d0 HBASE-26131 Backport HBASE-25484 "Add trace support for WAL sync" to branch-2 (#3597)
8/17 commits of HBASE-22120, original commit 2be2c63f0d

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by:  Peter Somogyi <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 321dcd4e77 HBASE-26130 Backport HBASE-25455 "Add trace support for HRegion read/… (#3594)
7/17 commits of HBASE-22120, original commit 03e12bfa4a

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by:  Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 83a27389a0 HBASE-26129 Backport HBASE-25481 "Add host and port attribute when tracing rpc call at client side" to branch-2 (#3589)
6/17 commits of HBASE-22120, original commit ae2c62ffaa

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by:  Peter Somogyi  <psomogyi@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 7ecf54f365 HBASE-26128 Backport HBASE-25454 "Add trace support for connection re… (#3561)
5/17 commits of HBASE-22120, original commit dcb78bd4bd and merged conflicts after rebasing on HBASE-26150 with commit 63d4970de4

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00
Tak Lon (Stephen) Wu 58b5c007b9 HBASE-26127 Backport HBASE-23898 "Add trace support for simple apis i… (#3556)
4/17 commits of HBASE-22120, original commit 805b2ae2ad

Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2021-09-01 15:29:09 -07:00