Commit Graph

18499 Commits

Author SHA1 Message Date
bsglz a8d637402c HBASE-26885 Addendum throw exception instead of return in TRSP to let the procedure retry (#4299)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 7f9d1f2259)
2022-04-04 20:27:17 +08:00
Duo Zhang f74d41486f HBASE-26921 Rewrite the counting cells part in TestMultiVersions (#4316)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 3e09e87fad)
2022-04-03 23:09:33 +08:00
Duo Zhang 787cc87794 HBASE-26920 Fix missing braces warnings in TestProcedureMember (#4315)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 78676bb85e)
2022-04-03 23:09:32 +08:00
Duo Zhang 8650de2c3c HBASE-26919 Rewrite the counting rows part in TestFromClientSide4 (#4314)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 4e12eee531)
2022-04-03 23:09:32 +08:00
Duo Zhang d4bba4fba2 HBASE-26916 Fix missing braces warnings in DefaultVisibilityExpressionResolver (#4313)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit c4ff355915)
2022-04-03 23:09:32 +08:00
Nick Dimiduk 235308d8bf HBASE-26545 Implement tracing of scan
* on `AsyncTable`, both `scan` and `scanAll` methods should result in `SCAN` table operations.
* the span of the `SCAN` table operation should have children representing all the RPC calls
  involved in servicing the scan.
* when a user provides custom implementation of `AdvancedScanResultConsumer`, any spans emitted
  from the callback methods should also be tied to the span that represents the `SCAN` table
  operation. This is easily done because these callbacks are executed on the RPC thread.
* when a user provides a custom implementation of `ScanResultConsumer`, any spans emitted from the
  callback methods should be also be tied to the span that represents the `SCAN` table
  operation. This accomplished by carefully passing the span instance around after it is created.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-04-01 12:47:33 +02:00
chenglei 69ea6f579f
HBASE-26811 Secondary replica may be disabled for read incorrectly forever (#4309) 2022-04-01 11:28:29 +08:00
chenglei 2bf81c6647
HBASE-26812 ShortCircuitingClusterConnection fails to close RegionScanners when making short-circuited calls (#4307) 2022-04-01 10:42:40 +08:00
dependabot[bot] 6d2f39beab HBASE-26912 Bump checkstyle from 8.28 to 8.29 (#4293)
Bumps [checkstyle](https://github.com/checkstyle/checkstyle) from 8.28 to 8.29.

updated-dependencies:
- dependency-name: com.puppycrawl.tools:checkstyle

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-31 14:50:29 -07:00
Wellington Ramos Chevreuil cebf8d7c58 HBASE-26838 Junit jar is not included in the hbase tar ball, causing … (#4223)
Signed-off-by: Josh Elser <elserj@apache.org>
Singed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Reviewed-by: Sean Busbey <busbey@apache.org>
2022-03-31 22:17:57 +01:00
Duo Zhang 75c81c7338 HBASE-26245 Store region server list in master local region (#4136)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit bb1bbddf74)
(cherry picked from commit 27111421d9)
2022-03-31 22:27:07 +08:00
Sean Busbey 08ecae97e3 HBASE-26871 addendum. use the jar command from JAVA_HOME (#4297) 2022-03-30 14:56:21 -07:00
Andrew Purtell b96c84371d HBASE-26903 Bump httpclient from 4.5.3 to 4.5.13 (#4296)
Bump httpclient from 4.5.3 to 4.5.13 to avoid a CVE of medium severity in this
dependency.

Newer httpclient versions enable a URI normalization algorithm by default that
rewrites URIs in a way that breaks some forms of valid REST gateway interactions,
so disable it when building the httpclient instance in Client.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2022-03-30 14:31:28 -07:00
dependabot[bot] af1cafc8ca HBASE-26902 Bump bcprov-jdk15on from 1.60 to 1.67 (#4292)
Bumps [bcprov-jdk15on](https://github.com/bcgit/bc-java) from 1.60 to 1.67.

updated-dependencies:
- dependency-name: org.bouncycastle:bcprov-jdk15on 

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-29 09:03:28 -07:00
bsglz e56388a34c HBASE-26885 The TRSP should not go on when it get a bogus server name… (#4276)
* HBASE-26885 The TRSP should not go on when it get a bogus server name from AM
2022-03-29 10:09:25 +08:00
Istvan Toth 9349db6e85 HBASE-26896 list_quota_snapshots fails with ‘ERROR NameError: uninitialized constant Shell::Commands::ListQuotaSnapshots::TABLE’ (#4284)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2022-03-28 21:03:41 +02:00
d-c-manning a28e7ac52d HBASE-26718 HFileArchiver can remove referenced StoreFiles from the archive (#4274)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-28 11:49:40 -07:00
Xiaolin Ha f0088df551 HBASE-26872 Load rate calculator for cost functions should be more precise (#4253)
Signed-off-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
2022-03-28 23:32:42 +08:00
Duo Zhang 4c6f09dabb HBASE-26617 Addendum add removeUnusedImports 2022-03-28 22:57:42 +08:00
Duo Zhang 665c37a564 HBASE-26617 Use spotless to reduce the pain on fixing checkstyle issues (#4214)
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2022-03-28 21:18:09 +08:00
haxiaolin 245229055e Revert "HBASE-26872 Load rate calculator for cost functions should be more precise (#4253)"
This reverts commit 568b335b44.
2022-03-28 18:19:01 +08:00
Duo Zhang 464ddfae15 HBASE-26832 Avoid repeated releasing of flushed wal entries in AsyncFSWAL#syncCompleted (#4281)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 4f491fd5e4)
2022-03-28 08:00:10 +08:00
Xiaolin Ha 568b335b44 HBASE-26872 Load rate calculator for cost functions should be more precise (#4253)
Signed-off-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
2022-03-27 11:29:25 +08:00
huaxiangsun 8482a285c6 HBASE-26864 SplitTableRegionProcedure calls openParentRegions() at a … (#4261)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-03-26 12:15:24 -07:00
Andrew Purtell 6902cb2568 HBASE-26826 Backport StoreFileTracker (HBASE-26067, HBASE-26584, and others) to branch-2.5
Previous cherry picks:

commit 6aaef89 HBASE-26064 Introduce a StoreFileTracker to abstract the store file tracking logic
commit 43b40e9 HBASE-25988 Store the store file list by a file #3578)
commit 6e05376 HBASE-26079 Use StoreFileTracker when splitting and merging #3617)
commit 090b2fe HBASE-26224 HBASE-26224 Introduce a MigrationStoreFileTracker to support migratin… #3656)
commit 0ee1689 HBASE-26246 Persist the StoreFileTracker configurations to TableDescriptor when creating table #3666)
commit 2052e80 HBASE-26248 Should find a suitable way to let users specify the store… #3665)
commit 5ff0f98 HBASE-26264 Add more checks to prevent misconfiguration on store file… #3681)
commit fc4f6d1 HBASE-26280 HBASE-26280 Use store file tracker when snapshoting #3685)
commit 06db852 HBASE-26326 CreateTableProcedure fails when FileBasedStoreFileTracker… #3721)
commit e4e7cf8 HBASE-26386 Refactor StoreFileTracker implementations to expose the s… #3774)
commit 08d1171 HBASE-26328 Clone snapshot doesn't load reference files into FILE SFT impl #3749)
commit 8bec26e HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configur… #3700)
commit a288365 HBASE-26271: Cleanup the broken store files under data directory #3786)
commit d00b5fa HBASE-26454 CreateTableProcedure still relies on temp dir and renames… #3845)
commit 771e552 HBASE-26286: Add support for specifying store file tracker when restoring or cloning snapshot
commit f16b7b1 HBASE-26265 Update ref guide to mention the new store file tracker im… #3942)
commit 755b3b4 HBASE-26585 Add SFT configuration to META table descriptor when creating META #3998)
commit 39c42c7 HBASE-26639 The implementation of TestMergesSplitsAddToTracker is pro… #4010)
commit 6e1f5b7 HBASE-26586 Should not rely on the global config when setting SFT implementation for a table while upgrading #4006)
commit f1dd865 HBASE-26654 ModifyTableDescriptorProcedure shoud load TableDescriptor… #4034)
commit 8fbc9a2 HBASE-26674 Should modify filesCompacting under storeWriteLock #4040)
commit 5aa0fd2 HBASE-26675 Data race on Compactor.writer #4035)
commit 3021c58 HBASE-26700 The way we bypass broken track file is not enough in Stor… #4055)
commit a8b68c9 HBASE-26690 Modify FSTableDescriptors to not rely on renaming when wr… #4054)
commit dffeb8e HBASE-26587 Introduce a new Admin API to change SFT implementation (#… #4080)
commit b265fe5 HBASE-26673 Implement a shell command for change SFT implementation #4113)
commit 4cdb380 HBASE-26640 Reimplement master local region initialization to better … #4111)
commit 77bb153 HBASE-26707: Reduce number of renames during bulkload (#4066) #4122)
commit a4b192e HBASE-26611 Changing SFT implementation on disabled table is dangerous #4082)
commit d3629bb HBASE-26837 Set SFT config when creating TableDescriptor in TestClone… #4226)
commit 541d748 HBASE-26881 Backport HBASE-25368 to branch-2 (#4267)

Fixups for precommit error prone, checkstyle, and javadoc warnings after applying cherry picks.

Signed-off-by: Josh Elser <elserj@apache.org>
Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
Wellington Ramos Chevreuil 2bbddfee1e HBASE-26881 Backport HBASE-25368 to branch-2 (#4267)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang 6a4b2428df HBASE-26837 Set SFT config when creating TableDescriptor in TestCloneSnapshotProcedure (#4226)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 5b28d76652)
2022-03-26 09:30:58 -07:00
Duo Zhang 9c8d1e4625 HBASE-26611 Changing SFT implementation on disabled table is dangerous (#4082)
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-03-26 09:30:58 -07:00
BukrosSzabolcs 4f9fbd8d58 HBASE-26707: Reduce number of renames during bulkload (#4066) (#4122)
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBulkloadBase.java
2022-03-26 09:30:58 -07:00
Duo Zhang c76c08308c HBASE-26640 Reimplement master local region initialization to better work with SFT (#4111)
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
LiangJun He 4c0786e2b0 HBASE-26673 Implement a shell command for change SFT implementation (#4113)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang 6c43e62163 HBASE-26587 Introduce a new Admin API to change SFT implementation (#4030) (#4080)
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
Reviewed-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang f0e1bc81f9 HBASE-26690 Modify FSTableDescriptors to not rely on renaming when writing TableDescriptor (#4054)
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang 46c10f78ff HBASE-26700 The way we bypass broken track file is not enough in StoreFileListFile (#4055)
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang 520b7e7c55 HBASE-26675 Data race on Compactor.writer (#4035)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2022-03-26 09:30:58 -07:00
Duo Zhang 9435769420 HBASE-26674 Should modify filesCompacting under storeWriteLock (#4040)
Signed-off-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang af921c3546 HBASE-26654 ModifyTableDescriptorProcedure shoud load TableDescriptor while executing (#4034)
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang 90981be650 HBASE-26586 Should not rely on the global config when setting SFT implementation for a table while upgrading (#4006)
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang 5ffa8315cf HBASE-26639 The implementation of TestMergesSplitsAddToTracker is problematic (#4010)
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/storefiletracker/TestStoreFileTracker.java
2022-03-26 09:30:58 -07:00
Wellington Ramos Chevreuil 45d7f3118a HBASE-26585 Add SFT configuration to META table descriptor when creating META (#3998)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>

(cherry picked from commit baeb51ff8a)
2022-03-26 09:30:58 -07:00
Wellington Ramos Chevreuil 4776c88e82 HBASE-26265 Update ref guide to mention the new store file tracker im… (#3942) 2022-03-26 09:30:58 -07:00
BukrosSzabolcs 16ad5f777d HBASE-26286: Add support for specifying store file tracker when restoring or cloning snapshot
Closes #3851

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
Wellington Ramos Chevreuil 6a794e397b HBASE-26454 CreateTableProcedure still relies on temp dir and renames… (#3845)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-26 09:30:58 -07:00
BukrosSzabolcs d6a409a695 HBASE-26271 Cleanup the broken store files under data directory (#3786)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
2022-03-26 09:30:58 -07:00
GeorryHuang baac21b047 HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables (#3700)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
Wellington Ramos Chevreuil b5a3225999 HBASE-26328 Clone snapshot doesn't load reference files into FILE SFT impl (#3749)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang 08d108d879 HBASE-26386 Refactor StoreFileTracker implementations to expose the set method (#3774)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
Wellington Ramos Chevreuil 4b493b226b HBASE-26326 CreateTableProcedure fails when FileBasedStoreFileTracker… (#3721)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang 4ae866f3bc HBASE-26280 Use store file tracker when snapshoting (#3685)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Reviewed-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
Duo Zhang 9ed3f3f161 HBASE-26264 Add more checks to prevent misconfiguration on store file tracker (#3681)
Signed-off-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00