f0e1bc81f9
HBASE-26690 Modify FSTableDescriptors to not rely on renaming when writing TableDescriptor ( #4054 )
...
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
46c10f78ff
HBASE-26700 The way we bypass broken track file is not enough in StoreFileListFile ( #4055 )
...
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
520b7e7c55
HBASE-26675 Data race on Compactor.writer ( #4035 )
...
Signed-off-by: Xin Sun <ddupgs@gmail.com>
2022-03-26 09:30:58 -07:00
9435769420
HBASE-26674 Should modify filesCompacting under storeWriteLock ( #4040 )
...
Signed-off-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
af921c3546
HBASE-26654 ModifyTableDescriptorProcedure shoud load TableDescriptor while executing ( #4034 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
2022-03-26 09:30:58 -07:00
90981be650
HBASE-26586 Should not rely on the global config when setting SFT implementation for a table while upgrading ( #4006 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
5ffa8315cf
HBASE-26639 The implementation of TestMergesSplitsAddToTracker is problematic ( #4010 )
...
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/storefiletracker/TestStoreFileTracker.java
2022-03-26 09:30:58 -07:00
45d7f3118a
HBASE-26585 Add SFT configuration to META table descriptor when creating META ( #3998 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
(cherry picked from commit baeb51ff8a
2022-03-26 09:30:58 -07:00
4776c88e82
HBASE-26265 Update ref guide to mention the new store file tracker im… ( #3942 )
2022-03-26 09:30:58 -07:00
16ad5f777d
HBASE-26286: Add support for specifying store file tracker when restoring or cloning snapshot
...
Closes #3851
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
6a794e397b
HBASE-26454 CreateTableProcedure still relies on temp dir and renames… ( #3845 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-26 09:30:58 -07:00
d6a409a695
HBASE-26271 Cleanup the broken store files under data directory ( #3786 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
2022-03-26 09:30:58 -07:00
baac21b047
HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables ( #3700 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
b5a3225999
HBASE-26328 Clone snapshot doesn't load reference files into FILE SFT impl ( #3749 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-26 09:30:58 -07:00
08d108d879
HBASE-26386 Refactor StoreFileTracker implementations to expose the set method ( #3774 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
4b493b226b
HBASE-26326 CreateTableProcedure fails when FileBasedStoreFileTracker… ( #3721 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
4ae866f3bc
HBASE-26280 Use store file tracker when snapshoting ( #3685 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Reviewed-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
9ed3f3f161
HBASE-26264 Add more checks to prevent misconfiguration on store file tracker ( #3681 )
...
Signed-off-by: Josh Elser <elserj@apache.org>
2022-03-26 09:30:58 -07:00
864fb9a8ca
HBASE-26248 Should find a suitable way to let users specify the store file tracker implementation ( #3665 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/storefiletracker/TestRegionWithFileBasedStoreFileTracker.java
2022-03-26 09:30:58 -07:00
b80efeae09
HBASE-26246 Persist the StoreFileTracker configurations to TableDescriptor when creating table ( #3666 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-26 09:30:58 -07:00
b7c1adbbfb
HBASE-26224 Introduce a MigrationStoreFileTracker to support migrating from different store file tracker implementations ( #3656 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/storefiletracker/TestMigrationStoreFileTracker.java
2022-03-26 09:30:58 -07:00
d08857df53
HBASE-26079 Use StoreFileTracker when splitting and merging ( #3617 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
2022-03-26 09:30:58 -07:00
60135108f4
HBASE-25988 Store the store file list by a file ( #3578 )
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-03-26 09:30:58 -07:00
073656bf88
HBASE-26064 Introduce a StoreFileTracker to abstract the store file tracking logic
...
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreFlusher.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultStoreEngine.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScannerClosure.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeStoreEngine.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestDateTieredCompactor.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactor.java
2022-03-26 09:30:58 -07:00
64a2e9c088
HBASE-26871 shaded mapreduce and shaded byo-hadoop client artifacts contain no classes ( #4279 )
...
- revert poms for both modules
- add a test to the shaded artifact validation that checks for java classes
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml
hbase-shaded/hbase-shaded-mapreduce/pom.xml
2022-03-26 09:12:05 -07:00
cfd0a1786a
HBASE-26876 Use toStringBinary for rowkey in RegionServerCallable error string ( #4269 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-03-26 06:49:42 +08:00
c41c18bb8d
HBASE-26875 RpcRetryingCallerImpl translateException ignores return value of recursive call ( #4270 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-25 09:01:27 -07:00
1404539731
HBASE-26878 TableInputFormatBase should cache RegionSizeCalculator ( #4271 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-24 16:16:44 -07:00
7906703569
HBASE-26175 MetricsHBaseServer should record all kinds of Exceptions ( #4248 )
...
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
2022-03-24 19:10:07 +08:00
2d1dc5349e
HBASE-21065 Try ROW_INDEX_V1 encoding on meta table ( #4268 )
...
Set encoding and blooms on meta as default. Also shutdown access to the
initial meta schema creating method; get from TableDescriptors if you
need access to schema or edit it as you would any other table if you
want to edit it.
Co-authored-by: Michael Stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-23 16:53:31 -07:00
1bee70578d
HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallContext is null ( #4266 )
2022-03-23 19:32:46 +08:00
fc9aecbaab
HBASE-26531 Trace coprocessor exec endpoints (addendum)
...
Stuck by this again. branch-2 needs this configuration instance to be initialized as an
HBaseConfiguration.
2022-03-22 14:10:39 +01:00
b910b11731
HBASE-26531 Trace coprocessor exec endpoints
...
Trace table ExecService invocations as table operations. Ensure span relationships for both table
and master invocations.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-22 14:10:39 +01:00
ae2000498a
HBASE-26834 Adapt ConnectionRule for both sync and async connections
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-21 16:52:35 +01:00
afd10a6fbd
HBASE-26870 Log4j2 integration is incorrect in nighly's client integration test ( #4250 )
...
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 8640453d70
2022-03-21 11:19:04 +08:00
c2ab765334
Revert "HBASE-26813 Remove javax.ws.rs-api dependency ( #4191 )"
...
MiniYARNCluster needs it
This reverts commit f497465d38
2022-03-19 19:46:08 +08:00
b11b18bf91
HBASE-26855 Delete unnecessary dependency on jaxb-runtime jar ( #4236 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-18 19:15:05 +01:00
31dd5ec054
HBASE-26861 Fix flaky TestSnapshotFromMaster.testSnapshotHFileArchiving ( #4242 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 520ad638d9
2022-03-19 00:46:36 +08:00
6be8c33789
HBASE-26858 Refactor TestMasterRegionOnTwoFileSystems to avoid dead loop ( #4238 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 31db8e3b3b
2022-03-19 00:46:08 +08:00
2e8f0a0326
HBASE-26833 Addendum fix NPE in TestReplicationSource.testTerminateClearsBuffer
...
(cherry picked from commit bd65ad42a2
2022-03-17 23:23:20 +08:00
c413bb1120
HBASE-26848 Addendum make tmpdir absolute
...
(cherry picked from commit caea843e94
2022-03-17 13:03:54 +08:00
f497465d38
HBASE-26813 Remove javax.ws.rs-api dependency ( #4191 )
...
This is no longer needed since we've transitioned to the shaded Jersey shipped in
hbase-thirdparty. Also drop supplemental models entry.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-16 17:25:21 +01:00
abdf727f4e
HBASE-26828 Addendum fix syntax error
...
(cherry picked from commit fec5478226
2022-03-16 21:54:20 +08:00
4f3decd08f
HBASE-26720 ExportSnapshot should validate the source snapshot before copying files ( #4216 )
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-03-16 20:27:39 +08:00
4ff63f6570
Hbase 26275 update error message when executing deleteall with ROWPREFIXFILTER in meta table ( #4205 )
...
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
2022-03-16 20:27:21 +08:00
e008cb836d
HBASE-26680 Close and do not write trailer for the broken WAL writer ( #4174 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-16 20:27:11 +08:00
231dca1d42
HBASE-26848 Set java.io.tmpdir on mvn command when running jenkins job ( #4231 )
...
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit cc13624232
2022-03-16 18:47:50 +08:00
ba2599f73a
HBASE-25896 Implement a Region Visualization on Master WebUI ( #4228 )
...
This is a demonstration of visualization of regions on the cluster. The visualization is a stacked
bar chart showing total storefile size per table per region server, with the x-axis being server
names, the y-axis being storfile size, and the bars stacked per table. The visualization is
generated entirely on the fly from within the browser, implemented using Vega Lite. So far, Vega
appears to handle rendering this visualization for a cluster of over 700 region servers with
approximately 300,000 regions.
Per [0], include an update to the top-level LICENSE.txt. Also update LICENSE files in all binary
distributions (i.e., jars), by way of LICENSE.vm. Vega uses a BSD 3-clause variant without
advertising clause, and as such is a "Category A" license, per [1].
No changes are made to the NOTICE files, as per the existing example of bundling the minified
JQuery, which is also a Category A license.
[0]: https://infra.apache.org/licensing-howto.html
[1]: https://www.apache.org/legal/resolved.html#category-a
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-03-15 21:17:54 +01:00
4d0016587b
HBASE-26833 Avoid waiting to clear buffer usage of ReplicationSourceShipper when aborting the RS ( #4218 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-15 22:15:47 +08:00
8349965e8a
HBASE-26840 Fix NPE in the retry of logroller ( #4224 )
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-03-15 22:15:36 +08:00
Duo Zhang
Wellington Ramos Chevreuil
BukrosSzabolcs
BukrosSzabolcs
GeorryHuang
Andrew Purtell
Sean Busbey
Bryan Beaudreault
Xiaolin Ha
chenglei
Nick Dimiduk
xicm