Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,350 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

18350 Commits

Author SHA1 Message Date
Duo Zhang 9ed3f3f161 HBASE-26264 Add more checks to prevent misconfiguration on store file tracker (#3681) 
Signed-off-by: Josh Elser <elserj@apache.org>
 2022-03-26 09:30:58 -07:00
Duo Zhang 864fb9a8ca HBASE-26248 Should find a suitable way to let users specify the store file tracker implementation (#3665) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/storefiletracker/TestRegionWithFileBasedStoreFileTracker.java
 2022-03-26 09:30:58 -07:00
Wellington Ramos Chevreuil b80efeae09 HBASE-26246 Persist the StoreFileTracker configurations to TableDescriptor when creating table (#3666) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-26 09:30:58 -07:00
Duo Zhang b7c1adbbfb HBASE-26224 Introduce a MigrationStoreFileTracker to support migrating from different store file tracker implementations (#3656) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/storefiletracker/TestMigrationStoreFileTracker.java
 2022-03-26 09:30:58 -07:00
Andrew Purtell d08857df53 HBASE-26079 Use StoreFileTracker when splitting and merging (#3617) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java
 2022-03-26 09:30:58 -07:00
Duo Zhang 60135108f4 HBASE-25988 Store the store file list by a file (#3578) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2022-03-26 09:30:58 -07:00
Duo Zhang 073656bf88 HBASE-26064 Introduce a StoreFileTracker to abstract the store file tracking logic 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreFlusher.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultStoreEngine.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScannerClosure.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeStoreEngine.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestDateTieredCompactor.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactor.java
 2022-03-26 09:30:58 -07:00
Sean Busbey 64a2e9c088 HBASE-26871 shaded mapreduce and shaded byo-hadoop client artifacts contain no classes (#4279) 
- revert poms for both modules
- add a test to the shaded artifact validation that checks for java classes

Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml
	hbase-shaded/hbase-shaded-mapreduce/pom.xml
 2022-03-26 09:12:05 -07:00
Bryan Beaudreault cfd0a1786a HBASE-26876 Use toStringBinary for rowkey in RegionServerCallable error string (#4269) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-26 06:49:42 +08:00
Bryan Beaudreault c41c18bb8d HBASE-26875 RpcRetryingCallerImpl translateException ignores return value of recursive call (#4270) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-25 09:01:27 -07:00
Bryan Beaudreault 1404539731 HBASE-26878 TableInputFormatBase should cache RegionSizeCalculator (#4271) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-24 16:16:44 -07:00
Xiaolin Ha 7906703569 HBASE-26175 MetricsHBaseServer should record all kinds of Exceptions (#4248) 
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2022-03-24 19:10:07 +08:00
Bryan Beaudreault 2d1dc5349e HBASE-21065 Try ROW_INDEX_V1 encoding on meta table (#4268) 
Set encoding and blooms on meta as default. Also shutdown access to the
initial meta schema creating method; get from TableDescriptors if you
need access to schema or edit it as you would any other table if you
want to edit it.

Co-authored-by: Michael Stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-23 16:53:31 -07:00
chenglei 1bee70578d
HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallContext is null (#4266) 2022-03-23 19:32:46 +08:00
Nick Dimiduk fc9aecbaab HBASE-26531 Trace coprocessor exec endpoints (addendum) 
Stuck by this again. branch-2 needs this configuration instance to be initialized as an
HBaseConfiguration.
 2022-03-22 14:10:39 +01:00
Nick Dimiduk b910b11731 HBASE-26531 Trace coprocessor exec endpoints 
Trace table ExecService invocations as table operations. Ensure span relationships for both table
and master invocations.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-22 14:10:39 +01:00
Nick Dimiduk ae2000498a HBASE-26834 Adapt ConnectionRule for both sync and async connections 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-21 16:52:35 +01:00
Duo Zhang afd10a6fbd HBASE-26870 Log4j2 integration is incorrect in nighly's client integration test (#4250) 
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 8640453d70)
 2022-03-21 11:19:04 +08:00
Duo Zhang c2ab765334 Revert "HBASE-26813 Remove javax.ws.rs-api dependency (#4191)" 
MiniYARNCluster needs it

This reverts commit f497465d38.
 2022-03-19 19:46:08 +08:00
Nick Dimiduk b11b18bf91 HBASE-26855 Delete unnecessary dependency on jaxb-runtime jar (#4236) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-18 19:15:05 +01:00
Duo Zhang 31dd5ec054 HBASE-26861 Fix flaky TestSnapshotFromMaster.testSnapshotHFileArchiving (#4242) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 520ad638d9)
 2022-03-19 00:46:36 +08:00
Duo Zhang 6be8c33789 HBASE-26858 Refactor TestMasterRegionOnTwoFileSystems to avoid dead loop (#4238) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 31db8e3b3b)
 2022-03-19 00:46:08 +08:00
Duo Zhang 2e8f0a0326 HBASE-26833 Addendum fix NPE in TestReplicationSource.testTerminateClearsBuffer 
(cherry picked from commit bd65ad42a2)
 2022-03-17 23:23:20 +08:00
Duo Zhang c413bb1120 HBASE-26848 Addendum make tmpdir absolute 
(cherry picked from commit caea843e94)
 2022-03-17 13:03:54 +08:00
Nick Dimiduk f497465d38 HBASE-26813 Remove javax.ws.rs-api dependency (#4191) 
This is no longer needed since we've transitioned to the shaded Jersey shipped in
hbase-thirdparty. Also drop supplemental models entry.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-16 17:25:21 +01:00
Duo Zhang abdf727f4e HBASE-26828 Addendum fix syntax error 
(cherry picked from commit fec5478226)
 2022-03-16 21:54:20 +08:00
xicm 4f3decd08f HBASE-26720 ExportSnapshot should validate the source snapshot before copying files (#4216) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-16 20:27:39 +08:00
xicm 4ff63f6570 Hbase 26275 update error message when executing deleteall with ROWPREFIXFILTER in meta table (#4205) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-16 20:27:21 +08:00
Xiaolin Ha e008cb836d HBASE-26680 Close and do not write trailer for the broken WAL writer (#4174) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-16 20:27:11 +08:00
Duo Zhang 231dca1d42 HBASE-26848 Set java.io.tmpdir on mvn command when running jenkins job (#4231) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit cc13624232)
 2022-03-16 18:47:50 +08:00
Nick Dimiduk ba2599f73a HBASE-25896 Implement a Region Visualization on Master WebUI (#4228) 
This is a demonstration of visualization of regions on the cluster. The visualization is a stacked
bar chart showing total storefile size per table per region server, with the x-axis being server
names, the y-axis being storfile size, and the bars stacked per table. The visualization is
generated entirely on the fly from within the browser, implemented using Vega Lite. So far, Vega
appears to handle rendering this visualization for a cluster of over 700 region servers with
approximately 300,000 regions.

Per [0], include an update to the top-level LICENSE.txt. Also update LICENSE files in all binary
distributions (i.e., jars), by way of LICENSE.vm. Vega uses a BSD 3-clause variant without
advertising clause, and as such is a "Category A" license, per [1].

No changes are made to the NOTICE files, as per the existing example of bundling the minified
JQuery, which is also a Category A license.

[0]: https://infra.apache.org/licensing-howto.html
[1]: https://www.apache.org/legal/resolved.html#category-a

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-15 21:17:54 +01:00
Xiaolin Ha 4d0016587b HBASE-26833 Avoid waiting to clear buffer usage of ReplicationSourceShipper when aborting the RS (#4218) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-15 22:15:47 +08:00
Xiaolin Ha 8349965e8a HBASE-26840 Fix NPE in the retry of logroller (#4224) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-15 22:15:36 +08:00
Duo Zhang 102e51a09c HBASE-26835 Rewrite TestLruAdaptiveBlockCache to make it more stable (#4219) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit 5ad51e8b2b)
 2022-03-15 20:19:39 +08:00
Nick Dimiduk dbb5cfac67 HBASE-25895 Implement a Cluster Metrics JSON endpoint (addendum) 2022-03-14 17:28:23 +01:00
Nick Dimiduk 7fb6a153a1 HBASE-25895 Implement a Cluster Metrics JSON endpoint 
Publishes a set of JSON endpoints following a RESTful structure, which expose a subset of the
`o.a.h.h.ClusterMetrics` object tree. The URI structure is as follows

    /api/v1/admin/cluster_metrics
    /api/v1/admin/cluster_metrics/live_servers
    /api/v1/admin/cluster_metrics/dead_servers

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-14 17:28:23 +01:00
meiyi d851133b6f HBASE-26670 HFileLinkCleaner should be added even if snapshot is disabled (#4032) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-14 12:25:44 +08:00
Duo Zhang c0a281fcd3 HBASE-26828 Increase the concurrency when running UTs in pre commit job (#4213) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
(cherry picked from commit a49d147d49)
 2022-03-14 12:21:11 +08:00
Duo Zhang a1994ee152 HBASE-26830 Rewrite TestLruBlockCache to make it more stable (#4212) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
(cherry picked from commit c3d0cc7040)
 2022-03-14 11:24:45 +08:00
Duo Zhang 6cef9e1602 HBASE-25465 Use javac --release option for supporting cross version compilation (#4164) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-12 18:30:40 +08:00
Duo Zhang 5844b53dea HBASE-26802 Backport the log4j2 changes to branch-2 (#4166) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-hadoop-compat/pom.xml
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
	hbase-shaded/hbase-shaded-client-byo-hadoop/pom.xml
	hbase-shaded/hbase-shaded-client/pom.xml
	hbase-shaded/hbase-shaded-mapreduce/pom.xml
	hbase-shaded/hbase-shaded-testing-util/pom.xml
	hbase-shaded/pom.xml
	hbase-testing-util/pom.xml
 2022-03-11 11:38:37 -08:00
Xiaolin Ha bf9233f511 HBASE-25709 Close region may stuck when region is compacting and skipped most cells read (#3117) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-11 09:28:59 -08:00
Xiaolin Ha 70ec3beef4 HBASE-26242 Allow split when store file count larger than the configed blocking file count (#3652) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-11 09:28:59 -08:00
Sergey Soldatov 21abf376e6 HBASE-26767 Rest server should not use a large Header Cache. 
Closes #4123

Co-authored-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2022-03-11 09:28:59 -08:00
Tak Lon (Stephen) Wu b08321c08c HBASE-26729 Backport "HBASE-26714 Introduce path configuration for system coprocessors" to branch-2 (#4093) 
- rename from HBaseCommonTestingUtil to HBaseCommonTestingUtility

Signed-off-by: Ankit Singhal <ankit@apache.org>
 2022-03-11 09:28:59 -08:00
Yutong Xiao 44cae8f31a HBASE-26688 Threads shared EMPTY_RESULT may lead to unexpected client job down. (#4048) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-11 09:28:59 -08:00
Nick Dimiduk eebc2e0bbb HBASE-26819 Minor code cleanup in and around RpcScheduler 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-11 11:16:05 +01:00
Nick Dimiduk 775c020625 HBASE-26817 Mark RpcExecutor as IA.LimitedPrivate COPROC and PHOENIX 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2022-03-11 11:08:54 +01:00
Andrew Purtell 057bc95268 HBASE-26761 TestMobStoreScanner (testGetMassive) can OOME (#4204) 
Change the timing of the test but not the objective by storing three large
MOB values into the row with three separate puts, each randomizing the
data. Increases running time of the testGetMassive case 2x but avoids OOME
in the test environment where the OOME was consistently reproductable.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2022-03-11 15:13:06 +08:00
Andrew Purtell cc256bd78f HBASE-26824 TestHBaseTestingUtil.testResolvePortConflict failing after HBASE-26582 (#4203) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-11 15:07:01 +08:00