Apache/hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,821 Commits 95 Branches 866 Tags
Commit Graph

8489 Commits

Author SHA1 Message Date
Bryan Beaudreault
cf6efffc77 HubSpot Backport: HBASE-26727 Fix CallDroppedException reporting 2022-02-03 10:51:17 -05:00
Richard Marscher
2450ffcc90 HubSpot Backport: HBASE-26623 Report CallDroppedException in exception metrics (#3980) 
`CallDroppedException` can be thrown with `CallRunner.drop()` by queue implementations that decide to drop calls to groom the RPC call backlog. The LifoCoDel queue does this I believe and with Pluggable queue it's possible for 3rd party queue implementations to be using `drop()` for similar reasons. It would be nice for the server to be tracking these exceptions in metrics since otherwise you might have to do some extra lifting on the client side.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
 2022-01-27 13:15:14 -05:00
Bryan Beaudreault
68bc63e9cb HubSpot Backport: HBASE-26713 Default to LATEST_TIMESTAMP if no timestamp sent along on Increment/Append 2022-01-27 10:33:53 -05:00
Bryan Beaudreault
0be4643029 HubSpot Backport: HBASE-26703 Allow configuration of IPC queue balancer 2022-01-25 15:36:43 -05:00
Richard Marscher
b0007c9975 HubSpot Backport HBASE-26576 Allow pluggable queue to be used with the fast path executor or normal balanced executor 2021-12-15 11:56:04 -05:00
Bryan Beaudreault
102c2bb129 HubSpot Backport HBASE-26304: Reflect out of band locality improvements in metrics and balancer 2021-11-15 14:28:30 -05:00
GeorryHuang
b988fd3bb2 HubSpot Backport: HBASE-26251 StochasticLoadBalancer metrics should update even if balancer doesn't run (#3706) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
 2021-10-22 14:56:34 -04:00
Duo Zhang
36adc2e6ac HubSpot Backport: HBASE-25032 Do not assign regions to region server which has not called regionServerReport yet (#3268) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-10-22 14:56:34 -04:00
richardantal
a396547140 HubSpot Backport: HBASE-26289 Hbase scan setMaxResultsPerColumnFamily not giving right results (#3688) 
Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-09-29 07:54:44 -04:00
clarax
49453b388e HubSpot Backport: HBASE-26297 Balancer run is improperly triggered by accuracy error of double comparison (#3698) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-29 07:54:28 -04:00
Bryan Beaudreault
1507f893a7 HubSpot Backport: HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>

(Re-application of HBASE-26147 with the correct author metadata)
 2021-09-24 15:51:16 -04:00
Bryan Beaudreault
8cc6bfb5b2 Upstream Callers (#15) 
Adds an upstream caller to the RequestHeader of hbase requests, and makes that available in RpcCallContext
One can inject an upstream caller by setting the conf `hbase.upstream.caller.impl` to a fully qualified class name of a class with a public constructor that implements `org.apache.hadoop.hbase.ipc.UpstreamCaller`
Upstream callers will be available via `RpcCallContext#getUpstreamCaller` or `RpcCallContext#getRequestUserName`. The latter combines the real username and upstream caller to create a fully qualified name like `<upstreamCaller>.via.<user>`
 2021-09-24 15:51:16 -04:00
Bryan Beaudreault
fd5b526398 Add maven and RPM building in blazar 
This uses our standard buildpack, but each module must get a custom .blazar.yaml to pass in required
properties to the build. One can distribute that .blazar.yaml using build-scripts/copy_blazar_yaml.sh.
In order to minimize the duplication, most of the customizations are consolidated in
build-scripts/prepare_environment.sh.

The RPM build is hooked into the standard build hbase-assembly, as a post-build step. The RPM build
scripts in hbase-assembly/rpm-build handle turning the assembled tar.gz into multiple RPMs. We required
some minor changes to the descriptor in hbase-assembly to account for the fact that our build processes
each module independently instead of as a single reactor build. As a result, moduleSet does not work, so
moduleSet usages were converted to dependencySet.
 2021-09-24 15:51:16 -04:00
Andrew Purtell
7374d396c2 Preparing hbase release 2.4.6RC0; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-09-03 09:41:52 -07:00
Rushabh Shah
f9a9b6eaee HBASE-26106 AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory. (#3636) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
 2021-09-02 12:08:49 -07:00
XinSun
067f609ec6 HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky (#3651) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>

(cherry picked from commit 345d7256c812dd5fbdfe9f378b2884dd945c5da2)
 2021-09-02 15:33:24 +08:00
chenglei
cf4c764d16 HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore (#3604) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-01 18:17:26 +08:00
Andrew Purtell
122efbcd82 HBASE-25642 Fix or stop warning about already cached block (#3638) 
Our logs have as a fairly common occurrence: 2021-03-05 22:24:31,034 WARN
[StoreFileOpener-foo-1] hfile.BlockCacheUtil: Caching an already cached
block: blah.bub. This is harmless and can happen in rare cases (see HBASE-8547)

Because it is harmless, log at DEBUG level, not WARN.

Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
 2021-08-31 09:26:14 -07:00
keyserbrian1
133237582f HBASE-26232 SEEK_NEXT_USING_HINT is ignored on reversed Scans (#3639) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-29 21:43:02 +08:00
Andrew Purtell
21e2c1b7c6 HBASE-26189 Reduce log level of CompactionProgress notice to DEBUG (#3579) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-08-27 10:15:19 -07:00
Pankaj
1a5b9b7ff6 HBASE-26227 Forward port HBASE-26223 test code to branch-2.4+ (#3632) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-27 16:47:06 +08:00
lpalasthy
da74b8c8fb HBASE-26219 Negative time is logged while waiting on regionservers (#3622) 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-25 16:40:16 +02:00
Xiaolin Ha
d6b2257655 HBASE-26087 JVM crash when displaying RPC params by MonitoredRPCHandler (#3489) 
Signed-off-by: stack <stack@apache.org>
 2021-08-18 14:41:56 +08:00
eomiks
96015e9669 HBASE-24652 master-status UI make date type fields sortable 2021-08-16 16:02:52 -07:00
stack
2ad7905ddc HBASE-26200 Revert "HBASE-25165 Change 'State time' in UI so sorts (#2508)" 
This reverts commit d07d181ea4a9da316659bb21fd4fffc979b5f77a.
 2021-08-16 13:46:05 -07:00
lzx404243
627e0d61ba HBASE-25680 Non-idempotent test in TestReplicationHFileCleaner (#2984) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-14 11:11:46 +08:00
chenglei
8fced2d7f8 HBASE-26026 HBase Write may be stuck forever when using CompactingMemStore (#3421) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-12 23:22:09 +08:00
Xiaolin Ha
2d89b08eb0 HBASE-26155 JVM crash when scan (#3553) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-12 17:20:18 +08:00
Rushabh Shah
9d34fa7d58
HBASE-26185 Return mutable list in AssignmentManager#getExcludedServersForSystemTable (#3577) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-08-12 13:48:17 +05:30
Viraj Jasani
cd007eb6b2
HBASE-26185 Fix TestMaster#testMoveRegionWhenNotInitialized in the presence of hbase.min.version.move.system.tables (#3573) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-08-10 10:59:14 +05:30
Richard Marscher
66f931c433 HBASE-6908: Add pluggable rpc queue implementation (#3522) 
Can pass in a FQCN to load as the call queue implementation.

Standardized arguments to the constructor are the max queue length, the
PriorityFunction, and the Configuration.

`PluggableBlockingQueue` abstract class provided to help guide the
correct constructor signature

Hard fails if the class fails to load as a `BlockingQueue<CallRunner>`

Signed-off-by: stack <stack@apache.org>
 2021-08-09 12:56:25 -07:00
Duo Zhang
3182507652 HBASE-26179 TestRequestTooBigException spends too much time to finish (#3571) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-08-07 23:08:10 +08:00
Xiaolin Ha
2ea2f800b0 HBASE-26170 handleTooBigRequest in NettyRpcServer didn't skip enough bytes (#3564) 
Signed-off-by: stack <stack@apache.com>
 2021-08-05 08:18:09 -07:00
chenglei
9a469aeaae HBASE-26142 NullPointerException when set 'hbase.hregion.memstore.mslab.indexchunksize.percent' to zero (#3531) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-08-05 21:04:57 +08:00
caroliney14
428cff499d HBASE-25469 Add detailed RIT info in JSON format for consumption as metrics (#3555) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-04 18:22:49 -07:00
Andrew Purtell
91130af844 Revert "Add detailed RIT info in JSON format for consumption as metrics (#3555)" 
Commit is missing JIRA identifier.

This reverts commit 9398745f449de90750797742d3a29521bf23f0c3.
 2021-08-04 18:22:30 -07:00
caroliney14
9398745f44 Add detailed RIT info in JSON format for consumption as metrics (#3555) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-08-04 18:07:25 -07:00
Ruanhui
348c39236c HBASE-26166 Fix a minor bug of table list on master ui (#3554) 
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: Rushabh Shah <shahrs87@gmail.com>
 2021-08-04 20:41:11 +08:00
Bryan Beaudreault
63b17d1674
HBASE-26154: Adds exception metrics for QuotaExceededException and RpcThrottlingException (#3546) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-08-02 15:19:59 +05:30
YutSean
cb1174e9e1 HBASE-26105 Rectify the expired TODO comment in CombinedBC. (#3509) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-30 21:43:38 +08:00
Andrew Purtell
4ec1333df8 Preparing development version 2.4.6-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-27 09:34:19 -07:00
Andrew Purtell
03b8c0cf42 Preparing hbase release 2.4.5RC1; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-27 09:34:14 -07:00
Duo Zhang
eb4728116d HBASE-26120 New replication gets stuck or data loss when multiwal groups more than 10 (#3528) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-07-27 08:15:48 -07:00
Andrew Purtell
845e6f0216 Preparing development version 2.4.6-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-26 17:24:44 -07:00
Andrew Purtell
4810fd1096 Preparing hbase release 2.4.5RC1; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-26 17:24:37 -07:00
Andrew Purtell
67163bb1da Preparing development version 2.4.6-SNAPSHOT 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-23 09:55:58 -07:00
Andrew Purtell
4a2828386f Preparing hbase release 2.4.5RC0; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-07-23 09:55:52 -07:00
Rushabh Shah
9a8fff78e7 HBASE-26093 Replication is stuck due to zero length wal file in oldWALs directory (#3504) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-07-23 09:33:50 -07:00
meiyi
a3940afd25 HBASE-24734 RegionInfo#containsRange should support check meta table (#3496) (#3517) 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2021-07-22 18:05:14 +08:00
gkanade
a72c62b022 HBASE-24984 WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used with multi operation (#3512) 
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
 2021-07-22 08:35:46 +05:30