Apache/hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,875 Commits 95 Branches 866 Tags
Commit Graph

8521 Commits

Author SHA1 Message Date
Bryan Beaudreault
a9d4d6e9e8 Hubspot Backport: HBASE-27224 HFile tool statistic sampling produces misleading results 2022-07-21 13:16:00 -04:00
Bryan Beaudreault
9ca1073fd5 HubSpot Backport: HBASE-27225 Add BucketAllocator bucket size statistic logging 2022-07-21 13:15:39 -04:00
Bryan Beaudreault
e4ce632d59 HubSpot Backport: HBASE-27053 IOException during caching of uncompressed block to the block cache (#4610) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: wenwj0 <wenweijian2@huawei.com>
 2022-07-16 21:33:20 -04:00
Bryan Beaudreault
7023542cce HubSpot Backport: HBASE-27078 Allow configuring a separate timeout for meta scans (#4585) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-12 17:42:51 -04:00
Bryan Beaudreault
9d85d622a1 HubSpot Backport: HBASE-27149 Server should close scanner if client times out before results are ready (#4604) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-11 22:49:33 -04:00
Xiaolin Ha
e21e95974d HubSpot Backport: HBASE-26175 MetricsHBaseServer should record all kinds of Exceptions (#4248) 
Signed-off-by: Pankaj Kumar <pankajkumar@apache.org>
 2022-07-11 18:25:04 -04:00
chenglei
aee754f018 Hubspot Backport: HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallContext is null (#4265) 2022-07-11 18:13:12 -04:00
Istvan Toth
421db7914a HubSpot Backport: HBASE-26777 BufferedDataBlockEncoder$OffheapDecodedExtendedCell.deepC… (#4139) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-11 18:13:07 -04:00
Bryan Beaudreault
cf3fe854b5 HubSpot Backport: HBASE-27186 Report block cache size metrics separately for L1 and L2 2022-07-11 10:16:32 -04:00
Bryan Beaudreault
679e40e068 HBASE-27048 Server side scanner time limit should account for time in queue (#4562) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-07-06 15:22:24 -04:00
Bryan Beaudreault
588ad6b6eb HubSpot Backport: HBASE-27170 ByteBuffAllocator leak when decompressing blocks near minSizeForReservoirUse 2022-07-01 16:13:45 -04:00
Bryan Beaudreault
f4752de8d8 HBASE-26945 Quotas causes too much load on meta for large clusters (#4576) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-06-24 08:09:03 -04:00
Bryan Beaudreault
3c3927c60f HBASE-26790 getAllRegionLocations can cache locations with null hostname (#4575) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-06-24 08:09:03 -04:00
Ruanhui
a9918c504b HBASE-26942 cache region locations when getAllRegionLocations (#4357) 
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2022-06-24 08:09:03 -04:00
Bryan Beaudreault
0a222c8fa8 Revert "HubSpot Backport: HBASE-25709 Close region may stuck when region is compacting and skipped most cells read (#3117)" 
This reverts commit 35ec27d079cdb0fe8882486534dd95c95748884f.
 2022-06-12 15:00:28 -04:00
Bri Augenreich
e827cfbbb6
HubSpot Backport: HBASE-26581 Add metrics for failed replication edits 
Co-authored-by: Briana Augenreich <baugenreich@hubspot.com>
 2022-04-27 08:41:36 -04:00
Bryan Beaudreault
b95fe05bcb HubSpot Backport: HBASE-26807 Unify CallQueueTooBigException special pause with CallDroppedException (#4273) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2022-04-23 21:18:14 -04:00
Bryan Beaudreault
fa033e4ac6 HubSpot Backport: HBASE-26783 ScannerCallable doubly clears meta cache on retries (#4147) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-26 12:59:50 -04:00
Xiaolin Ha
35ec27d079 HubSpot Backport: HBASE-25709 Close region may stuck when region is compacting and skipped most cells read (#3117) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-26 12:59:50 -04:00
Andrew Purtell
954004af49 HubSpot Backport: HBASE-26715 Blocked on SyncFuture in AsyncProtobufLogWriter#write (#4184) 
Co-authored-by: Bryan Beaudreault <bbeaudreault@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani<virajjasani@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-26 12:59:50 -04:00
Bryan Beaudreault
44c62c58ba fix 2022-03-26 12:59:50 -04:00
Xiaolin Ha
e81f6ced7b HubSpot Backport: HBASE-26552 Introduce retry to logroller to avoid abort (#4170) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2022-03-26 12:33:31 -04:00
Michael Stack
30a299eb55 HubSpot Backport: HBASE-21065 Try ROW_INDEX_V1 encoding on meta table 
Set encoding and blooms on meta as default. Also shutdown access to the
initial meta schema creating method; get from TableDescriptors if you
need access to schema or edit it as you would any other table if you
want to edit it.
 2022-03-24 07:18:52 -04:00
Baiqiang Zhao
a0a9da8bcf HubSpot Backport: HBASE-25687 Backport "HBASE-25681 Add a switch for server/table query… (#3074) 
Signed-off-by: stack <stack@apache.org>
 2022-03-17 14:41:44 -04:00
Wei-Chiu Chuang
c01cc7cb6a HubSpot Backport: HBASE-21946 Use ByteBuffer pread instead of byte[] pread in HFileBlock when applicable (#3434) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 4a3c7d73b04928ca0d63b7117186ddb9757997f2)
 2022-03-17 14:16:41 -04:00
Bryan Beaudreault
0c21465878 Revert "Upstream Callers (#15)" 
See HBasePlanning/issues/806 for details

This reverts commit 8cc6bfb5b22b2b16f2618f02b3aeb15d98d050f0.
 2022-03-15 12:11:10 -04:00
Bryan Beaudreault
374ed22789 HubSpot Backport: HBASE-26814 Default StoreHotnessProtector to off, with logs to guide when to turn it on 2022-03-09 17:48:17 -05:00
Bryan Beaudreault
8a27821582 HubSpot Backport: HBASE-26784 Use HIGH_QOS for ResultScanner.close requests (#4163) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2022-03-08 08:52:13 -05:00
Bri Augenreich
4a38b00588
HubSpot Backport: HBASE-26308 Sum of multiplier of cost functions is not populated properly when we have a shortcut for trigger 2022-03-03 10:21:07 -05:00
Bryan Beaudreault
814065d0d9 HubSpot Backport: HBASE-26731 Add metrics for active and expired scanners 2022-03-01 16:38:09 -05:00
Bryan Beaudreault
60e936a463 HubSpot Backport HBASE-26727 Fix CallDroppedException reporting 2022-02-13 19:22:32 -05:00
Bryan Beaudreault
66b2e38300 HubSpot Addendum HBASE-26703: Add executor name to QueueBalancer constructor args 2022-02-04 09:55:43 -05:00
Bryan Beaudreault
cf6efffc77 HubSpot Backport: HBASE-26727 Fix CallDroppedException reporting 2022-02-03 10:51:17 -05:00
Richard Marscher
2450ffcc90 HubSpot Backport: HBASE-26623 Report CallDroppedException in exception metrics (#3980) 
`CallDroppedException` can be thrown with `CallRunner.drop()` by queue implementations that decide to drop calls to groom the RPC call backlog. The LifoCoDel queue does this I believe and with Pluggable queue it's possible for 3rd party queue implementations to be using `drop()` for similar reasons. It would be nice for the server to be tracking these exceptions in metrics since otherwise you might have to do some extra lifting on the client side.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
 2022-01-27 13:15:14 -05:00
Bryan Beaudreault
68bc63e9cb HubSpot Backport: HBASE-26713 Default to LATEST_TIMESTAMP if no timestamp sent along on Increment/Append 2022-01-27 10:33:53 -05:00
Bryan Beaudreault
0be4643029 HubSpot Backport: HBASE-26703 Allow configuration of IPC queue balancer 2022-01-25 15:36:43 -05:00
Richard Marscher
b0007c9975 HubSpot Backport HBASE-26576 Allow pluggable queue to be used with the fast path executor or normal balanced executor 2021-12-15 11:56:04 -05:00
Bryan Beaudreault
102c2bb129 HubSpot Backport HBASE-26304: Reflect out of band locality improvements in metrics and balancer 2021-11-15 14:28:30 -05:00
GeorryHuang
b988fd3bb2 HubSpot Backport: HBASE-26251 StochasticLoadBalancer metrics should update even if balancer doesn't run (#3706) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Bryan Beaudreault <bbeaudreault@hubspot.com>
 2021-10-22 14:56:34 -04:00
Duo Zhang
36adc2e6ac HubSpot Backport: HBASE-25032 Do not assign regions to region server which has not called regionServerReport yet (#3268) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-10-22 14:56:34 -04:00
richardantal
a396547140 HubSpot Backport: HBASE-26289 Hbase scan setMaxResultsPerColumnFamily not giving right results (#3688) 
Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-09-29 07:54:44 -04:00
clarax
49453b388e HubSpot Backport: HBASE-26297 Balancer run is improperly triggered by accuracy error of double comparison (#3698) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-29 07:54:28 -04:00
Bryan Beaudreault
1507f893a7 HubSpot Backport: HBASE-26147 Add a dry run mode to the balancer, where moves are calculated but not actually executed 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>

(Re-application of HBASE-26147 with the correct author metadata)
 2021-09-24 15:51:16 -04:00
Bryan Beaudreault
8cc6bfb5b2 Upstream Callers (#15) 
Adds an upstream caller to the RequestHeader of hbase requests, and makes that available in RpcCallContext
One can inject an upstream caller by setting the conf `hbase.upstream.caller.impl` to a fully qualified class name of a class with a public constructor that implements `org.apache.hadoop.hbase.ipc.UpstreamCaller`
Upstream callers will be available via `RpcCallContext#getUpstreamCaller` or `RpcCallContext#getRequestUserName`. The latter combines the real username and upstream caller to create a fully qualified name like `<upstreamCaller>.via.<user>`
 2021-09-24 15:51:16 -04:00
Bryan Beaudreault
fd5b526398 Add maven and RPM building in blazar 
This uses our standard buildpack, but each module must get a custom .blazar.yaml to pass in required
properties to the build. One can distribute that .blazar.yaml using build-scripts/copy_blazar_yaml.sh.
In order to minimize the duplication, most of the customizations are consolidated in
build-scripts/prepare_environment.sh.

The RPM build is hooked into the standard build hbase-assembly, as a post-build step. The RPM build
scripts in hbase-assembly/rpm-build handle turning the assembled tar.gz into multiple RPMs. We required
some minor changes to the descriptor in hbase-assembly to account for the fact that our build processes
each module independently instead of as a single reactor build. As a result, moduleSet does not work, so
moduleSet usages were converted to dependencySet.
 2021-09-24 15:51:16 -04:00
Andrew Purtell
7374d396c2 Preparing hbase release 2.4.6RC0; tagging and updates to CHANGES.md and RELEASENOTES.md 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-09-03 09:41:52 -07:00
Rushabh Shah
f9a9b6eaee HBASE-26106 AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory. (#3636) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Conflicts:
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java
 2021-09-02 12:08:49 -07:00
XinSun
067f609ec6 HBASE-25773 TestSnapshotScannerHDFSAclController.setupBeforeClass is flaky (#3651) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>

(cherry picked from commit 345d7256c812dd5fbdfe9f378b2884dd945c5da2)
 2021-09-02 15:33:24 +08:00
chenglei
cf4c764d16 HBASE-26210 HBase Write should be doomed to hang when cell size exceeds InmemoryFlushSize for CompactingMemStore (#3604) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-09-01 18:17:26 +08:00
Andrew Purtell
122efbcd82 HBASE-25642 Fix or stop warning about already cached block (#3638) 
Our logs have as a fairly common occurrence: 2021-03-05 22:24:31,034 WARN
[StoreFileOpener-foo-1] hfile.BlockCacheUtil: Caching an already cached
block: blah.bub. This is harmless and can happen in rare cases (see HBASE-8547)

Because it is harmless, log at DEBUG level, not WARN.

Signed-off-by: Baiqiang Zhao <zhaobaiqiang@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
 2021-08-31 09:26:14 -07:00