Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,638 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

8811 Commits

Author SHA1 Message Date
Duo Zhang f6ff519dd0 HBASE-25591 Upgrade opentelemetry to 0.17.1 (#2971) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang bb8c4967f8 HBASE-25535 Set span kind to CLIENT in AbstractRpcClient (#2907) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 2be2c63f0d HBASE-25484 Add trace support for WAL sync (#2892) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 03e12bfa4a HBASE-25455 Add trace support for HRegion read/write operation (#2861) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang ae2c62ffaa HBASE-25481 Add host and port attribute when tracing rpc call at client side (#2857) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 805b2ae2ad HBASE-23898 Add trace support for simple apis in async client (#2813) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 2420286715 HBASE-25401 Add trace support for async call in rpc client (#2790) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-04-25 09:23:23 +08:00
Duo Zhang 302d9ea8b8 HBASE-25373 Remove HTrace completely in code base and try to make use of OpenTelemetry 
Signed-off-by: stack <stack@apache.org>
 2021-04-25 09:23:23 +08:00
Andrew Purtell 9895b2dfdf
HBASE-25756 Support alternate compression for major and minor compactions (#3142) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-04-23 15:45:26 -07:00
Duo Zhang 96fefce9c3
HBASE-25802 Miscellaneous style improvements for load balancer related classes (#3192) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-23 15:20:27 +08:00
haxiaolin 996862c1cc
HBASE-25754 StripeCompactionPolicy should support compacting cold regions (#3152) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-04-23 14:58:53 +08:00
Toshihiro Suzuki 5f4e2e111b
HBASE-25766 Introduce RegionSplitRestriction that restricts the pattern of the split point (#3150) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-04-22 13:53:36 +09:00
Duo Zhang 50920ee306
HBASE-25774 TestSyncReplicationStandbyKillRS#testStandbyKillRegionServer is flaky (#3189) 
Wait for the restarter thread to finish before checking the state
Add more detailed logs

Signed-off-by: meiyi <myimeiyi@gmail.com>
 2021-04-22 10:10:15 +08:00
Duo Zhang d5c5e48839
HBASE-25793 Move BaseLoadBalancer.Cluster to a separated file (#3185) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-22 09:59:49 +08:00
haxiaolin 0d257baf29
HBASE-25763 TestRSGroupsWithACL.setupBeforeClass is flaky (#3158) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-21 14:41:51 +08:00
Duo Zhang 781da1899a
HBASE-25290 Remove table on master related code in balancer implementation (#3162) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-20 21:31:09 +08:00
Nick Dimiduk b65890da1d Revert "HBASE-25739 TableSkewCostFunction need to use aggregated deviation (#3067)" 
This reverts commit 533c84d330.
 2021-04-16 09:35:02 -07:00
Duo Zhang bf78246b4f
HBASE-25775 Use a special balancer to deal with maintenance mode (#3161) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-04-16 09:50:24 +08:00
clarax 533c84d330
HBASE-25739 TableSkewCostFunction need to use aggregated deviation (#3067) 
Signed-off-by: Michael Stack <stack@apache.org>
Reviewed-by: David Manning <david.manning@salesforce.com>
 2021-04-15 13:12:07 -07:00
xiaoyu 6cf4fdde61
HBASE-25776 Use Class.asSubclass to fix the warning in StochasticLoadBalancer.loadCustomCostFunctions (#3163) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-04-15 23:34:06 +05:30
Nick Dimiduk bc52bca741
HBASE-25770 Http InfoServers should honor gzip encoding when requested (#3159) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2021-04-15 09:07:13 -07:00
Duo Zhang 5910e9e2d1
HBASE-25767 CandidateGenerator.getRandomIterationOrder is too slow on large cluster (#3149) 
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-13 23:00:54 +08:00
Duo Zhang de012d7d1f
HBASE-25759 The master services field in LocalityBasedCostFunction is never used (#3144) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-12 22:27:01 +08:00
Duo Zhang f9e928e5a7
HBASE-25184 Move RegionLocationFinder to hbase-balancer (#2543) 
Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2021-04-10 21:10:53 +08:00
Nick Dimiduk 5f1f8be667 HBASE-25744 Change default of `hbase.normalizer.merge.min_region_size.mb` to `0` 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-04-09 15:00:38 -07:00
Geoffrey Jacoby 74e533d5ab
HBASE-25751 - Add writable TimeToPurgeDeletes to ScanOptions (#3137) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-04-09 13:05:47 -07:00
meiyi ad06aa2082
HBASE-25747 Remove unused getWriteAvailable method in OperationQuota (#3133) 
Signed-off-by: stack <stack@apache.org>
 2021-04-09 10:23:34 +08:00
Pankaj 6444e94c38
HBASE-25717 RegionServer aborted with due to ClassCastException (#3108) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-04-08 16:27:17 +05:30
stack d9f4f41f76 HBASE-25735 Add target Region to connection exceptions 
Addendum to fix broke compile.
 2021-04-07 07:56:25 -07:00
Jan Hentschel 048ca4e43f
HBASE-25174 Remove deprecated fields in HConstants (#2558) 
Remove the deprecated fields, which can be removed in 3.0.0. Marked the
constant OLDEST_TIMESTAMP as InterfaceAudience.Private as it is only use
in classes, which are also marked as InterfaceAudience.Private.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-04-03 23:12:16 +08:00
Jan Hentschel 5a63fe65aa
HBASE-25199 Remove deprecated HStore#getStoreHomedir methods (#2562) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-04-03 23:10:20 +08:00
d-c-manning 7a31557c51
HBASE-25726 MoveCostFunction is not included in the list of cost functions for StochasticLoadBalancer (#3116) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-04-02 09:22:13 -07:00
lujiefsi e14ec57eee
HBASE-25558:Adding audit log for execMasterService (#3101) 
Signed-off-by: stack <stack@apache.org>
 2021-03-31 16:12:31 -07:00
Toshihiro Suzuki 46f7d9dd4b
HBASE-25703 Support conditional update in MultiRowMutationEndpoint (#3098) 
Signed-off-by: Michael Stack <stack@apache.org>
 2021-03-30 09:18:56 +09:00
Josh Elser 57a49f5ca7
HBASE-25692 Always try to close the WAL reader when we catch any exception (#3090) 
There are code paths in which we throw non-IOExceptions when
initializing a WAL reader. However, we only close the InputStream to the
WAL filesystem when the exception is an IOException. Close it if it is
open in all cases.

Co-authored-by: Josh Elser <jelser@cloudera.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-03-29 12:15:58 -07:00
Michael Stack 6a8998b29c HBASE-25695 Link to the filter on hbase:meta from user tables panel on master page (#3092) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-03-27 20:36:55 -07:00
Toshihiro Suzuki 93b1163a8b
HBASE-25702 Remove RowProcessor (#3097) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-03-28 07:38:42 +09:00
caroliney14 1e4639d2eb
HBASE-25032 Wait for region server to become online before adding it to online servers in Master (#2769) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2021-03-25 10:39:07 -07:00
Andrew Purtell f6bb4bb93e
HBASE-25693 NPE getting metrics from standby masters (MetricsMasterWrapperImpl.getMergePlanCount) (#3091) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-03-24 19:09:10 -07:00
huaxiangsun 1e3fe3ceac
HBASE-25691 Test failure: TestVerifyBucketCacheFile.testRetrieveFromFile (#3081) 
The issue is that FileInputStream is created with try-with-resources, so its close() is called right after the try sentence.
FileInputStream is a finalize class, when this object is garbage collected, its close() is called again.
To avoid this double-free resources, add guard against it.

Signed-off-by: stack <stack@apache.org>
 2021-03-24 09:01:17 -07:00
XinSun 3358091b7e
HBASE-25683 Simplify UTs using DummyServer (#3069) 
Co-authored-by: sunxin <sunxin@apache.com>
Signed-off-by: stack <stack@apache.org>
 2021-03-22 08:54:09 -07:00
Duo Zhang ba3610d097
HBASE-19577 Use log4j2 instead of log4j for logging (#1708) 
Signed-off-by: stack <stack@apache.org>
 2021-03-20 09:21:25 +08:00
Baiqiang Zhao a3938c8725
HBASE-25681 Add a switch for server/table queryMeter (#3070) 
Signed-off-by: stack <stack@apache.org>
 2021-03-19 16:23:41 -07:00
shahrs87 fea4bd12e2 HBASE-25679 Size of log queue metric is incorrect (#3071) 
Co-authored-by: Rushabh <rushabh.shah@salesforce.com>
Signed-off-by: stack <stack@apache.org>
 2021-03-19 16:16:48 -07:00
Toshihiro Suzuki f4059907e2
HBASE-25678 Support nonce operations for Increment/Append in RowMutations and CheckAndMutate (#3064) 
Signed-off-by: stack <stack@apache.org>
 2021-03-19 21:25:40 +09:00
haxiaolin 585aca1f05
HBASE-25518 Support separate child regions to different region servers (#3001) 
Signed-off-by: stack <stack@apache.org>
 2021-03-18 12:38:17 -07:00
bsglz d93035a131
HBASE-25643 The delayed FlushRegionEntry should be removed when we ne… (#3049) 
Signed-off-by: AnoopSamJohn <anoopsamjohn@apache.org>
Signed-off-by: stack <stack@apache.org>
 2021-03-18 12:13:06 -07:00
Michael Stack 7ac1c8bbf8 HBASE-25677 Server+table counters on each scan #nextRaw invocation becomes a bottleneck when heavy load (#3061) 
Don't have every handler update regionserver metrics on each
scan#nextRaw; instead, do a batch update just before Scan
returns. Otherwise, all running handlers end up contending
on metrics update.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 Update of regionserver metrics counters moved out to caller where
 can be done as a batch update instead of per-next.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java
 Class doc to encourage batch updating metrics.
 Remove the single update as unused anymore.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
 Count calls to nextRaw. Update regionserver count in finally block when
 scan is done rather than per nextRaw call. Move all metrics updates to
 finally.

Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Baiqiang Zhao <ZhaoBQ>
 2021-03-18 11:46:37 -07:00
Sandeep Pal ff3821814a
HBASE-25627: HBase replication should have a metric to represent if the source is stuck getting initialized (#3018) 
Introduces a new metric that tracks number of replication sources that are stuck in initialization.

Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-03-17 09:10:44 -07:00
DivyeshChandra bcf503e6c2
HBASE-25653 Add units and round off region size to 2 digits after decimal (#3046) 
Signed-off-by: stack <stack@duboce.net>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
 2021-03-16 21:32:12 -07:00
bitterfox ebb0adf500
HBASE-25665 Option to use hostname instead of canonical hostname for secure HBase cluster connection (#3051) 2021-03-16 21:04:25 -07:00
Baiqiang Zhao db2e6d8c63
HBASE-25597 Add row info in Exception when cell size exceeds maxCellSize (#2976) 
Signed-off-by: stack <stack@apache.org>
 2021-03-15 15:49:33 -07:00
haxiaolin 0ef892b68a
HBASE-25621 Balancer should check region plan source to avoid misplace region groups (#3002) 
Signed-off-by: stack <stack@duboce.net>
 2021-03-15 14:47:27 -07:00
haxiaolin 625bea3ecc
HBASE-25595 TestLruBlockCache.testBackgroundEvictionThread is flaky (#2974) 
Signed-off-by: stack <stack@apache.org>
 2021-03-15 14:25:38 -07:00
Michael Stack 630f47e4ec HBASE-25660 Print split policy in use on Region open (as well as split policy vitals) (#3044) 
Add a toString to all split policy implementations listing name and
 vitals. Use this toString in the Region open message. Ditto for flush
 policy for the Region.

 Signed-off-by: Huaxiang Sun<huaxiangsun@apache.org>
 2021-03-15 14:12:31 -07:00
haxiaolin aeec8ca64b
HBASE-25635 CandidateGenerator may miss some region balance actions (#3024) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-03-15 21:28:22 +08:00
Duo Zhang 876fec1648
HBASE-25657 Fix spotbugs warnings after upgrading spotbugs to 4.x (#3041) 
Signed-off-by: meiyi <myimeiyi@gmail.com>
Signed-off-by: stack <stack@apache.org>
 2021-03-12 14:34:10 +08:00
shahrs87 7386fb6e1f
HBASE-25622 Result#compareResults should compare tags. (#3026) 
Signed-off-by: stack <stack@apache.org>
 2021-03-11 21:51:07 -08:00
Michael Stack 1a69a52653
HBASE-25570 On largish cluster, "CleanerChore: Could not delete dir..." makes master log unreadable (#2949) 
Turn down the amount we log. If you want to see the full exception
enable TRACE-level logging.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: shahrs87
 2021-03-11 21:35:24 -08:00
Peter Somogyi 0cc1ae48ed
HBASE-25587 [hbck2] Schedule SCP for all unknown servers (#2978) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-03-11 17:20:36 +01:00
meiyi 0e6c2c419c
HBASE-25636 Expose HBCK report as metrics (#3031) 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2021-03-11 15:12:30 +08:00
Duo Zhang d79019ba3b
HBASE-25629 Reimplement TestCurrentHourProvider to not depend on unstable TZs (#3013) 
Signed-off-by: XinSun <ddupgs@gmail.com>
 2021-03-10 23:45:38 +08:00
Wellington Ramos Chevreuil 373dc7788d
HBASE-25548 Optionally allow snapshots to preserve cluster's max file… (#2923) 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-03-10 10:20:10 +00:00
Anoop Sam John d818eff93a
HBASE-25582 Support setting scan ReadType to be STREAM at cluster level (#3035) 
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-03-10 10:18:12 +05:30
Narges Shadab 573daed419
HBASE-25646: Possible Resource Leak in CatalogJanitor #3036 
Close the stream using a try-with-resources block.

Reviewed-by: Aman Poonia <aman.poonia.29@gmail.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-03-09 09:07:02 -08:00
niuyulin 109bd24065
HBASE-25630 Set switch compaction after bulkload default as false (#3022) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-03-09 12:14:53 +08:00
Bharath Vissapragada c1dacfd577
HBASE-25547 (addendum): Roll ExecutorType into ExecutorConfig (#2996) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-03-08 15:00:18 -08:00
Duo Zhang 9b0485fb98
HBASE-23578 [UI] Master UI shows long stack traces when table is broken (#3014) 
Co-authored-by: Shuhei Yamasaki <yamasakisua@oss.nttdata.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-03-07 00:14:21 +08:00
niuyulin dd4a11e421
HBASE-25637 Rename method completeCompaction to refreshStoreSizeAndTotalBytes (#3023) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-03-06 18:41:07 +08:00
Rahul Kumar 830d2895b2
HBASE-25460 : Expose drainingServers as cluster metric (#2995) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-03-04 12:47:57 +05:30
Andrew Purtell f93c9c6f15
HBASE-25385 TestCurrentHourProvider fails if the latest timezone changes are not present (#3012) 
Disable this problematic test for now. May be removed after additional
discussion.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-03-03 18:12:15 -08:00
niuyulin e80b901e47
HBASE-25603 Add switch for compaction after bulkload (#2982) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-03-04 10:09:08 +08:00
Akshay Sudheer 5d9a6ed1fe
HBASE-25367 Sort broken after Change 'State time' in UI (#2964) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-03-03 13:57:42 +05:30
Narges Shadab e099ef349b
HBASE-25626 Possible Resource Leak in HeterogeneousRegionCountCostFunction 
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by shahrs87 <shahrs87@gmail.com>
 2021-03-03 11:45:56 +05:30
Akshay Sudheer 157200ef83
HBASE-25402 Sorting order by start key or end key is not considering empty start key/end key (#2955) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-03-03 09:38:55 +05:30
Balazs Meszaros a984358d9f
HBASE-25586 Fix HBASE-22492 on branch-2 (SASL GapToken) (#2961) 
ServerCall.java: calling wrapWithSasl() was moved to getResponse(), so
the SASL wrapping is delayed until the reply is sent back to the client.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2021-02-26 14:05:59 +01:00
XinSun 8d0de96976
HBASE-25590 Bulkload replication HFileRefs cannot be cleared in some cases where set exclude-namespace/exclude-table-cfs (#2969) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-02-26 09:50:23 +08:00
Sandeep Pal 3f1c486ddb
HBASE-25596: Fix NPE and avoid permanent unreplicated data due to EOF (#2987) 
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-02-25 13:36:11 -08:00
XinSun 51a3d45f9d
HBASE-25598 TestFromClientSide5.testScanMetrics is flaky (#2977) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-02-24 14:15:51 +08:00
Andrew Purtell 8f03c4429f
HBASE-25556 Frequent replication "Encountered a malformed edit" warnings (#2965) 
ProtobufLogReader#readNext may be called by code that attempts to advance
the reader but does not necessarily expect to succeed, for example
WALEntryStream#tryAdvanceEntry. Much of the logging in this method is
at TRACE level. Other logging at WARN level will be frequently emitted, as
often as several per minute, and this will cause false positive assessment
from operators that they are experiencing a bug. Fix the mixed intent with
respect to log levels in readNext. Log at only DEBUG level or below.

Signed-off-by: Sean Busbey <busbey@apache.org>
 2021-02-23 14:38:45 -08:00
Toshihiro Suzuki 5fa15cfde3
HBASE-25575 Should validate Puts in RowMutations (#2954) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-02-22 10:46:16 +09:00
Toshihiro Suzuki d8b86627ea
HBASE-25574 Revisit put/delete/increment/append related RegionObserver methods (#2953) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-02-22 09:37:30 +09:00
XinSun ed90a14995
HBASE-25562 ReplicationSourceWALReader log and handle exception immediately without retrying (#2943) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: shahrs87
 2021-02-20 10:20:54 +08:00
shahrs87 88057d8ab6
HBASE-25539: Add age of oldest wal metric (#2945) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-02-18 20:58:24 -08:00
Sandeep Pal 1beda0d598
HBASE-25541 : Setting the path to null when we dequeue the current log (#2959) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2021-02-18 11:53:52 +05:30
Bharath Vissapragada 618236dd90
HBASE-25547: Thread pools should release unused resources (#2922) 
Plumbs the configuration needed to enable core thread timeout on non-critical thread pools.
Currently only enabled for thread pools with op-codes RS_LOG_REPLAY_OPS, RS_PARALLEL_SEEK, MASTER_SNAPSHOT_OPERATIONS, MASTER_MERGE_OPERATIONS. Others can be added later as
needed.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-02-16 11:12:00 -08:00
pustota2009 b6649a8784
HBASE-23887 Bug fix heavyEvictionMbSizeLimit (ADDENDUM) (#2957) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-02-15 20:43:33 +05:30
pustota2009 b2f85e5fd2
HBASE-23887 AdaptiveLRU cache (#2934) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-02-13 19:57:28 +05:30
GeorryHuang 3e743df6d0
HBASE-25364 Redo the getMidPoint() in HFileWriterImpl to get rid of the double comparison process (#2741) 
Signed-off-by: niuyulin <nyl353@163.com>
Signed-off-by: stack <stack@apache.org>
 2021-02-10 08:31:07 -08:00
Baiqiang Zhao 25e3633e62
HBASE-25534 Honor TableDescriptor settings earlier in normalization (#2917) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-02-09 16:45:26 -08:00
XinSun 4a3ff98943
HBASE-25559 Terminate threads of oldsources while RS is closing (#2938) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-02-09 16:32:46 +08:00
haxl 0353909bc2
HBASE-25507 Leak of ESTABLISHED sockets when compaction encountered "java.io.IOException: Invalid HFile block magic" (#2882) 
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
Signed-off-by: stack <stack@apache.org>
 2021-02-08 22:14:00 -08:00
Michael Stack ae063f953e HBASE-25542 Add client detail to scan name so when lease expires, we … (#2930) 
* HBASE-25542 Add client detail to scan name so when lease expires, we have clue on who was scanning

When we create a scanner lease, record client ip and port (removed
unnecessary store of scannerName).

Signed-off-by: Clara Xiong <clarax98007@gmail.com>
 2021-02-08 14:40:31 -08:00
GeorryHuang ce9c9b453f
HBASE-25550 More readable Competition Time (#2925) 2021-02-08 13:12:24 -08:00
XinSun 355bccbd0b
HBASE-25560 Remove unused parameter named peerId in the constructor method of CatalogReplicationSourcePeer (#2939) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-02-08 12:29:05 -08:00
XinSun d6aff6cbae
HBASE-25553 It is better for ReplicationTracker.getListOfRegionServers to return ServerName instead of String (#2928) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-02-07 17:13:47 +08:00
Baiqiang Zhao 304379605a
HBASE-25554 NPE when init RegionMover (#2927) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-02-05 14:01:53 +05:30
YutSean 7905749733
HBASE-25543 When configuration hadoop.security.authorization is set to false, the system will still try to authorize an RPC and raise AccessDeniedException (#2919) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
 2021-02-05 15:37:34 +08:00
stack 3a690931f5 HBASE-25546 PerfTestCompactionPolicies is failing because of NPE (addendum2); add class comment 2021-02-04 11:27:06 -08:00
robin7roy 1e8ff16eae
HBASE-25546 PerfTestCompactionPolicies is failing because of NPE (addendum) (#2926) 
Added HBaseClassTestRule for PerfTestCompactionPolicies
 2021-02-04 07:25:40 -08:00
robin7roy d6d67d1e9f
HBASE-25546 PerfTestCompactionPolicies is failing because of NPE (#2921) 
Adds mock for getRegionInfo which was missing earlier.

Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-02-03 10:34:34 +05:30
Pankaj a04ea7ea44
HBASE-24900 Make retain assignment configurable during SCP (#2313) 
Retain assignment will be useful in non-cloud scenario where RegionServer and Datanode are deployed in same machine and will avoid remote read.

Signed-off-by: Guanghao Zhang <zghao@apache.org> 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
 2021-02-01 10:37:57 +05:30
shahrs87 d234b4dec2
[HBASE-25536] Remove 0 length wal file from logQueue if it belongs to old sources (#2908) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-01-29 17:47:30 +05:30
Bharath Vissapragada de51a40b53
HBASE-25528: Dedicated merge dispatch threadpool on master (#2904) 
Adds "hbase.master.executor.merge.dispatch.threads" and defaults to 2.

Also adds additional logging that includes the number of split plans
and merge plans computed for each normalizer run.

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-01-28 09:33:35 -08:00
Bo Cui 4aff481318
HBASE-25506 ServerManager#startChore affects MTTR of HMaster (#2889) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-01-28 22:55:05 +08:00
Aman Poonia 4a66f68f9c
HBASE-25523 Region normalizer chore thread is getting killed (#2903) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-01-26 11:55:12 +05:30
Baiqiang Zhao 0b0cdcac53
HBASE-25522 Remove deprecated methods in ReplicationPeerConfig (#2898) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-01-24 21:07:03 +05:30
huaxiangsun 5c13c68494
HBASE-25368 Filter out more invalid encoded name in isEncodedRegionNa… (#2868) 
HBASE-25368 Filter out more invalid encoded name in isEncodedRegionName(byte[] regionName)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2021-01-20 09:04:50 -08:00
Duo Zhang a37e727990
HBASE-25509 ChoreService.cancelChore will not call ScheduledChore.cle… (#2890) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-01-20 16:10:36 +08:00
Baiqiang Zhao 871eb09b3d
HBASE-25482 Improve SimpleRegionNormalizer#getAverageRegionSizeMb (#2858) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-01-19 11:09:50 -08:00
Baiqiang Zhao bc4f5c2709
HBASE-25513 When the table is turned on normalize, the first region may not be merged even the size is 0 (#2887) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2021-01-19 07:53:51 -08:00
Anjan Das d5debe15b5
HBASE-25475 : Unset zk based wal splitting explicitly in tests (ADDENDUM) (#2891) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-01-17 16:31:07 +05:30
Christine Feng 643548f5f5
HBASE-25478 - Implement retries when enabling tables in TestRegionReplicaReplicationEndpoint (#2866) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-01-16 13:33:13 +05:30
Anjan Das 1474ebcc69
HBASE-25475: Improve UT added as part of HBASE-25445 in TestSplitWALManager (#2855) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-01-15 12:18:12 +05:30
Baiqiang Zhao 9b670a489b
HBASE-25431 MAX_FILESIZE and MEMSTORE_FLUSHSIZE should not be set negative number (#2803) 
Signed-off-by: stack <stack@apache.org>
 2021-01-14 22:00:50 -08:00
caroliney14 dfefff7e59
HBASE-25329 Dump ritsOverThreshold in logs (#2761) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-01-13 12:10:12 +05:30
Viraj Jasani a3913a39c3
HBASE-25211 : Rack awareness in RegionMover (#2795) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2021-01-12 15:31:26 -08:00
Tak Lon (Stephen) Wu 686b72c44e
HBASE-25249 Adding StoreContext (#2800) 
Adding StoreContext which contains the metadata about the HStore. This
metadata can be used across the HFileWriter/Readers and other HStore
consumers without the need of passing around the complete store and
exposing its internals.

Co-authored-by: Abhishek Khanna <akkhanna@amazon.com>

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
 2021-01-08 20:16:45 -08:00
xijiawen 5c233e9785
HBASE-25459 WAL can't be cleaned in some scenes (#2848) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-01-08 16:00:15 +00:00
Bo Cui a348204d4a
HBASE-25483 set the loadMeta log level to debug (#2859) 
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-01-08 19:27:23 +05:30
xijiawen 7d70141b90
HBASE-25434 SlowDelete & SlowPut metric value should use updateDelete & updatePut (#2837) 
Signed-off-by: Zheng Wang <wangzheng@apache.org>
 2021-01-08 18:58:30 +08:00
Duo Zhang 45ef88dc8d
HBASE-25471 Move RegionScannerImpl out of HRegion (#2853) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2021-01-08 17:50:08 +08:00
Anjan Das 4b6215297d
HBASE-25445: Use WAL FS instead of master FS in SplitWALManager (#2844) 
Signed-off-by: Pankaj <pankajkumar@apache.org>
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-01-07 15:31:50 +05:30
mokai a414361ed9
HBASE-24755 [LOG][RSGroup]Error message is confusing while adding a offline RS to rsgroup (#2846) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2021-01-06 19:57:58 +05:30
Duo Zhang bedb45d4ac
HBASE-25458 HRegion methods cleanup (#2838) 
Signed-off-by: meiyi <myimeiyi@gmail.com>
 2021-01-06 15:13:10 +08:00
Wellington Ramos Chevreuil 85842634e5
HBASE-24813 ReplicationSource should clear buffer usage on Replicatio… (#2546) 
Signed-off-by: Ankit Singhal <ankit@apache.org>
 2021-01-05 10:21:26 +00:00
Bo Cui 600be60a4b
HBASE-25447 remoteProc is suspended due to OOM ERROR (#2824) 
Some OMME can not cause the JVM to exit, like "java.lang.OutOfMemoryError: Direct buffer memory", "java.lang.OutOfMemoryError: unable to create new native thread", as they dont call vmError#next_OnError_command. So abort HMaster when uncaught exception occurs in TimeoutExecutor, the new active Hmaster will resume the suspended procedure.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.com>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
 2021-01-04 21:04:38 +05:30
申胜利 eef076828a
Shenshengli hbase-25450 The parameter "hbase.bucketcache.size" is misdescribed (#2821) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: stack <stack@apache.org>
 2021-01-03 09:27:08 -08:00
GeorryHuang 8b9a074bf7
HBASE-24751 Display Task completion time and/or processing duration on Web UI (#2815) 
Signed-off-by: stack <stack@apache.org>
 2021-01-02 11:22:36 -08:00
lujiefsi 126d01dae3
HBASE-25456 : add security check for setRegionStateInMeta (#2833) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2021-01-01 12:17:34 +05:30
leyangyueshan 043da5f5ee
HBASE-25435 Slow metric value can be configured (#2823) 
* HBASE-25435 Slow metric value can be configured

* fix  HBASE-25435

* hbase-25435 add blank

* fix hbase-25435-2

Co-authored-by: stevenxi <stevenxi@tencent.com>
 2020-12-31 22:37:31 +08:00
Duo Zhang fbf00f9c28
HBASE-25451 Upgrade commons-io to 2.8.0 (#2825) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
 2020-12-31 16:57:27 +08:00
lujiefsi 29a9a16de4
HBASE-25441 : add security check for some APIs in RSRpcServices (#2810) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-12-31 00:22:26 +05:30
Duo Zhang 55a4eca9e6
HBASE-25452 Use MatcherAssert.assertThat instead of org.junit.Assert.assertThat (#2826) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-12-30 09:47:56 +08:00
Pankaj c96fbf0407
HBASE-25379 Make retry pause time configurable for regionserver short operation RPC (reportRegionStateTransition/reportProcedureDone) (#2757) 
* HBASE-25379 Make retry pause time configurable for regionserver short operation RPC (reportRegionStateTransition/reportProcedureDone)
* HBASE-25379 RemoteProcedureResultReporter also should retry after the configured pause time
* Addressed the review comments

Signed-off-by: Yulin Niu <niuyulin@apache.org>
 2020-12-29 22:25:36 +05:30
lujiefsi d963342f8a
HBASE-25432:add security checks for setTableStateInMeta and fixMeta (#2809) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-12-28 10:57:30 -08:00
ramkrish86 140c7f6ea0
HBASE-24850 CellComparator perf improvement (#2802) 
* Using ContiguousCellFormat as a marker alone

* Commit the new file

* Fix the comparator logic that was an oversight

* Fix the sequenceId check order

* Adding few more static methods that helps in scan flow like query
matcher where we have more cols

* Remove ContiguousCellFormat and ensure compare() can be inlined

* applying negation as per review comment

* Fix checkstyle comments

* fix review comments

* Address review comments

* Fix the checkstyle issues

* Fix javadoc

Signed-off-by: stack <stack@apache.org>
Signed-off-by: AnoopSamJohn <anoopsamjohn@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
 2020-12-28 13:02:06 +05:30
Huang Zhuoyue 0f868da05d HBASE-25443 Improve the experience of using the Master webpage by change the loading process of snapshot list to asynchronous 2020-12-25 10:09:32 +08:00
Mohammad Arshad dcb38f47db
HBASE-25371: When openRegion fails during initial verification(before… (#2785) 
Signed-off-by: stack <stack@apache.org>
 2020-12-21 08:11:22 -08:00
scotthunt 904b555edc HBASE-25084 Add "regexstringnocase" to ParseFilter for case-insensitivity (#2784) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-12-21 08:44:33 +00:00
Wellington Chevreuil d13010fa26 Revert "[PATCH] Add "regexstringnocase" to ParseFilter for case-insensitivity (#2784)" 
Will re-commit with the proper jira ID

This reverts commit 51cee0016f.
 2020-12-21 08:41:52 +00:00
scotthunt 51cee0016f
[PATCH] Add "regexstringnocase" to ParseFilter for case-insensitivity (#2784) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-12-21 08:39:13 +00:00
Duo Zhang 772350b26e
HBASE-25419 Remove deprecated methods in RpcServer implementation (#2791) 
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: stack <stack@apache.com>
 2020-12-20 12:00:07 +08:00
Duo Zhang 1540b89cee
HBASE-25420 Some minor improvements in rpc implementation (#2792) 
Signed-off-by: XinSun <ddupgs@gmail.com>
Signed-off-by: stack <stack@apache.com>
 2020-12-20 11:26:36 +08:00
Akshay Sudheer 7b1e9cd0d2
HBASE-25404 Procedures table Id under master web UI gets word break to single character (#2783) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-12-19 12:37:21 -08:00
Duo Zhang 33441f540a Revert "HBASE-25368 Filter out more invalid encoded name in isEncodedRegionName(byte[] regionName) (#2753)" 
This reverts commit c327680125.
 2020-12-19 20:28:30 +08:00
Sandeep Pal f600856a3b
HBASE-25383: Ability to update and remove peer base config 
Closes #2778

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Geoffrey Jacoby <gjacoby@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-12-18 14:19:17 +05:30
stack 888a4fb86a HBASE-25400 [Flakey Tests] branch-2 TestRegionMoveAndAbandon 2020-12-16 10:39:11 -08:00
shahrs87 74d68180e6
HBASE-25246 Backup/Restore hbase cell tags 
Closes #2745

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-12-16 18:52:36 +05:30
huaxiangsun c327680125
HBASE-25368 Filter out more invalid encoded name in isEncodedRegionName(byte[] regionName) (#2753) 
Signed-off-by: stack <stack@apache.com>
 2020-12-15 21:52:54 -08:00