Commit Graph

8531 Commits

Author SHA1 Message Date
niuyulin 8a6299bb38
HBASE-25076 fix typo in MasterRegion java doc (#2432)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>

Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-09-21 19:00:49 +02:00
niuyulin 36e8b0beec
HBASE-25075 Fix typo in ReplicationProtobufUtil (#2431)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>

Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-09-21 18:44:47 +02:00
stack 3e13b3f059
HBASE-24896 'Stuck' in static initialization creating RegionInfo instance
Closes #2422

Untangle RegionInfo, RegionInfoBuilder, and MutableRegionInfo static
initializations some. Move MutableRegionInfo from inner-class of
RegionInfoBuilder to be (package private) standalone. Undo static
initializing references from RI to RIB.

Co-authored-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-20 14:18:32 +05:30
Mohammad Arshad fd7260b506
HBASE-25069: Display region name instead of encoded region name in HBCK report page. (#2428)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-20 09:00:28 +08:00
Joseph295 58618e35ad
HBASE-24991 Replace MovedRegionsCleaner with guava cache (#2357)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-19 14:53:13 +08:00
Hyeran Lee 6371914adb
HBASE-25057: Fix typo memeber (#2414)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-09-19 14:36:06 +08:00
GeorryHuang 16831ec515
HBASE-24857:Fix several problems when starting webUI (#2245)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-09-19 14:34:30 +08:00
Joseph295 9fc586a7eb
HBASE-25047 WAL split edits number is negative in RegionServerUI (#2408)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-17 09:38:28 +08:00
Viraj Jasani 35d59dc570
HBASE-24528 : BalancerDecision queue implementation in HMaster with Admin API
* Admin API getLogEntries() for ring buffer use-cases: so far, provides balancerDecision and slowLogResponse
* Refactor RPC call for similar use-cases
* Single RPC API getLogEntries() for both Master.proto and Admin.proto

Closes #2261

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-09-16 20:56:54 +05:30
Duo Zhang 1bb19e0cdd
HBASE-25037 Lots of thread pool are changed to non daemon after HBASE-24750 which causes trouble when shutting down (#2407)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-16 21:11:47 +08:00
XinSun a918bf0d7b
HBASE-25012 HBASE-24359 causes replication missed log of some RemoteException (#2384)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-16 18:01:55 +08:00
bsglz c5ca191921
HBASE-24875 Remove the force param for unassign since it dose not take effect any more (#2254)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-09-15 22:05:03 -05:00
Mohammad Arshad 6d985dd7c6
HBASE-25009: Hbck chore logs wrong message when loading regions from RS (#2379)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-16 09:08:04 +08:00
Duo Zhang 724a0e5500
HBASE-25022 Remove 'hbase.testing.nocluster' config (#2394)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-14 20:15:43 +08:00
Wellington Ramos Chevreuil 2042523f43
HBASE-24877 Add option to avoid aborting RS process upon uncaught exc… (#2255)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-09-14 11:13:14 +01:00
Clara Xiong fe776306d0
HBASE-25006 Make the cost functions optional for StochastoicBalancer
Closes #2386

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-13 14:16:02 +05:30
ankitjain64 9c5dbb29c0
HBASE-24764: Add support of adding default peer configs via hbase-site.xml for all replication peers. (#2284)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-09-10 10:08:44 -07:00
WenFeiYi bbfbe33520
HBASE-24958 CompactingMemStore.timeOfOldestEdit error update (#2321) 2020-09-10 09:46:04 -07:00
Viraj Jasani ce59a2ba30
HBASE-25004 : Log RegionTooBusyException details (#2371)
Signed-off-by: mnpoonia <apoonia@salesforce.com>
Signed-off-by: stack <stack@apache.org>
2020-09-10 21:22:37 +05:30
Duo Zhang 23713f4693
HBASE-25005 Refactor CatalogJanitor (#2373)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-10 11:59:15 +08:00
Sandeep Pal 84a34be761
HBASE-24974: Provide a flexibility to print only row key and filter for multiple tables in the WALPrettyPrinter (#2345)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-09-09 10:16:45 -07:00
Duo Zhang 0511089066
HBASE-25000 Move delete region info related methods to RegionStateStore (#2366)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-09 23:11:28 +08:00
niuyulin 2e96a5b2d3
HBASE-24993 Remove OfflineMetaRebuildTestCore (#2359)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-09-09 09:47:15 +08:00
Javier Akira Luca de Tena 2e638de6a3
HBASE-24994 Add hedgedReadOpsInCurThread metric (#2365)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-09-09 09:45:46 +08:00
Mohammad Arshad 2250b51fe7
HBASE-24995: MetaFixer fails to fix overlaps when multiple tables have overlaps (#2361)
Signed-off-by: stack <stack@apache.org>
2020-09-08 12:00:44 -07:00
Viraj Jasani 0d95a8f91b
HBASE-24979 : Client operation timeout test for batch requests
Closes #2347

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-08 17:51:38 +05:30
Toshihiro Suzuki d48c732851
HBASE-24602 Add Increment and Append support to CheckAndMutate (#2228) 2020-09-08 15:05:47 +09:00
Duo Zhang be984cc8d4
HBASE-24986 Move ReplicationBarrier related methods to a separated class (#2354)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-07 20:35:27 +08:00
XinSun c2721c8235
HBASE-24759 Refuse to update configuration of default group (#2126)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-04 08:19:39 +08:00
Mohammad Arshad a352706700
HBASE-24940: runCatalogJanitor() API should return -1 to indicate already running status
Closes #2331

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-03 15:52:18 +05:30
Qi Yu 4dc08144f3
HBASE-24973 Remove read point parameter in method StoreFlush#performFlush and StoreFlush#createScanner (#2337)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-02 17:43:26 +08:00
Duo Zhang 85a174f5b3 HBASE-24964 addendum fix TestMetaTableMetrics (#2340)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-02 10:28:53 +08:00
Andrew Purtell 74807b951b
HBASE-24971 Upgrade JQuery to 3.5.1 (#2336)
- Update embedded minimized jquery to 3.5.1
- Update embedded minimized jquery.tablesorter to 2.31.3

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-01 10:39:12 -07:00
Zheng Wang ee450ae943
HBASE-24569 Get hostAndWeights in addition using localhost if it is null in local mode
Closes #1909

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-09-01 18:16:10 +05:30
XinSun 25fcc40f6a
HBASE-24913 Refactor TestJMXConnectorServer (#2286)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-08-31 10:23:59 +08:00
Mohammad Arshad bb64070bd3
HBASE-24916: Region hole contains wrong regions pair when hole is cre… (#2304)
Signed-off-by: stack <stack@apache.org>
2020-08-30 10:04:40 -07:00
Viraj Jasani 1c568ec95d
HBASE-24968 : Move META_COMPARATOR to subclass MetaCellComparator
* Break subclass referencing of MetaCellComparator from superclass CellComparatorImpl
  static initializer by moving META_COMPARATOR to subclass MetaCellComparator

Closes #2329

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-08-30 22:03:57 +05:30
Zheng Wang 2eb2c96960
HBASE-24894 [Flakey Test] TestStochasticLoadBalancer.testMoveCostMultiplier
Closes #2270

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-30 20:12:16 +05:30
Duo Zhang 2c4204ad75
HBASE-24964 Remove MetaTableAccessor.tableExists (#2330)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-30 17:31:26 +08:00
Duo Zhang ef5da4a61d
HBASE-24949 Optimize FSTableDescriptors.get to not always go to fs when cache miss (#2317)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-08-29 22:26:36 +08:00
XinSun 7909e29de5
HBASE-24760 Add a config hbase.rsgroup.fallback.enable for RSGroup fallback feature (#2149)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-08-28 16:52:45 +08:00
bsglz 047e0618d2
HBASE-24898 Addendum to deal with Daylight Saving Time (#2310)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-08-27 11:17:23 +08:00
WenFeiYi f9ce1df3f2
HBASE-24892 config 'hbase.hregion.memstore.mslab.indexchunksize' not be used (#2265)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-08-27 08:24:40 +08:00
Bharath Vissapragada 01cf60067c
HBASE-24765: Dynamic master discovery (#2130)
This patch adds the ability to discover newly added masters
dynamically on the master registry side. The trigger for the
re-fetch is either periodic (5 mins) or any registry RPC failure.
Master server information is cached in masters to avoid repeated
ZK lookups.

Updates the client side connection metrics to maintain a counter
per RPC type so that clients have visibility into counts grouped
by RPC method name.

I didn't add the method to ZK registry interface since there
is a design discussion going on in splittable meta doc. We can
add it later if needed.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-08-25 15:09:03 -07:00
Duo Zhang 227084c41f
HBASE-24945 Remove MetaTableAccessor.getRegionCount (#2303)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-25 21:15:22 +08:00
Duo Zhang 1220a8775c
HBASE-24942 MergeTableRegionsProcedure should not call clean merge region (#2301)
Signed-off-by: stack <stack@apache.org>
2020-08-25 12:19:19 +08:00
Duo Zhang 57e49b3959
HBASE-23834 HBase fails to run on Hadoop 3.3.0/3.2.2/3.1.4 due to jetty version mismatch (#2222)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-08-25 12:05:52 +08:00
Michael Stack 6ad73b9668 HBASE-24885 STUCK RIT by hbck2 assigns (#2283)
Adds region state check on hbck2 assigns/unassigns. Returns pid of -1
if in inappropriate state with logging explaination which suggests
passing override if operator wants to assign/unassign anyways. Here
is an example of what happens now if hbck2 tries an unassign and
Region already unassigned:

  2020-08-19 11:22:06,926 INFO  [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=50086] assignment.AssignmentManager(820): Failed {ENCODED => d1112e553991e938b6852f87774c91ee, NAME => 'TestHbck,zzzzz,1597861310769.d1112e553991e938b6852f87774c91ee.', STARTKEY => 'zzzzz', ENDKEY => ''} unassign, override=false; set override to by-pass state checks.
  org.apache.hadoop.hbase.client.DoNotRetryRegionException: Unexpected state for state=CLOSED, location=null, table=TestHbck, region=d1112e553991e938b6852f87774c91ee
          at org.apache.hadoop.hbase.master.assignment.AssignmentManager.preTransitCheck(AssignmentManager.java:583)
          at org.apache.hadoop.hbase.master.assignment.AssignmentManager.createOneUnassignProcedure(AssignmentManager.java:812)
          at org.apache.hadoop.hbase.master.MasterRpcServices.unassigns(MasterRpcServices.java:2616)
          at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$HbckService$2.callBlockingMethod(MasterProtos.java)
          at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:397)
          at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
          at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
          at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)

Previous it would just create the unassign anyways. Now must pass override
to queue the procedure regardless. Safer.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
 javadoc on assigns/unassigns. Minor refactor in assigns/unassigns to cater to
 case where procedure may come back null (if override not set and fails state checks).

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 checkstyle cleanups.
 Clarifying javadoc on how there is no state checking when bulk assigns creating/enabling
 tables.

 createOneAssignProcedure and createOneUnassignProcedure now handle exceptions which now
 can be thrown if no override and region state is not appropriate.

 Aggregation of createAssignProcedure and createUnassignProcedure instances adding in
 region state check invoked if override is NOT set.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateNode.java
 Change to setProcedure so it returns passed proc as result instead of void

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-08-24 09:22:16 -07:00
XinSun e0c9f911ad
HBASE-24871 Replication may loss data when refresh recovered replication sources (#2249)
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-08-24 21:43:15 +08:00
niuyulin 2874f00a2f
HBASE-24928 balanceRSGroup should skip generating balance plan for disabled table and splitParent region (#2292)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-08-24 11:52:21 +08:00
Yechao Chen 9f62a82334
HBASE-24811 Use class access static field or method (#2189)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-08-23 11:24:44 +08:00
Zheng Wang 0fe97947de
HBASE-24898 Use EnvironmentEdge.currentTime() instead of System.currentTimeMillis() in CurrentHourProvider
Closes #2272

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-22 15:30:09 +05:30
1996fanrui 5b515de792
HBASE-24915 Improve BlockCache read performance by specifying BlockType (#2288)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-08-21 09:31:39 -07:00
Duo Zhang 5c298790f2
HBASE-24926 Should call setFailure in MergeTableRegionsProcedure when isMergeable returns false (#2290)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-21 22:01:33 +08:00
Guanghao Zhang 54fe81eb56
HBASE-24912 Enlarge MemstoreFlusherChore/CompactionChecker period for unit test (#2285)
Signed-off-by: stack <stack@apache.org>
2020-08-21 12:41:20 +08:00
XinSun 65d28da7c2
HBASE-24914 Remove duplicate code appearing continuously in method ReplicationPeerManager.updatePeerConfig (#2287)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-08-20 14:44:56 -07:00
Toshihiro Suzuki 1164531d5a
HBASE-24884 BulkLoadHFilesTool/LoadIncrementalHFiles should accept -D options from command line parameters (#2260)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-08-19 13:36:17 +09:00
Nick Dimiduk ecc0c67077 HBASE-24627 Normalize one table at a time
Introduce an additional method to our Admin interface that allow an
operator to selectivly run the normalizer. The IPC protocol supports
general table name select via compound filter.

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-18 09:46:29 -07:00
niuyulin ea26463a33
HBASE-24880 Remove ReplicationPeerConfigUpgrader
Closes #2266

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-17 19:20:53 +05:30
Nick Dimiduk 2e8120a63f HBASE-24583 Normalizer can't actually merge empty regions...
when neighbor is larger than average size

* add `testMergeEmptyRegions` to explicitly cover different
  interleaving of 0-sized regions.
* fix bug where merging a 0-size region is skipped due to large
  neighbor.
* remove unused `splitPoint` from `SplitNormalizationPlan`.
* generate `toString`, `hashCode`, and `equals` methods from Apache
  Commons Lang3 template on `SplitNormalizationPlan` and
  `MergeNormalizationPlan`.
* simplify test to use equality matching over `*NormalizationPlan`
  instances as plain pojos.
* test make use of this handy `TableNameTestRule`.
* fix line-length issues in `TestSimpleRegionNormalizer`

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
Signed-off-by: Aman Poonia <aman.poonia.29@gmail.com>
2020-08-12 10:04:45 -07:00
Duo Zhang a535c7b1af
HBASE-24856 Fix error prone error in FlushTableSubprocedure (#2242)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-08-12 09:51:12 +08:00
Viraj Jasani ea130249ae
HBASE-24750 : Adding default UncaughtExceptionHandler for Thread factories (ADDENDUM)
Closes #2231
2020-08-11 17:18:47 +05:30
Bharath Vissapragada 1d65ad464f
HBASE-24846: Address compaction races in TestFIFOCompactionPolicy (#2203)
Looped through the test 100 times and it passes. Without the patch it fails
every ~10 runs or so.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2020-08-10 16:53:14 -07:00
Bharath Vissapragada 8ad59775c9
Revert "HBASE-21905 (addendum): Address compaction races in TestFIFOCompactionPolicy (#2203)"
This reverts commit 8003a15835.

Reverting to apply again as HBASE-24846.
2020-08-10 16:52:36 -07:00
Nick Dimiduk c1c2e160ec HBASE-24835 Normalizer should log a successful run at INFO level
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-10 09:57:48 -07:00
chenglei 7b099eaa75
HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length(addendum) (#2055)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-08-10 09:57:26 +08:00
Duo Zhang 726756a8b9
HBASE-23157 WAL unflushed seqId tracking may wrong when Durability.ASYNC_WAL is used (#762)
Signed-off-by: stack <stack@apache.org>
2020-08-10 09:11:31 +08:00
stack 11ac0a1ba6 HBASE-24830 Some tests involving RS crash fail with NullPointerException after HBASE-24632 in branch-2 2020-08-09 11:22:19 -07:00
Bo Cui d492aea3f8
HBASE-21721 FSHLog : reduce write#syncs() times (#2217)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Anoop <anoopsamjohn@apache.org>
2020-08-08 09:39:48 +05:30
Josh Elser 124af6392c HBASE-24779 Report on the WAL edit buffer usage/limit for replication
Closes #2193

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-08-07 12:59:17 -04:00
Viraj Jasani 0b604d921a
HBASE-24750 : All ExecutorService should use guava ThreadFactoryBuilder
Closes #2196

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
2020-08-07 20:24:36 +05:30
bsglz 485e0d2fa4
HBASE-24694 Support flush a single column family of table (#2179)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-08-07 11:28:53 +01:00
niuyulin f710d2d654
HBASE-24659 Calculate FIXED_OVERHEAD automatically (#2018)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>

SIgned-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-08-06 14:50:32 -07:00
Michael Stack 0ae125ac6f HBASE-24817 Allow configuring WALEntry filters on ReplicationSource (#2198)
Allow specifying base WALEntry filter on construction of
ReplicationSource. Add means of being able to filter WALs by name.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
 Add constructor that allows passing a predicate for filtering *in* WALs
 and a list of filters for filtering *out* WALEntries. The latter was
 hardcoded to filter out system-table WALEntries. The former did not
 exist but we'll need it if Replication takes in more than just the
 default Provider.

Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-06 12:35:42 -07:00
Bharath Vissapragada 8003a15835
HBASE-21905 (addendum): Address compaction races in TestFIFOCompactionPolicy (#2203)
Looped through the test 100 times and it passes. Without the patch it fails
every ~10 runs or so.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2020-08-06 09:33:06 -07:00
sguggilam f8b887b327
HBASE-24823 Port HBASE-22762 Print the delta between phases in the split/merge/compact/flush transaction journals
Closes #2204

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-06 18:50:12 +05:30
Zheng Wang c39cad220d
HBASE-24821 simplify the logic of getRegionInfo in TestFlushFromClient to reduce redundancy code
Closes #2200

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-05 19:59:50 +05:30
Duo Zhang d2f5a5f27b
HBAE-24507 Remove HTableDescriptor and HColumnDescriptor (#2186)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
2020-08-04 10:31:42 +08:00
Michael Stack 148c185486 HBASE-24808 skip empty log cleaner delegate class names (WAS => cleaner.CleanerChore: Can NOT create CleanerDelegate= ClassNotFoundException) (#2181)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-08-03 13:16:27 -07:00
Michael Stack 21a0b8ea11 HBASE-24767 Change default to false for HBASE-15519 per-user metrics Set hbase.regionserver.user.metrics.enabled default to false; i.e. off. (#2132)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-08-03 11:37:54 -07:00
Viraj Jasani 9ec484a031
HBASE-24795 : TestRegionMover2#testWithSplitRegions fix (ADDENDUM) 2020-08-03 20:26:46 +05:30
Viraj Jasani ad7caf754a
HBASE-24795 : RegionMover to deal with unknown region while (un)loading
* RegionMover to ignore move failures for split/merged regions with ack mode
* Refactor MoveWithAck and MoveWithoutAck as high level classes
* UT for RegionMover gracefully handling split/merged regions while loading regions and throwing failure while loading offline regions

Closes #2172

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
2020-08-03 17:25:52 +05:30
gkanade 9a1bad84bf
HBASE-24713 RS startup with FSHLog throws NPE after HBASE-21751 (#2125)
* null check for writer if not initialized yet during syncrunner run

* Revert "null check for writer if not initialized yet during syncrunner run"

This reverts commit 72932ad0df.

* Writer check while trying to attain safe point
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
2020-08-03 10:04:55 +05:30
Zheng Wang 492cf108a9
HBASE-24704 Make Table Schema easier to view with multiple families
Closes #2042

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-02 23:26:04 +05:30
Sean Busbey 86ebbdd8a2
HBASE-24805 HBaseTestingUtility.getConnection should be threadsafe
* refactor how we use connection and async connection to rely on their access methods
* refactor initialization and cleanup of the shared connection
* incompatibly change HCTU's Configuration member variable to be final so it can be safely accessed from multiple threads.

Closes #2180

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-01 23:19:07 -05:00
Toshihiro Suzuki e22a2d2700
HBASE-24680 Refactor the checkAndMutate code on the server side (#2094)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-08-01 21:02:17 +09:00
Anoop Sam John a3f623eea7
HBASE-24695 FSHLog - close the current WAL file in a background thread. (#2168)
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-08-01 17:27:20 +05:30
Sean Busbey 840a55761b
HBASE-24794 hbase.rowlock.wait.duration should not be <= 0 (#2174)
if hbase.rowlock.wait.duration is <=0 then log a message and treat it as a value of 1ms.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-30 12:26:12 -05:00
ddupg 1b9269de4d
HBASE-24797 Move log code out of loop
Closes #2175

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-30 15:15:52 +05:30
Bharath Vissapragada f07f30ae24
HBASE-20226: Parallelize region manifest deletes (#2159)
We observed this delete call to be a bottleneck for table with lots of
regions. Patch attempts to parallelize them.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-07-29 10:59:23 -07:00
Sandeep Pal d65fb87a2e
HBASE-24788: Fix the connection leaks on getting hbase admin from unclosed connection (#2162)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-29 09:09:07 -07:00
WenFeiYi 5f27a009ab
HBASE-24752 NPE/500 accessing webui on master startup
Closes #2148

Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-29 20:19:29 +05:30
Michael Stack 652f1e664b HBASE-24766 Document Remote Procedure Execution (#2131)
Add outline of how the mechanism works generally.

Signed-off-by: Anoop Sam John anoopsamjohn@apache.org
Signed-off-by: Viraj Jasani vjasani@apache.org
Signed-off-by: Jan Hentschel jan.hentschel@ultratendency.com
2020-07-29 07:44:51 -07:00
Michael Stack 345b77a4ca
HBASE-24632 Enable procedure-based log splitting as default in hbase3 Add deprecation of 'classic' zk-based WAL splitter. (#2156)
Also fix three bugs:

 * We were trying to delete non-empty directory; weren't doing
 accounting for meta WALs where meta had moved off the server
 (successfully)
 * We were deleting split WALs rather than archiving them.
 * We were not handling corrupt files.

Deprecations and removal of tests of old system.

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-07-29 07:37:28 -07:00
YeChao Chen 5480f350c2
HBASE-24790 Remove unused counter from SplitLogCounters (#2164)
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-29 17:22:17 +05:30
Duo Zhang 7e6e7a7051
HBASE-24770 Reimplement the Constraints API and revisit the IA annotations on related classes (#2140)
Signed-off-by: stack <stack@apache.org>
2020-07-28 09:24:55 +08:00
Wellington Ramos Chevreuil 8c0d7fa5b8
HBASE-24758 Avoid flooding replication source RSes logs when no sinks… (#2118)
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-27 10:08:13 +01:00
WenFeiYi 975cdf7b88
HBASE-24665 all wal of RegionGroupingProvider together roll (#2021)
Co-authored-by: wen_yi <liu.wenwen@immomo.com>
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-24 22:54:25 +05:30
Viraj Jasani 09e7ccd42d
HBASE-24757 : ReplicationSink should limit row count in batch mutation based on hbase.rpc.rows.warning.threshold
Closes #2127

Signed-off-by: stack <stack@apache.org>
2020-07-24 13:16:02 +05:30
Duo Zhang 0b85729da4
HBASE-24762 Purge protobuf java 2.5.0 dependency (#2128)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-07-24 11:48:35 +08:00
Guanghao Zhang 32c7012ad7
HBASE-24743 Reject to add a peer which replicate to itself earlier (#2122)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-07-24 08:08:20 +08:00
surbhi 8191fbdd7d HBASE-22146 Removing a namespace-level space quota does not remove policies against contained tables
Closes #1935

Signed-off-by: Josh Elser <elserj@apache.org>
2020-07-21 20:12:46 -04:00
Viraj Jasani f35c5eaadd
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses) (#2109)
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-21 20:24:13 +05:30
Viraj Jasani 32796ad24e
Revert "HBASE-24743 Reject to add a peer which replicate to itself earlier (#2071)"
This reverts commit 6cf013ddc4.

TestReplicationAdmin and TestReplicationShell are broken on branch-2 and master respectively
2020-07-21 20:17:27 +05:30
Mingliang Liu 3d270ba6f9
HBASE-24696 Include JVM information on Web UI under "Software Attributes"
Closes #2087

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-07-21 16:53:58 +05:30
Viraj Jasani f2c087caeb
Revert "HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses)"
This reverts commit 16f306b4a1.

TestAdminShell and TestThriftHBaseServiceHandler are failing consistently
2020-07-21 13:50:27 +05:30
sguggilam 430602aed8
HBASE-24740 Enable journal logging for HBase snapshot operation (#2104)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-07-20 18:11:41 -07:00
Guanghao Zhang 6cf013ddc4
HBASE-24743 Reject to add a peer which replicate to itself earlier (#2071)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-07-21 08:37:56 +08:00
Sanjeet Nishad a857c6d90c
HBASE-24714 Error message is displayed in UI of table's compaction state if any region of that table is not open
Closes #2047

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-20 21:40:23 +05:30
Zheng Wang 9b32943697
HBASE-24709 Support MoveCostFunction use a lower multiplier in offpeak hours
Closes #2099

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-07-19 20:09:12 +05:30
Michael Stack 9b02a26a1d HBASE-24698 Turn OFF Canary WebUI as default (#2037)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-07-17 15:52:52 -07:00
Wellington Ramos Chevreuil bd42c75cac
HBASE-24747 Log an ERROR if HBaseSaslRpcServer initialisation fails with an uncaught exception (#2079)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-07-17 18:50:03 +01:00
Mohammad Arshad 1781185b7b
HBASE-24675: On Master restart all servers are assigned to default rsgroup
Closes #2053

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-17 22:38:19 +05:30
Zheng Wang 6ae2fdd6b2
HBASE-24746 The sort icons overlap the col name in master UI
Closes #2078

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-17 21:47:17 +05:30
Viraj Jasani 16f306b4a1
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses)
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-17 19:16:22 +05:30
Lars a4e5ccc798 HBASE-24742 Improve performance of SKIP vs SEEK logic. 2020-07-16 13:17:05 -07:00
huaxiangsun 2505c7760d
HBASE-24581 Skip compaction request/check for replica regions at the early stage. (#1986)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-07-14 15:49:20 -07:00
huaxiangsun 1360bee7f9
HBASE-24705 MetaFixer#fixHoles() does not include the case for read replicas (i.e, replica regions are not created) (#2062)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-14 15:48:19 -07:00
BukrosSzabolcs 3e709c6f53
HBASE-24720: Meta replicas not cleaned when disabled (#2057)
- make sure to always clean up excess meta replicas not just when their
number get decreased
- make sure NotServingRegionException is handled properly even when
wrapped
- add test

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-07-14 18:24:55 +02:00
Mohammad Arshad 70cfe2525e
HBASE-24721: rename_rsgroup overwriting the existing rsgroup
Closes #2059

Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-14 21:30:23 +05:30
bsglz 3bd54217a0
HBASE-24664 Some changing of split region by overall region size rath… (#2011)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-07-13 10:00:48 +01:00
Geoffrey Jacoby af1cc2fc44
HBASE-23744 - FastPathBalancedQueueRpcExecutor should enforce queue length of 0
Closes #1094

Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-13 12:15:10 +05:30
bsglz 724f0478ed
HBASE-24404 Support flush a single column family of region (#2032)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-10 18:13:50 +01:00
Zheng Wang b2321b3583
HBASE-24586 Add table level locality in table.jsp
Closes #1926

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-10 21:59:59 +05:30
Viraj Jasani f0bc33122e
HBASE-24712 : Remove flakes from TestMasterNoCluster#testStopDuringStart (#2045)
Forward-port from 91da188d78
2020-07-10 20:41:03 +05:30
Joseph295 67dfbe0406
HBASE-24663 Add procedure process time statistics UI (#2035)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-10 15:03:52 +08:00
niuyulin 5de1dff8f6
HBASE-24653 Show snapshot owner on Master WebUI (#2016)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-10 15:03:05 +08:00
XinSun 1d0e285a6f
HBASE-24489 Rewrite TestClusterRestartFailover.test since namespace table is gone on on master (#2025)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-10 10:03:57 +08:00
Wellington Chevreuil 7b92510bb4 Revert "HBASE-21596 Delete for a specific cell version can bring back version… (#2009)"
This reverts commit 1db89773e6.
2020-07-09 15:55:57 +01:00
Wellington Ramos Chevreuil 1db89773e6
HBASE-21596 Delete for a specific cell version can bring back version… (#2009)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-08 15:17:14 +01:00
huaxiangsun 90f4ff7d7c
HBASE-24688 AssignRegionHandler uses EventType.M_RS_CLOSE_META instead of EventType.M_RS_OPEN_META for meta region (#2029)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-07 09:58:25 -07:00
Toshihiro Suzuki 34e807a8b5
HBASE-24650 Change the return types of the new CheckAndMutate methods introduced in HBASE-8458 (#1991)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-07-07 12:49:20 +09:00
WenFeiYi 287f29818f
HBASE-24546 CloneSnapshotProcedure unlimited retry (#1900)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-06 13:45:19 +08:00
WenFeiYi 44691ce23c
HBASE-24578 [WAL] Add a parameter to config RingBufferEventHandler's SyncFuture count (#1923)
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-07-06 11:24:09 +08:00
chenglei a7a0e1a596
HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length. (#1970)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-07-04 21:00:35 +08:00
Baiqiang Zhao afe2eac7b6
HBASE-24560 Add a new option of designatedfile in RegionMover
Closes #1901

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-01 20:26:51 +05:30
xincunSong 9ad16aa376
HBASE-24624 Optimize table.jsp code (#1963)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2020-07-01 08:49:51 +08:00
stack 2c43f6cde7 HBASE-24661 TestHeapSize.testSizes failure
Revert "HBASE-24648 Remove the legacy 'forceSplit' related code at region server side (#1990)"

This reverts commit bfec964aef.
2020-06-30 15:57:49 -07:00
pengmq1 be57e40f36
HBASE-24289 Heterogeneous Storage for Date Tiered Compaction (#1730)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-30 15:10:04 +08:00
stack bfec964aef HBASE-24648 Remove the legacy 'forceSplit' related code at region server side (#1990)
Addendum to fix TestHeapSize
2020-06-29 14:20:39 -07:00
Duo Zhang 37b863bd0b
HBASE-24648 Remove the legacy 'forceSplit' related code at region server side (#1990)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-29 22:50:42 +08:00
Duo Zhang 46bf8944fd
HBASE-24646 Set the log level for ScheduledChore to INFO in HBTU (#1987)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-06-28 23:26:05 +08:00
Viraj Jasani 47742b2083
HBASE-24588 : createTableBegsSplit should consider tuning mergeEnabled for table (ADDENDUM) (#1988) 2020-06-27 19:09:19 +05:30
Duo Zhang af49b87367 HBASE-24635 Addendum make test more robust 2020-06-27 11:29:43 +08:00
Duo Zhang 16a25b74db
HBASE-24635 Split TestMetaWithReplicas (#1980)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-27 10:36:07 +08:00
Viraj Jasani 51ad588565
HBASE-24588 : normalization log format (ADDENDUM) 2020-06-26 23:09:37 +05:30
Viraj Jasani 71af97cd05
HBASE-24588 : Submit task for NormalizationPlan (#1933)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-06-26 22:08:52 +05:30
bsglz c0461207ee
HBASE-24382 Flush partial stores of region filtered by seqId when arc… (#1737)
* HBASE-24382 Flush partial stores of region filtered by seqId when archive wal due to too many wals

* fix checkstyle and javadoc issue

* fix javadoc issues

* move the geting of stores to HRegion, since it should not be part of FlushPolicy, and comment fix

* fix checkstyle issue

* add some comment

* remove the forceFlushAllStores since we can use families to determine how to select stores to flush

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@duboce.net>
2020-06-25 22:03:18 -07:00
Bharath Vissapragada 84e246f9b1
HBASE-24603: Make Zookeeper sync() call synchronous (#1945)
Writing a test for this is tricky. There is enough coverage for
functional tests. Only concern is performance, but there is enough
logging for it to detect timed out/badly performing sync calls.

Additionally, this patch decouples the ZK event processing into it's
own thread rather than doing it in the EventThread's context. That
avoids deadlocks and stalls of the event thread.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-25 13:27:38 -07:00
Michael Stack e54e3afc5c HBASE-24616 Remove BoundedRecoveredHFilesOutputSink dependency on a TableDescriptor (#1955)
Purge query Master for table descriptors; make do w/ generic options.

Logging cleanup.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
 Undo fetching Table Descriptor. Not reliably available at recovery time.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-25 12:10:17 -07:00