Commit Graph

2489 Commits

Author SHA1 Message Date
Duo Zhang 108e23630a HBASE-24506 async client deadlock (#1858)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-06-08 09:57:33 +08:00
Toshihiro Suzuki 467efa573c HBASE-24515 batch Increment/Append fails when retrying the RPC
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2020-06-08 10:41:41 +09:00
Duo Zhang db8924f975 HBASE-24513 The default readRpcTimeout and writeRpcTimeout is incorrectly calculated in AsyncConnectionConfiguration (#1863)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-06-07 18:36:05 +08:00
Ravi Kishore Valeti 3c0a250e58
HBASE-24499 Use a client property for SaslConnection MAX_ATTEMPTS (#1857)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-05 20:32:49 +05:30
Duo Zhang 72dadcd615 HBASE-24500 The behavior of RegionInfoBuilder.newBuilder(RegionInfo) is strange (#1850)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-05 15:24:39 +08:00
Duo Zhang 82b8feecef HBASE-24505 Reimplement Hbck.setRegionStateInMeta (#1852)
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
2020-06-05 08:55:27 +08:00
Viraj Jasani cc4c28c575
HBASE-24456 : Create ImmutableScan and use it for CustomizedScanInfoBuilder (#1818)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-04 16:30:32 +05:30
Duo Zhang 4a84e28937 Revert "HBASE-24500 The behavior of RegionInfoBuilder.newBuilder(RegionInfo) is strange (#1840)"
This reverts commit 8815215b3e.
2020-06-04 12:17:15 +08:00
Duo Zhang 8815215b3e HBASE-24500 The behavior of RegionInfoBuilder.newBuilder(RegionInfo) is strange (#1840)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2020-06-04 11:18:25 +08:00
Wellington Ramos Chevreuil 11d093bc39 HBASE-21406 "status 'replication'" should not show SINK if the cluste… (#1761)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>

(Cherry picked from commit e5345b3a7c)
2020-06-03 09:33:36 +01:00
Viraj Jasani b2d24a1812
HBASE-23941 : FilterBy operator support in get_slowlog_responses API (#1793)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-05-30 13:57:32 +05:30
Viraj Jasani 6f5e5e4828
HBASE-23938 : System table hbase:slowlog to store complete slow/large… (#1681)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
2020-05-20 15:30:30 +05:30
Mingliang Liu 5893c4f393
HBASE-23969 Meta browser should show all `info` columns (#1710)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-18 11:48:36 -07:00
Duo Zhang dc2146069c
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for … (#1697)
Signed-off-by: stack <stack@apache.org>
2020-05-13 17:59:21 +08:00
Michael Stack 231d7f8256 HBASE-24354 Make it so can make an hbase1 schema and hbase2 schema equate (#1696) 2020-05-12 12:34:14 -07:00
huaxiangsun fca0f3f810
HBASE-24316 GCMulitpleMergedRegionsProcedure is not idempotent (#1660) (#1671)
It addresses couple issues:
   1. Make sure deleteMergeQualifiers() does not delete the row if there is no columns with "merge" keyword.
   2. GCMulitpleMergedRegionsProcedure now acquire an exclusive lock on the child region.

Signed-off-by: stack <stack@apache.org>
2020-05-06 16:07:21 -07:00
Duo Zhang 7f4683bafa HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry … (#1593)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-06 16:11:28 +08:00
Michael Stack 80561504d2
HBASE-24284 [h3/jdk11] REST server won't start Exclude transitive includes of jax-rs 1.x and then explicitly include jax-rs 2.x glassfish impl for REST context when hadoop3. (#1625) 2020-05-05 15:27:11 -07:00
Michael Stack 1dd4b13e17
HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration (#1572)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Fix weird brackets around each region name when logging.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
  Log when we hit the max merge limit. Also up limit to 64.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
 Make logs make more sense to operator.

hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
 Make RegionName show when you mouseover so long names don't mess up
 display of holes and overlaps.

Address Mingliang Liu liuml07 feedback

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-05-04 08:21:52 -07:00
stack 93f0d4ce06 HBASE-24247 Failed multi-merge because two regions not adjacent (legitimately).
Addendum: Restore old semantic where merge of non-adjacent required
'force'
2020-04-29 14:34:37 -07:00
Duo Zhang 1f556b3ad9 HBASE-24281 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:25:41 +08:00
Duo Zhang 379b48bb5d Revert "HBASE-24181 Remove some stale hadoop.version properties in the pom for sub modules (#1602)"
This reverts commit dc9608ec0b.
2020-04-29 18:25:23 +08:00
Duo Zhang dc9608ec0b HBASE-24181 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:08:33 +08:00
Michael Stack 9e697d0ea6
HBASE-24247 Failed multi-merge because two regions not adjacent (legitimately) (#1570)
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add new isOverlap method that takes list of RegionInfos checking that
 current RegionInfo is overlapped by the passed in Regions.

    Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
    Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-04-28 13:22:00 -07:00
stack 3f43b9e349 HBASE-24158 [Flakey Tests] TestAsyncTableGetMultiThreaded
Addendum to address NPE
2020-04-16 08:02:54 -07:00
Viraj Jasani a0a769d96f
HBASE-24195 : Admin.getRegionServers() should return live servers exc… (#1523)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-16 20:16:30 +05:30
Bharath Vissapragada 91aa3ce97c HBASE-24156: Switch the default registry to ZK based
In branch-2 we want to retain the ZK based registry as the default.
2020-04-09 16:13:45 -07:00
Bharath Vissapragada db7ad07081 Revert "Switch the default registry to ZK based"
This reverts commit 558ee079fd because
it wasn't tagged with a jira ID. Will be re-applied.
2020-04-09 16:13:45 -07:00
Viraj Jasani 3433c7a2db
HBASE-23937 : Support Online LargeLogs similar to SlowLogs APIs (#1346)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-04-01 19:56:42 +05:30
Viraj Jasani 0b2b63ea84
HBASE-23678 : Builder API for version management - setVersionsWithTim… (#1381)
Signed-off-by: Xu Cang <xucang@apache.org>
2020-04-01 16:27:36 +05:30
stack b1eff98789 HBASE-24079 [Flakey Tests] Misc fixes and debug; fix BindException in Thrift tests; add waits on quota table to come online; etc.
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientAsyncPrefetchScanner.java
 Refactor to avoid NPE timing issue referencing lock during Construction.

hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
 Comment

hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
 Refactor. Catch NPE during startup and return it instead as failed initialization.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java
 Catch IndexOutOfBounds exception and convert to non-split request.

hbase-server/src/test/java/org/apache/hadoop/hbase/TestCachedClusterId.java
 Make less furious. Make it less flakie.

hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerSideScanMetricsFromClientSide.java
 Debug. Catch exception to log, then rethrow.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
 Guess that waiting longer on compaction to succeed may help make this
 less flakey.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
 Be explicit about timestamping to avoid concurrent edit landing
 server-side and messing up test expectation.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMasterRegistry.java
 Add wait on meta before proceeding w/ test.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestScannersFromClientSide.java
 Be explicit that edits are distinct.

hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCacheRefCnt.java
 Add @Ignore on RAM test... Fails sporadically.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionMoveAndAbandon.java
 Add wait for all RegionServers going down before proceeding; was
 messing up RS accounting.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/BalancerTestBase.java
 Make balancer test sloppier; less restrictive; would fail on occasion
 by being just outside test limits.

hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestQuotaObserverChoreRegionReports.java
 Add wait on quota table coming up; helps make this less flakie.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
 Be explicity about timestamps; see if helps w/ flakie failure.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java
 Catch and ignore if issue in shutdown; don't care if after test.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java
 Comment.

hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
 Add retry to see if helps w/ odd failure; grant hasn't propagated?

hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLWithMultipleVersions.java
 Explicit w/ timestamps so no accidental overlap of puts.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
 Hack to deal w/ BindException on startup.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThrift2ServerCmdLine.java
 Use loopback.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandler.java
 Disable flakie test.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-03-30 16:46:48 -07:00
stack 50161f2de4 HBASE-24034 [Flakey Tests] A couple of fixes and cleanups
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupMajorCompactionTTL.java
 Remove spurious assert. Just before this it waits an arbitrary 10
 seconds. Compactions could have completed inside this time. The spirit
 of the test remains.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java
 Get log cleaner to go down promptly; its sticking around. See if this
 helps with TestMasterShutdown

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
 We get a rare NPE trying to sync. Make local copy of SyncFuture and see
 if that helps.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
 Compaction  may have completed when not expected; allow for it.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestBlockEvictionFromClient.java
 Add wait before testing. Compaction may not have completed. Let
 compaction complete before progressing and then test for empty cache.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
 Less resources.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestDefaultLoadBalancer.java
 Less resources.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
 Wait till online before we try and do compaction (else request is
 ignored)

hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
 Disable test that fails randomly w/ mockito complaint on some mac os
 x's.

TestMasterShutdown... fix NPE in RSRpcDispatcher... catch it and covert
to false and have master check for successful startup.
2020-03-23 16:21:19 -07:00
Nick Dimiduk ffb2359146
HBASE-24013 Bump branch-2 version to 2.4.0-SNAPSHOT (#1309)
Increment version in poms with

```
$ mvn org.codehaus.mojo:versions-maven-plugin:2.7:set -DnewVersion=2.4.0-SNAPSHOT -DgenerateBackupPoms=false
```

Verified no dangling references with

```
$ find . -iname '*pom.xml' -exec grep -n '2.3.0-SNAPSHOT' {} +
```

Verified build with

```
$ JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home mvn clean package -DskipTests
$ JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home mvn clean package -DskipTests -Dhadoop.profile=3.0
```

Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-03-19 08:01:43 -07:00
Duo Zhang 26ab594d09 HBASE-23987 NettyRpcClientConfigHelper will not share event loop by default which is incorrect (#1288)
Signed-off-by: stack <stack@apache.org>
2020-03-14 14:06:45 +08:00
xuqinya1 6de2925404
HBASE-23967 Improve the accuracy of the method sizeToString (#1273)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-03-13 19:43:49 +05:30
Duo Zhang 7eeb6a0815 HBASE-23077 move entirely to spotbugs (#1265)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-03-12 11:42:23 +08:00
Michael Stack 2655f9647e
HBASE-23956 Use less resources running tests (#1266)
Add being able to configure netty thread counts. Enable socket reuse
(should not have any impact).

hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BlockingRpcConnection.java
 Rename the threads we create in here so they are NOT named same was
 threads created by Hadoop RPC.

hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/DefaultNettyEventLoopConfig.java
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcClient.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java
 Allow configuring eventloopgroup thread count (so can override for
 tests)

hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/HttpProxyExample.java
 Enable socket resuse.

hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
 Enable socket resuse and config for how many threads to use.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
hbase-server/src/main/java/org/apache/hadoop/hbase/util/ModifyRegionUtils.java
 Thread name edit; drop the redundant 'Thread' suffix.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
 Make closeable and shutdown executor when called.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
 Call close on HFileReplicator

hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationBase.java
 HDFS creates lots of threads. Use less of it so less threads overall.

hbase-server/src/test/resources/hbase-site.xml
hbase-server/src/test/resources/hdfs-site.xml
 Constrain resources when running in test context.

hbase-server/src/test/resources/log4j.properties
 Enable debug on netty to see netty configs in our log

pom.xml
 Add system properties when we launch JVMs to constrain thread counts in
 tests

 Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-11 10:25:11 -07:00
Michael Stack a6700bc6b5
HBASE-23851 Log networks and bind addresses when multicast publisher/listener enabled (#1173)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-03-10 10:55:44 -07:00
stack c5e0a1397b Revert "Put into all site.xmls and check system property is set"
Pushed by mistake

This reverts commit c8c2a87505.
2020-03-04 09:57:33 -08:00
stack c8c2a87505 Put into all site.xmls and check system property is set 2020-03-04 08:55:45 -08:00
Toshihiro Suzuki c3edceb6ae
HBASE-23146 Support CheckAndMutate with multiple conditions (#1209)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-04 15:08:31 +09:00
Viraj Jasani 6366b73134
HBASE-22978 : Online slow response log (#1228)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-03-01 00:32:35 +05:30
stack 836e1a1caf HBASE-23904 Procedure updating meta and Master shutdown are incompatible: CODE-BUG
Restore behavior from before HBASE-21789 (hbase-2.2.0) where we convert
all exceptions to IOEs, even RuntimeExceptions. Actual fix is this change (in case
obscured by doc and lambda simplification):

     } catch (Throwable e) {
-      Throwables.propagateIfPossible(e, IOException.class);
+      // Throw if an IOE else wrap in an IOE EVEN IF IT IS a RuntimeException (e.g.
+      // a RejectedExecutionException because the hosting exception is shutting down.
+      // This is old behavior worth reexamining. Procedures doing merge or split
+      // currently don't handle RuntimeExceptions coming up out of meta table edits.
+      // Would have to work on this at least. See HBASE-23904.
+      Throwables.throwIfInstanceOf(e, IOException.class);
2020-02-28 12:32:56 -08:00
stack 7df9490d60 HBASE-23899 [Flakey Test] Stabilizations and Debug
A miscellaney. Add extra logging to help w/ debug to a bunch of tests.
Fix some issues particular where we ran into mismatched filesystem
complaint. Some modernizations, removal of unnecessary deletes
(especially after seeing tests fail in table delete), and cleanup.
Recategorized one tests because it starts four clusters in the one
JVM from  medium to large. Finally, zk standalone server won't come
on occasion; added debug and thread dumping to help figure why (
manifests as test failing in startup saying master didn't launch).

hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
  Fixes occasional mismatched filesystems where the difference is file:// vs file:///
  or we pick up hdfs schema when it a local fs test. Had to do this
  vetting of how we do make qualified on a Path in a few places, not
  just here as a few tests failed with this same issue. Code in here is
  used by a lot of tests that each in turn suffered this mismatch.

  Refactor for clarity

hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
  Unused import.

hbase-procedure/src/test/java/org/apache/hadoop/hbase/procedure2/store/wal/TestWALProcedureStore.java
  This test fails if tmp dir is not where it expects because tries to
  make rootdir there. Give it a rootdir under test data dir.

hbase-server/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java
  This change is probably useless. I think the issue is actually
  a problem addressed later where our test for zk server being
  up gets stuck and never times out.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSplitOrMergeStatus.java
 Move off deprecated APIs.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/BalancerTestBase.java
 Log when we fail balance check for DEBUG Currently just says 'false'

hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestSplitWALProcedure.java
 NPEs on way out if setup failed.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
 Add logging when assert fails to help w/ DEBUG

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerAbortTimeout.java
 Don't bother removing stuff on teardown. All gets thrown away anyways.
 Saw a few hangs in here in the teardown where hdfs was down before
 expected messing up shutdown.

hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 Add timeout on socket; was seeing check for zk server getting stuck
 and never timing out (test time out in startup)

hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotWithTemporaryDirectory.java
 Write to test data dir instead.
 Be careful about how we make qualified paths.

hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanBase.java
 Remove snowflake configs.

hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStatus.java
 Add a hacky pause. Tried adding barriers but didn't work. Needs deep
 dive.

hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 Remove code copied from zk and use zk methods directly instead.
 A general problem is that zk cluster doesn't come up occasionally but
 no clue why. Add thread dumping and state check.
2020-02-28 12:14:41 -08:00
Bharath Vissapragada 558ee079fd Switch the default registry to ZK based
In branch-2 we want to retain the ZK based registry as the default.
2020-02-27 10:33:42 -08:00
Bharath Vissapragada 5848b7d835 HBASE-23804: Fix default master addr hostname in master registry (#1137)
Master rpc server end point doesn't bind to localhost's
IP address by default. Instead, it looks up the hostname and
binds to the endpoint to which it resolves. MasterRegistry should
do the same when building the default server end point to talk to.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
(cherry picked from commit c34dbc3c29)
2020-02-27 10:33:42 -08:00
Bharath Vissapragada 69e3e0e2ef HBASE-23647: Make MasterRegistry the default impl. (#1039)
Signed-off-by: Stack <stack@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 229b8aaaf3)
2020-02-27 10:33:42 -08:00
Bharath Vissapragada be30d43a6c HBASE-23305: Master based registry implementation (#954)
Implements a master based registry for clients.

 - Supports hedged RPCs (fan out configured via configs).
 - Parameterized existing client tests to run with multiple registry combinations.
 - Added unit-test coverage for the new registry implementation.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 62da419b23)
2020-02-27 10:33:42 -08:00
Bharath Vissapragada 1b52501b1b HBASE-23648: Re-use underlying connection registry in RawAsyncHBaseAdmin (#994)
* HBASE-23648: Re-use underlying connection registry in RawAsyncHBaseAdmin

No need to create and close a new registry on demand. Audited other
usages of getRegistry() and the code looks fine.

* Fix checkstyle issues in RawAsyncHBaseAdmin

(cherry picked from commit 07c38260f5)
2020-02-27 10:33:42 -08:00
Bharath Vissapragada c650f28ab4 HBASE-23604: Clarify AsyncRegistry usage in the code. (#957)
* HBASE-23604: Cleanup AsyncRegistry interface

- Cleans up the method names to make more sense and adds a little
more javadocs for context. In future patches we can revisit
the name of the actual class to make it more self explanatory.

- Does AsyncRegistry -> ConnectionRegistry rename.
"async" ness of the registry is kind of implicit based on
the interface contents and need not be reflected in the name.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 12bb41eb2c)
2020-02-27 10:33:42 -08:00
Bharath Vissapragada 488460e840 HBASE-23304: RPCs needed for client meta information lookup (#904)
* HBASE-23304: RPCs needed for client meta information lookup

This patch implements the RPCs needed for the meta information
lookup during connection init. New tests added to cover the RPC
code paths. HBASE-23305 builds on this to implement the client
side logic.

Fixed a bunch of checkstyle nits around the places the patch
touches.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 4f8fbba0c0)
2020-02-27 10:33:42 -08:00
Bharath Vissapragada 89581d9d21 HBASE-23281: Track meta region locations in masters (#830)
* HBASE-23281: Track meta region changes on masters

This patch adds a simple cache that tracks the meta region replica
locations. It keeps an eye on the region movements so that the
cached locations are not stale.

This information is used for servicing client RPCs for connections
that use master based registry (HBASE-18095). The RPC end points
will be added in a separate patch.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
(cherry picked from commit 8571d389cf)
2020-02-27 10:33:42 -08:00
Sakthi 4b704346b5 HBASE-22827 Expose multi-region merge in shell and Admin API
mergeRegionsAsync admin API with two regions as parameters deprecated since 2.3.0 and removed from 4.0
merge_region shell command now supports multiple regions merge since 2.3.0 & 3.0.0

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
(cherry picked from commit 5f61df4792)
2020-02-17 22:39:16 -08:00
Nick Dimiduk bb8d9df5b1
Revert "HBASE-23304: RPCs needed for client meta information lookup (apache#904) (#1098)"
This reverts commit 71f035450d.
2020-02-04 16:31:24 -08:00
Michael Stack 12f4e0977c
HBASE-23780 Edit of test classifications (#1109)
These classifications come of running at various fork counts.. A test
may complete quick if low fork count but if it is accessing disk, it
will run much slower if fork count is high. This edit accommodates
some of this phenomenon.


Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-02-03 09:16:47 -08:00
stack 97e82184a1 HBASE-23753 Update of errorprone generated failures
hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
 Complains about mismatch in types when Compare. Implement Compare in
 base Interface.

hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
 Complains pbs never return null.

hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSinkManager.java
 Needed redo because errorprone complains can't mock Service from guava.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicasWithRestartScenarios.java
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSnapshotScannerHDFSAclController.java
 Unrelated...adding one-liner debug statements chasing other test
 failures.
2020-01-28 17:01:30 -08:00
Bharath Vissapragada 71f035450d
HBASE-23304: RPCs needed for client meta information lookup (apache#904) (#1098)
* HBASE-23257: Track clusterID in stand by masters (#798)

This patch implements a simple cache that all the masters
can lookup to serve cluster ID to clients. Active HMaster
is still responsible for creating it but all the masters
will read it from fs to serve clients.

RPCs exposing it will come in a separate patch as a part of
HBASE-18095.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Guangxu Cheng <guangxucheng@gmail.com>
(cherry picked from commit c2e01f2398)

* HBASE-23275: Track active master's address in ActiveMasterManager (#812)

Currently we just track whether an active master exists.
It helps to also track the address of the active master in
all the masters to help serve the client RPC requests to
know which master is active.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit efebb843af)

* HBASE-23281: Track meta region locations in masters (#830)

* HBASE-23281: Track meta region changes on masters

This patch adds a simple cache that tracks the meta region replica
locations. It keeps an eye on the region movements so that the
cached locations are not stale.

This information is used for servicing client RPCs for connections
that use master based registry (HBASE-18095). The RPC end points
will be added in a separate patch.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
(cherry picked from commit 8571d389cf)

* HBASE-23304: RPCs needed for client meta information lookup (#904)

* HBASE-23304: RPCs needed for client meta information lookup

This patch implements the RPCs needed for the meta information
lookup during connection init. New tests added to cover the RPC
code paths. HBASE-23305 builds on this to implement the client
side logic.

Fixed a bunch of checkstyle nits around the places the patch
touches.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 4f8fbba0c0)
2020-01-28 16:45:49 -08:00
Josh Elser 427e367d95 HBASE-23722 Real user might be null in non-proxy-user case
Closes #1085

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-23 16:07:00 -05:00
Josh Elser f8a28c3ea6 HBASE-23709 Unwrap the real user to properly dispatch proxy-user auth'n
REST and Thrift servers started failing because the check in
BuiltinProviderSelector wasn't checking the "real" user for kerberos
credentials. This resulted in the KerberosAuthnProvider not being
invoked when it should have been.

Closes #1080

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-01-22 10:13:23 +01:00
Michael Stack ac852c2b66
HBASE-23055 Alter hbase:meta (#1043)
Make hbase:meta region schema dynamic.

Patch has been under development a good while and its focus has changed
a few times so its bloated with fixup from older versions.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
 Shut down access to internals and removed unused methods.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
 Cleanup/refactor section on replica-handling.

M hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
 Get hbase:meta schema from filesystem rather than from hard-coding.
2020-01-21 13:17:27 -08:00
Peter Somogyi ac5ac1dcb3 HBASE-23347 Allow custom authentication methods for RPCs; addendum (#1060)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-21 18:48:04 +01:00
Nick Dimiduk 74bfe023e5
HBASE-23653 Expose content of meta table in web ui (#1021)
Adds a display of the content of 'hbase:meta' to the Master's
table.jsp, when that table is selected. Supports basic pagination,
filtering, &c.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-01-16 08:47:00 -08:00
Josh Elser bef1eb33f4 HBASE-23347 Allow custom authentication methods for RPCs
Decouple the HBase internals such that someone can implement
their own SASL-based authentication mechanism and plug it into
HBase RegionServers/Masters.

Comes with a design doc in dev-support/design-docs and an example in
hbase-examples known as "Shade" which uses a flat-password file
for authenticating users.

Closes #884

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-01-16 11:04:08 -05:00
stack da782e4233 Revert "HBASE-23055 Alter hbase:meta"
This reverts commit d64b0e3612.
2020-01-11 09:21:15 -08:00
belugabehr 76a6fe6a8b HBASE-23628: Remove Apache Commons Digest Base64 (#977)
Signed-off-by: stack <stack@apache.org>
(cherry picked from commit 280b94434c)
2020-01-10 15:45:59 -06:00
stack d64b0e3612 HBASE-23055 Alter hbase:meta
Make it so hbase:meta can be altered. TableState for hbase:meta
    was hardcoded ENABLED. Make it dynamic. State is now kept in
    current active Master. It is transient so falls back to default
    if Master crashes. Add to registry a getMetaTableState
    which reads mirrored state from zookeeper (NOT from Master and
    defaults ENABLED if no implementation or error fetching state).
    hbase:meta schema will be bootstrapped from the filesystem.
    Changes to filesystem schema are atomic so we should be ok if
    Master fails mid-edit (TBD). Undoes a bunch of guards that
    prevented our being able to edit hbase:meta.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     Add fetching of hbase:meta table state from registry. Adds cache of
     tablestates w/ a ttl of 1 second (adjustable).

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
      Add querying registry for hbase:meta table state.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
      Add querying of mirrored table state for hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta. Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-01-09 09:52:21 -08:00
Nick Dimiduk f98085e44e HBASE-23333 Include Call.toShortString() in sendCall exceptions 2020-01-03 12:23:37 -08:00
stack c44a5c47dd HBASE-23369 Auto-close 'unknown' Regions reported as OPEN on RegionServers
Master force-closes unknown/incorrect Regions OPEN on RS

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Added a note and small refactor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Fix an NPE when CJ ran.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Minor clean up of log message; make it clearer.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
 Make it so closeRegionSilentlyAndWait can be used w/o timeout.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 If a RegionServer Report notes a Region is OPEN and the Master does not
 know of said Region, close it (We used to crash out the RegionServer)

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateNode.java
 Minor tweak of toString -- label should be state, not rit (confusing).

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
 Doc.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/TransitRegionStateProcedure.java
 Add region name to exception.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/HBCKServerCrashProcedure.java
 Be more careful about which Regions we queue up for reassign. This
 procedure is run by the operator so could happen at any time. We
 will likely be running this when Master has some accounting of
 cluster members so check its answers for what Regions were on
 server before running.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 Doc and we were misrepresenting the case where a Region as not in RIT
 when we got CLOSE -- we were reporting it as though it was already
 trying to CLOSE.
2020-01-03 10:21:55 -08:00
stack 1b366719a0 HBASE-23596 HBCKServerCrashProcedure can double assign
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

Change its behavior so it will only look in hbase:meta
if the call to the super class turns up zero references.
Only then will it search hbase:meta for references to
'Unknown Servers'. Normal operation where we read Master
context is usual and sufficient. The scan of hbase:meta
is only for case where Master state has been corrupted
and we need to clear out 'Unknown Servers'.
2020-01-02 14:33:24 -08:00
Viraj Jasani 17652a7b32
HBASE-23590 : Update maxStoreFileRefCount to maxCompactedStoreFileRefCount for auto region recovery based on old reader references
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-01-01 22:50:37 +05:30
Duo Zhang 5cae75e124 HBASE-23326 Implement a ProcedureStore which stores procedures in a HRegion (#941)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-12-25 12:21:26 +08:00
Ankit Singhal 6e6c7b3c2d HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-12-22 20:13:50 -08:00
XinSun c9ab545f54 HBASE-23376 NPE happens while replica region is moving (#906)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-19 15:11:57 +08:00
Bharath Vissapragada a3fcc8badb HBASE-23575 Remove dead code in AsyncRegistry (#940)
Removes a bunch of dead code and fixes some checkstyle nits.

(cherry picked from commit efa4fe901a)
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-12-16 10:25:09 -06:00
Junegunn Choi dcd02d1236
HBASE-23582 Unbalanced braces in string representation of table descriptor
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2019-12-16 10:37:23 +01:00
Michael Stack 18e3b48eb3 HBASE-23554 Encoded regionname to regionname utility (#923)
Adds shell command regioninfo:

  hbase(main):001:0>  regioninfo '0e6aa5c19ae2b2627649dc7708ce27d0'
  {ENCODED => 0e6aa5c19ae2b2627649dc7708ce27d0, NAME => 'TestTable,,1575941375972.0e6aa5c19ae2b2627649dc7708ce27d0.', STARTKEY => '', ENDKEY => '00000000000000000000299441'}
  Took 0.4737 seconds

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-11 08:57:24 -08:00
Nick Dimiduk 967f9d4e94 HBASE-23373 Log `RetriesExhaustedException` context with full time precision (#903)
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-12-06 17:03:59 -08:00
ddupg 997684f24d HBASE-23345 Table need to replication unless all of cfs are excluded
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-12-04 09:19:38 +08:00
Wellington Ramos Chevreuil 70bbc38aae HBASE-23313 [hbck2] setRegionState should update Master in-memory sta… (#864)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-11-27 12:15:11 +00:00
bsglz 11d0925814 HBASE-23334 The table-lock node of zk is not needed since HBASE-16786 (#873)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2019-11-25 19:51:04 +08:00
Michael Stack 12b2e1fd52
HBASE-23328 info:regioninfo goes wrong when region replicas enabled (#863)
Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
Signed-off-by: Guangxu Cheng <guangxucheng@gmail.com>
2019-11-21 22:57:22 -08:00
stack 70771b603e HBASE-23315 Miscellaneous HBCK Report page cleanup
* Add a bit of javadoc around SerialReplicationChecker.
 * Miniscule edit to the profiler jsp page and then a bit of doc on how to make it work that might help.
 * Add some detail if NPE getting BitSetNode to help w/ debug.
 * Change HbckChore to log region names instead of encoded names; helps doing diagnostics; can take region name and query in shell to find out all about the region according to hbase:meta.
 * Add some fix-it help inline in the HBCK Report page – how to fix.
 * Add counts in procedures page so can see if making progress; move listing of WALs to end of the page.
2019-11-19 07:33:13 -08:00
stack 44c8b58cec HBASE-23282 HBCKServerCrashProcedure for 'Unknown Servers'
Have the existing scheduleRecoveries launch a new HBCKSCP
instead of SCP. It gets regions to recover from Master
in-memory context AND from a scan of hbase:meta. This
new HBCKSCP is For processing 'Unknown Servers', servers that
are 'dead' and purged but still have references in
hbase:meta. Rare occurance but needs tooling to address.
Later have catalogjanitor take care of these deviations
between Master in-memory and hbase:meta content (usually
because of overdriven cluster with failed RPCs to hbase:meta,
etc)

Changed expireServers in ServerManager so could pass in
custom reaction to expired server.... This is how we
run our custom HBCKSCP while keeping all other aspects
of expiring services (rather than try replicate it
externally).
2019-11-18 14:26:11 -08:00
Udai Bhan Kashyap ab63bde013 HBASE-22969 A new binary component comparator(BinaryComponentComparator) to perform comparison of arbitrary length and position (#829)
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2019-11-18 13:30:08 +01:00
Josh Elser 46a18833a0 HBASE-23082 Backport of low latency space quotas for hbase snapshots
Includes the following, incorporating HBASE-20439 and HBASE-20440, too.

1)
HBASE-18133 Decrease quota reaction latency by HBase

Certain operations in HBase are known to directly affect
the utilization of tables on HDFS. When these actions
occur, we can circumvent the normal path and notify the
Master directly. This results in a much faster response to
changes in HDFS usage.

This requires FS scanning by the RS to be decoupled from
the reporting of sizes to the Master. An API inside each
RS is made so that any operation can hook into this call
in the face of other operations (e.g. compaction, flush,
bulk load).

2)
HBASE-18135 Implement mechanism for RegionServers to report file archival for space quotas

This de-couples the snapshot size calculation from the
SpaceQuotaObserverChore into another API which both the periodically
invoked Master chore and the Master service endpoint can invoke. This
allows for multiple sources of snapshot size to reported (from the
multiple sources we have in HBase).

When a file is archived, snapshot sizes can be more quickly realized and
the Master can still perform periodical computations of the total
snapshot size to account for any delayed/missing/lost file archival RPCs.

3)
HBASE-20531 RS may throw NPE when close meta regions in shutdown procedure.
2019-11-04 16:54:18 -05:00
Wellington Chevreuil 18f1aacae0 HBASE-23238 Additional test and checks for null references on ScannerCallableWithReplicas (#780)
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 577db5d7e5)
2019-11-01 10:32:12 +00:00
Wellington Chevreuil d2a027d8be HBASE-23136 PartionedMobFileCompactor bulkloaded files shouldn't get replicated (addressing buklload replication related issue raised in HBASE-22380)
Signed-off-by: Josh Elser <elserj@apache.org>
(cherry picked from commit 4d414020bb, then resolved conflicts)
2019-10-29 11:14:32 +00:00
Viraj Jasani 33e8156ebc HBASE-22460 : Reopen regions with very high Store Ref Counts (#750)
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2019-10-24 21:20:24 +05:30
Duo Zhang f19b8096f1 HBASE-23193 ConnectionImplementation.isTableAvailable can not deal with meta table on branch-2.x 2019-10-24 10:51:50 +08:00
meiyi d841245115 HBASE-23170 Admin#getRegionServers use ClusterMetrics.Option.SERVERS_NAME (#721) 2019-10-18 10:09:42 +08:00
Guanghao Zhang 18ff188d9d HBASE-23155 May NPE when concurrent AsyncNonMetaRegionLocator#updateCachedLocationOnError (#718) 2019-10-15 11:34:12 +08:00
Karthik Palanisamy bebf251179 HBASE-23152 Compaction_switch does not work by RegionServer name (#713)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 Signed-off-by: stack <stack@apache.org>
2019-10-12 09:09:44 -07:00
Viraj Jasani 44a7e8b3f1 HBASE-23093 Avoid Optional Anti-Pattern where possible (#673)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-11 17:47:05 +08:00
Viraj Jasani 15ec3c458c HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command (#611)
Signed-off-by: huzheng <openinx@gmail.com>
2019-10-08 09:25:57 +08:00
Istvan Toth e3078aa5aa HBASE-23119 ArrayIndexOutOfBoundsException in PrivateCellUtil#qualifierStartsWith (#688)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-10-04 15:14:30 +02:00
stack 61bc57f525 Revert " HBASE-23055 Alter hbase:meta (#655)"
"Too radical for branch-2"

This reverts commit ff217d4269.
2019-10-01 08:34:09 -07:00
Michael Stack ff217d4269
HBASE-23055 Alter hbase:meta (#655)
Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.


   Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2019-09-30 09:27:41 -07:00
Wellington Ramos Chevreuil 38c8bd3731
HBASE-22380 break circle replication when doing bulkload (#494)
Signed-off-by:  stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Norbert Kalmar <nkalmar@cloudera.com>
2019-09-21 14:39:06 +01:00
Stig Døssing 047bbfd4c2 HBase-22027: Split non-MR related parts of TokenUtil off into a ClientTokenUtil, and move ClientTokenUtil to hbase-client (#649) 2019-09-20 17:38:41 -07:00
Peter Somogyi ffec1500f6 Revert "HBASE-22700 refactor isMetaClearingException (#578)"
This reverts commit 5ccab83d6c.
2019-09-19 13:22:12 +02:00
Guanghao Zhang fb82c8ecb0 HBASE-23044 CatalogJanitor#cleanMergeQualifier may clean wrong parent regions (#637)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 08:10:26 +08:00
Viraj Jasani faed3e457d HBASE-23024 Replace initcause with Constructor arg (#627)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-17 13:40:07 +02:00
johnhomsea 5ccab83d6c HBASE-22700 refactor isMetaClearingException (#578) 2019-09-16 10:31:52 -07:00
Viraj Jasani 63568854b6 HBASE-22760 : Pause/Resume/Query Snapshot Auto Cleanup Activity (#618) 2019-09-13 10:37:22 +05:30
Guanghao Zhang 9f31d744f6 HBASE-23014 Should not show split parent regions in hbck report UI (#609) 2019-09-11 10:13:00 -07:00
stack 5f71c487e7 HBASE-22796 [HBCK2] Add fix of overlaps to fixMeta hbck Service
Signed-off-by: Sakthi <sakthi@apache.org>
2019-09-10 17:05:25 -07:00
Duo Zhang 1568e05507 HBASE-22963 Netty ByteBuf leak in rpc client implementation (#577)
Signed-off-by: Michael Stack <stack@apache.org>
2019-09-08 22:10:18 +08:00
stack bfa19f3a5a HBASE-22970 split parents show as overlaps in the HBCK Report
Check if overlap is split parent.

Cleaned up the HBCK Report page too with some notes that it is made of
two reports; have the two sections display the same.
2019-09-03 14:50:08 -07:00
meiyi b0456f3297 HBASE-22878 Show table throttle quotas in table jsp (#510) 2019-09-03 09:53:44 +08:00
ravowlga123 34eab1e3dd
HBASE-22886 Code Coverage Improvement: Create Unit Tests for class ConnectionId
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-09-02 22:39:10 +02:00
chenxu14 f00f56fd63 HBASE-22905 Avoid temp ByteBuffer allocation in (#538)
BlockingRpcConnection#writeRequest
2019-09-01 22:02:51 -07:00
meiyi 41fddd411e HBASE-22945 Show quota infos in master UI (#560)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-01 22:43:29 +08:00
syedmurtazahassan f088dd5b02 HBASE-22766 Code Coverage Improvement: Create Unit Tests for ResultStatsUtil (#520)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-08-31 22:40:38 +08:00
linkaline 84b4a61e88 HBASE-22881 Fix non-daemon threads in hbase server implementation (#512) (#536)
Signed-off-by: stack <stack@apache.org>
2019-08-28 09:48:27 -07:00
wuguihu b5818a7f3f
HBASE-21400 correct spelling error of 'initilize' in comment
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-23 21:32:56 +02:00
Jan Hentschel 6bc78d3b70
HBASE-22844 Fixed Checkstyle violations in client snapshot exceptions
Signed-off-by: stack <stack@apache.org>
2019-08-21 10:31:28 +02:00
Itsuki Toyota dfe7e27e25 HBASE-22833 MultiRowRangeFilter should provide a method for creating… (#493)
* HBASE-22833: MultiRowRangeFilter should provide a method for creating a filter which is functionally equivalent to multiple prefix filters

* Delete superfluous comments

* Add description for MultiRowRangeFilter constructor

* Add null check for rowKeyPrefixes

* Fix checkstyle

Signed-off-by: huzheng <openinx@gmail.com>
2019-08-16 10:06:44 +08:00
Sakthi ba55315aab HBASE-22845 Revert MetaTableAccessor#makePutFromTableState access to public (#489)
HBCK2 is dependent on it
2019-08-13 11:27:28 -07:00
stack 665e0ead27 Revert "HBASE-22699 refactor isMetaClearingException (#436)"
This reverts commit cc6e157112.
2019-08-12 11:19:42 -07:00
johnhomsea cc6e157112 HBASE-22699 refactor isMetaClearingException (#436)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-08-12 16:12:58 +08:00
syedmurtazahassan c5b29265a3
HBASE-22725 Remove all remaining Javadoc warnings
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-11 22:02:53 +02:00
stack 225c6cd5bc HBASE-22777 Add a multi-region merge (for fixing overlaps)
Makes MergeTableRegionsProcedure do more than just two regions at a
time. Compatible as MTRP was done considering one day it'd do more than
two at a time.

Changes hardcoded assumption that merge parent regions are named
mergeA and mergeB in a column on the resultant region. Instead
can have N columns on the merged region, one for each parent
merged. Column qualifiers all being with 'merge'.

Most of code below is undoing the assumption that there are two
parents on a merge only.
2019-08-07 16:58:06 -07:00
stack c03bf73e24 HBASE-22771 [HBCK2] fixMeta method and server-side support; AMENDMENT remove mistaken .rej commit 2019-08-05 22:39:54 -07:00
stack 2bc9526080 HBASE-22771 [HBCK2] fixMeta method and server-side support
This is a first cut at this patch. Implements hold fixing only
currently.

Add a fixMeta method to Hbck Interface.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Bug fix. If hole is on end of last table, I wasn't seeing it.

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
 Add a general meta fixer class. Explains up top why this stuff doesn't
 belong inside MetaTableAccessor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 Break out the filesystem messing so don't have to copy it nor do more
 than is needed doing fixup for Region holes.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java
 Change behavious slightly. If directory exists, don't fail as we did
 but try and keep going and create .regioninfo file if missing (or
 overwrite if in place). This should make it idempotent. Can rerun
 command. Lets see if any repercussions in test suite.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
 Add test.

Signed-off-by: Zheng Hu <openinx@gmail.com>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-08-05 22:33:20 -07:00
Jan Hentschel d7af4f9802
HBASE-22785 Fixed Checkstyle issues in exceptions and enhanced Javadoc
Signed-off-by: stack <stack@apache.org>
2019-08-05 15:08:24 +02:00
Jan Hentschel ce709de6f4
HBASE-22786 Fixed Checkstyle issues in tests in hbase-client
Signed-off-by: stack <stack@apache.org>
2019-08-05 11:54:11 +02:00
Wellington Chevreuil 5cd0433dc9 HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore 2019-08-01 16:56:57 +01:00
Guanghao Zhang 6a20872e01 HBASE-22737 Add a new admin method and shell cmd to trigger the hbck chore to run (#425)
Signed-off-by: stack <stack@apache.org>
2019-08-01 09:02:26 +08:00
Thanh NM eb700db44c HBASE-22714 - Fix opertationTimeout typo (#392)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-07-30 21:59:22 +08:00
openinx 300f906200 HBASE-22758 Remove the unneccesary info cf deletion in DeleteTableProcedure#deleteFromMeta (#424) 2019-07-30 21:37:52 +08:00
stack 4565e9f43b HBASE-22723 Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta
Refactor of CatalogJanitor so it generates a
Report on the state of hbase:meta when it runs. Also
refactor so CJ runs even if RIT (previous it would
punt on running if RIT) so it can generate a 'Report'
on the interval regardless. If RIT, it just doesn't
go on to do the merge/split GC as it used to.

If report finds an issue, dump as a WARN message
to the master log.

Follow-on is to make the Report actionable/available
for the Master to pull when it goes to draw the hbck
UI page (could also consider shipping the Report as
part of ClusterMetrics?)

Adds new, fatter Visitor to CJ, one that generates
Report on each run keeping around more findings as
it runs.

Moved some methods around so class reads better;
previous methods were randomly ordered in the class.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Make a few handy methods public.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add utility as defaults on the Inteface; i.e. is this the first region
 in table, is it last, does a passed region come next, or does passed
 region overlap this region (added tests for this new stuff).

M hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
 Bugfix... handle case where buffer passed is null.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Lots of change, reorg., but mostly adding consistency checking
 to the visitor used scanning hbase:meta on a period and the
 generation of a Report on what the scan has found traversing
 hbase:meta. Added a main so could try the CatalogJanitor against
 a running cluster.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitorCluster.java
 Fat ugly test for CatalogJanitor consistency checking.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
 Add tests for new functionality in RI.

M hbase-shell/src/main/ruby/hbase/table.rb
 Bug fix for case where meta has a null regioninfo; scan was aborting.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-07-29 16:07:10 -07:00
Gabriel 5ab3f76a16 HBASE-22750 Correct @throws in comment.(In class org.apache.hadoop.hbase.client.RetryingCallerInterceptor) (#418) 2019-07-26 21:35:52 -07:00
Viraj Jasani 3a72ccefbd
HBASE-22648 Snapshot TTL (#371)
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotDescriptionUtils.java
2019-07-22 15:08:20 -07:00
Guanghao 4c89354f3b HBASE-22673 Avoid to expose protobuf stuff in Hbck interface
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-07-19 08:10:38 +08:00
syedmurtazahassan b6e66699a9
HBASE-22571 Javadoc Warnings related to @return tag
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-07-13 13:48:45 +02:00
Bing Xiao 12c933ac51 HBASE-21751 WAL creation fails during region open may cause region assign forever fail
Signed-off-by: Allan Yang <allan163@apache.org>
    Signed-off-by: Michael Stack <stack@apache.org>
2019-07-01 22:19:42 -07:00
syedmurtazahassan 8fa7800389 HBASE-22572 Javadoc Warnings: @link reference not found (#306)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: stack <stack@apache.org>
2019-07-01 21:18:11 -07:00
zhangduo 7cd1caa1ec HBASE-22477 Throwing exception when meta region is not in OPEN state in client registry may crash a master 2019-07-02 10:36:49 +08:00
huzheng c19592a971 HBASE-21916 Abstract an ByteBuffAllocator to allocate/free ByteBuffer in ByteBufferPool 2019-06-25 21:12:37 +08:00
Jan Hentschel 6cff25cc23
HBASE-22344 Documented the deprecation of public and limited private APIs (#208)
Signed-off-by: stack <stack@apache.org>
2019-06-17 22:29:51 +02:00
Andrew Purtell d3fba8c0c5
HBASE-22530 The metrics of store files count of region are returned to clients incorrectly (Eungsop Yoo)
Signed-off-by: Xu Cang <xucang@apache.org>
2019-06-13 17:45:28 -07:00
Duo Zhang d2eee1400d HBASE-22569 Should treat null consistency as Consistency.STRONG in ConnectionUtils.timelineConsistentRead 2019-06-13 08:20:50 +08:00
meiyi 17ea99c83d HBASE-22513 Admin#getQuota does not work correctly if exceedThrottleQuota is set 2019-06-03 17:51:28 +08:00
Andrew Purtell d06c3af980
HBASE-22459 Expose store reader reference count (#248) 2019-05-30 15:08:32 -07:00
Andrew Purtell 2c55bd9344
HBASE-22449 https everywhere in Maven metadata (#247) 2019-05-21 12:38:42 -07:00
zhangduo 6a28e76f7e HBASE-22442 Nightly build is failing with hadoop 3.x 2019-05-19 07:56:02 +08:00
Andrew Purtell cc0b70db1f
HBASE-22377 Provide API to check the existence of a namespace which does not require ADMIN permissions (#225)
Signed-off-by: Xu Cang <xucang@apache.org>
2019-05-14 12:51:51 -07:00
Duo Zhang 6bd6a953a7 HBASE-22399 Change default hadoop-two.version to 2.8.x and remove the 2.7.x hadoop checks 2019-05-14 16:45:32 +08:00
zhangduo 912bbab1bf HBASE-21658 Addendum fix infinite wait when there are no meta locations yet 2019-05-14 08:13:02 +08:00
zhangduo ee1ebac4c2 HBASE-21658 Should get the meta replica number from zk instead of config at client side 2019-05-11 09:31:46 +08:00
zhangduo f1a00b9d57 HBASE-22236 AsyncNonMetaRegionLocator should not cache HRegionLocation with null location 2019-04-30 17:18:56 +08:00
Duo Zhang 75055c831e HBASE-22325 AsyncRpcRetryingCaller will not schedule retry if we hit a NotServingRegionException but there is no TableName provided 2019-04-30 15:20:09 +08:00
Duo Zhang 37c0ab30c7 HBASE-22322 Use special pause for CallQueueTooBigException 2019-04-30 10:07:47 +08:00
Sakthi c3e490f75c HBASE-22086: Space Quota issue: Deleting snapshot doesn't update the usage of table
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-04-29 14:44:51 +08:00
Duo Zhang 940f374a07 HBASE-22317 Support reading from meta replicas 2019-04-28 15:59:11 +08:00
Sean Busbey 4862a596ef HBASE-22083 move eclipse settings into a profile.
Signed-off-by: stack <stack@apache.org>

 Conflicts:
	hbase-backup/pom.xml
	hbase-hadoop-compat/pom.xml
	hbase-protocol/pom.xml
2019-04-25 14:38:38 -05:00
Duo Zhang a3e36cb545 HBASE-22307 Addendum deprecated PreemptiveFastFailException 2019-04-25 16:57:39 +08:00
Yu Li 54b944a10f HBASE-22283 Print row and table information when failed to get region location 2019-04-24 15:52:52 +08:00
李小保 1bd5b5cf7b HBASE-22250 The same constants used in many places should be placed in constant classes
Signed-off-by: stack <stack@apache.org>
2019-04-23 21:24:46 -07:00
zouxiangwen 4138fce370 HBASE-22292 PreemptiveFastFailInterceptor clean repeatedFailuresMap issue
Signed-off-by: stack <stack@apache.org>
2019-04-23 08:02:48 -07:00
Igor Rudenko 7046996d64 HBASE-22047 LeaseException in Scan should be retried
Signed-off-by: stack <stack@apache.org>
2019-04-23 06:26:33 -07:00
Jan Hentschel c40e6e2339 HBASE-22231 Removed unused and '*' import 2019-04-23 12:53:52 +02:00
zhangduo 8973885cf4 HBASE-22279 Add a getRegionLocator method in Table/AsyncTable interface 2019-04-22 18:39:53 +08:00
zhangduo 38c7ab59a2 HBASE-22282 Should deal with error in the callback of RawAsyncHBaseAdmin.splitRegion methods 2019-04-22 16:49:02 +08:00
zhangduo 3a1dc8ec82 HBASE-22278 RawAsyncHBaseAdmin should not use cached region location 2019-04-21 22:35:00 +08:00
zhangduo 393c5b9502 HBASE-22261 Make use of ClusterStatusListener for async client 2019-04-21 22:26:58 +08:00
zhangduo 65fb40d7ee HBASE-22267 Implement client push back for async client 2019-04-21 16:51:55 +08:00
zhangduo cbe4ec2c51 HBASE-22244 Make use of MetricsConnection in async client 2019-04-18 22:20:24 +08:00
Jan Hentschel b5b1f86dc7
HBASE-22187 Removed remaining usages of ClusterConnection.clearRegionCache 2019-04-12 09:59:52 +02:00
Josh Elser 0f155b9ec9 HBASE-22144 Correct MultiRowRangeFilter to work with reverse scans
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2019-04-11 17:41:24 -04:00
stack c7e92bbc0d HBASE-22155 Move 2.2.0 on to hbase-thirdparty-2.2.0 2019-04-09 09:39:31 -07:00
Jingyun Tian 8a6b8e81aa HBASE-21965 Fix failed split and merge transactions that have failed to roll back 2019-04-09 20:11:56 +08:00
meiyi 0e85571b39 HBASE-22117 Move hasPermission/checkPermissions from region server to master
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-04-09 17:06:09 +08:00
zhangduo 838e86952f HBASE-22178 Introduce a createTableAsync with TableDescriptor method in Admin 2019-04-07 21:19:25 +08:00
zhangduo b04b1ecc74 HBASE-22108 Avoid passing null in Admin methods
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-04-07 21:08:55 +08:00
zhangduo 0b46ac5db3 HBASE-22179 Fix RawAsyncHBaseAdmin.getCompactionState 2019-04-07 08:35:03 +08:00
zhangduo 448f26775c HBASE-22177 Do not recreate IOException in RawAsyncHBaseAdmin.adminCall 2019-04-07 08:33:49 +08:00
Duo Zhang e35ff3acb7 HBASE-22158 RawAsyncHBaseAdmin.getTableSplits should filter out none default replicas
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-04-05 20:33:54 +08:00
Duo Zhang 7154f86b4b HBASE-22157 Include the cause when constructing RestoreSnapshotException in restoreSnapshot
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-04-04 10:45:30 +08:00
zhangduo 6fad60232b HBASE-22135 AsyncAdmin will not refresh master address
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-04-01 09:24:00 +08:00
pingsutw b78c7a4f00 HBASE-22101 AsyncAdmin.isTableAvailable should not throw TableNotFoundException
Signed-off-by: zhangduo <zhangduo@apache.org>
2019-03-30 20:43:12 +08:00
zhangduo 4cf8e3b04c HBASE-22121 AsyncAdmin can not deal with non default meta replica
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-03-30 20:19:02 +08:00
Sakthi 37b2cab98a HBASE-22094: Throw TableNotFoundException if table not exists in AsyncAdmin.compact
Signed-off-by: zhangduo <zhangduo@apache.org>
2019-03-27 20:14:29 +08:00
meiyi 89dbab7c9f HBASE-21911 Move getUserPermissions from regionserver to master
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-03-27 11:22:57 +08:00
zhangduo bb52a6766a HBASE-22074 Should use procedure store to persist the state in reportRegionStateTransition 2019-03-26 18:14:17 +08:00
yaojingyi 1926b86a8c HBASE-21964 unset Quota by Throttle Type
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-03-26 09:46:01 +08:00
zhangduo 8fb8886e5c Revert "HBASE-21965 Fix failed split and merge transactions that have failed to roll back"
This reverts commit b2a4265992.
2019-03-24 13:50:39 +08:00
Jingyun Tian b2a4265992 HBASE-21965 Fix failed split and merge transactions that have failed to roll back 2019-03-22 17:12:34 +08:00
niuyulin ffcc003f85 HBASE-22065 Add listTableDescriptors(List<TableName>) method in AsyncAdmin
Signed-off-by: zhangduo <zhangduo@apache.org>
2019-03-21 22:36:10 +08:00
xujunhong c14f362bf8 HBASE-22063 Deprecated Admin.deleteSnapshot(byte[])
Signed-off-by: zhangduo <zhangduo@apache.org>
2019-03-21 22:25:27 +08:00
Rishabh Jain 3a0cc2a098 HBASE-22042 Missing @Override annotation for RawAsyncTableImpl.scan
Signed-off-by: zhangduo <zhangduo@apache.org>
2019-03-19 17:01:44 +08:00
meiyi 3317f03d41 HBASE-22015 UserPermission should be annotated as InterfaceAudience.Public
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-03-19 14:55:15 +08:00
zhangduo 2277ca422e HBASE-22040 Add mergeRegionsAsync with a List of region names method in AsyncAdmin
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-03-18 21:05:27 +08:00
Duo Zhang 40bf6c5069 HBASE-22039 Should add the synchronous parameter for the XXXSwitch method in AsyncAdmin
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-03-16 10:09:07 +08:00
Nihal Jain 04b78bb416 HBASE-20662 Increasing space quota on a violated table does not remove SpaceViolationPolicy.DISABLE enforcement
Signed-off-by: Josh Elser <elserj@apache.org>
2019-03-15 14:00:52 -04:00
zhangduo 31b3716e84 HBASE-22001 Polish the Admin interface
Signed-off-by: stack <stack@apache.org>
2019-03-11 14:43:12 +08:00
Norbert Kalmar 003095cbca HBASE-21416 - fix TestRegionInfoDisplay flaky test 2019-03-08 09:19:00 -08:00
zhangduo d084423b52 HBASE-22007 Add restoreSnapshot and cloneSnapshot with acl methods in AsyncAdmin
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-03-08 18:19:11 +08:00
johnhomsea 1667762937 HBASE-22000 Deprecated isTableAvailable with splitKeys
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-03-07 09:18:14 +08:00
zhangduo 21c7e880d4 HBASE-21983 Should track the scan metrics in AsyncScanSingleRegionRpcRetryingCaller if scan metrics is enabled 2019-03-03 19:08:28 +08:00
zhangduo 00e6208de0 HBASE-21978 Should close AsyncRegistry if we fail to get cluster id when creating AsyncConnection
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-03-03 08:19:31 +08:00
meiyi 0d882bbc2b HBASE-21974 Change Admin#grant/revoke parameter from UserPermission to user and Permission
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-03-02 23:47:56 +08:00
Syeda 5c03df5e50 HBASE-21487 Concurrent modify table ops can lead to unexpected results
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-03-02 13:07:35 +08:00
Duo Zhang 8677031707 HBASE-21976 Deal with RetryImmediatelyException for batching request
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-03-02 12:14:32 +08:00
Guanghao Zhang e412e036e1 HBASE-20724 Sometimes some compacted storefiles are still opened after region failover 2019-02-28 09:51:29 +08:00
meiyi ea0675eb96 HBASE-21820 Implement CLUSTER quota scope
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-27 18:24:03 +08:00
Duo Zhang 11624c63e3 HBASE-21961 Infinite loop in AsyncNonMetaRegionLocator if there is only one region and we tried to locate before a non empty row
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-27 17:55:09 +08:00
Duo Zhang 80830ee555 HBASE-21943 The usage of RegionLocations.mergeRegionLocations is wrong for async client
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-26 11:22:11 +08:00
Wellington Chevreuil a8af0e5894 HBASE-21505 - proposal for a more consistent report on status
Signed-off-by: Jingyun Tian <tianjy@apache.org>
2019-02-25 10:07:51 +08:00
zhangduo d7c5d73de0 HBASE-21944 Validate put for batch operation
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-24 20:20:16 +08:00
zhangduo d47d64c6e3 Revert "HBASE-21943 The usage of RegionLocations.mergeRegionLocations is wrong for async client"
This reverts commit 3ccc6fec6c.
2019-02-24 20:14:27 +08:00
zhangduo 1568aad302 HBASE-21945 Maintain the original order when sending batch request
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-24 17:56:16 +08:00
Duo Zhang 3ccc6fec6c HBASE-21943 The usage of RegionLocations.mergeRegionLocations is wrong for async client
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-24 17:34:29 +08:00
Guanghao Zhang 607ac735c4 HBASE-21922 BloomContext#sanityCheck may failed when use ROWPREFIX_DELIMITED bloom filter 2019-02-23 23:29:53 +08:00
meiyi 9e45752d6e HBASE-21783 Support exceed user/table/ns throttle quota if region server has available quota
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-22 17:43:40 +08:00
Duo Zhang 761aef6d9d HBASE-20587 Replace Jackson with shaded thirdparty gson
Signed-off-by: Michael Stack <stack@apache.org>
2019-02-22 16:40:45 +08:00
meiyi 61621a33ce HBASE-21938 Add a new ClusterMetrics.Option SERVERS_NAME to only return the live region servers's name without metrics
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-22 10:45:35 +08:00
zhangduo 164c250691 HBASE-21927 Always fail the locate request when error occur
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-02-20 19:48:32 +08:00
zhangduo f198311556 HBASE-21928 Deprecated HConstants.META_QOS
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-02-20 14:16:54 +08:00
zhangduo 4b36de3161 HBASE-21930 Deal with ScannerResetException when opening region scanner
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-02-20 11:04:15 +08:00
Nihal Jain f671703c7f HBASE-17942 Disable region splits and merges per table
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-19 07:20:34 +08:00
Duo Zhang 3efccfe510 HBASE-21907 Should set priority for rpc request
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-18 19:46:31 +08:00
zhangduo d88a87fd04 HBASE-21909 Validate the put instance before executing in AsyncTable.put method
Signed-off-by: Michael Stack <stack@apache.org>
2019-02-16 18:06:39 +08:00
Duo Zhang 48f2ef432b HBASE-21910 The nonce implementation is wrong for AsyncTable
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-15 20:24:48 +08:00
Duo Zhang dec5bf3087 HBASE-21900 Infinite loop in AsyncMetaRegionLocator if we can not get the location for meta
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-14 19:50:23 +08:00
Duo Zhang 33bdf09f77 HBASE-21888 Add a isClosed method to AsyncConnection
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-14 11:41:30 +08:00
zhangduo beb7d01092 HBASE-21859 Add clearRegionLocationCache method for AsyncConnection
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-13 10:07:43 +08:00
zhangduo 3759bf5309 HBASE-21862 IPCUtil.wrapException should keep the original exception types for all the connection exceptions
Signed-off-by: Michael Stack <stack@apache.org>
2019-02-10 09:37:34 +08:00
zhangduo 94c9aecea2 HBASE-21857 Do not need to check clusterKey if replicationEndpoint is provided when adding a peer
Signed-off-by: Xu Cang <xucang@apache.org>
2019-02-08 19:48:07 +08:00
zhangduo 54a8cc87c6 HBASE-21829 Use FutureUtils.addListener instead of calling whenComplete directly 2019-02-03 15:29:20 +08:00
zhangduo ed60532543 HBASE-21828 Make sure we do not return CompletionException when locating region 2019-02-02 21:14:09 +08:00
zhangduo e61c12c46f HBASE-21684 Throw DNRIOE when connection or rpc client is closed 2019-01-31 20:22:00 +08:00
Tommy Li 8699ec0fc8 HBASE-21775 ADDENDUM - fix TestAsyncProcess 2019-01-30 15:41:41 -08:00
meiyi ec9029f77e HBASE-21739 Move grant/revoke from regionserver to master
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-01-30 19:03:43 +08:00
wanghuan 69c331efed HBASE-21699 Fixed create table failed when using SPLITS_FILE => 'splits.txt'
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-01-30 15:57:21 +08:00
Guanghao Zhang 16665b6e93 HBASE-21799 Update branch-2 version to 2.3.0-SNAPSHOT 2019-01-29 21:53:21 +08:00
Duo Zhang e736d78362 HBASE-21792 Mark HTableMultiplexer as deprecated and remove it in 3.0.0
Signed-off-by: Michael Stack <stack@apache.org>
2019-01-29 09:08:26 +08:00
zhangduo f359d260d3 HBASE-21789 Rewrite MetaTableAccessor.multiMutate with Table.coprocessorService
Signed-off-by: Michael Stack <stack@apache.org>
2019-01-27 11:37:11 +08:00
Tommy Li 5f25985b98 HBASE-21775 The BufferedMutator doesn't ever refresh region location cache
Fix server location metadata refresh in multi-actions

Signed-off-by: stack <stack@apache.org>
2019-01-26 13:11:12 -08:00
Duo Zhang 1ac090b3f0 HBASE-21762 Move some methods in ClusterConnection to Connection
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-01-25 21:14:07 +08:00
meiyi 1e7f306e17 HBASE-21713 Support set region server throttle quota
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-01-25 15:47:32 +08:00
Wellington Chevreuil aae52f787c HBASE-20215 - Rename CollectionUtils to ConcurrentMapUtils 2019-01-24 20:55:55 -08:00
zhangduo de31d56c2d HBASE-21770 Should deal with meta table in HRegionLocator.getAllRegionLocations
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-01-25 11:05:29 +08:00
zhangduo 67ba050fce HBASE-21761 Align the methods in RegionLocator and AsyncTableRegionLocator
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-01-24 15:21:26 +08:00
xujunhong 7cebc0a74c HBASE-21715 set timeout instead of throwing Exception when calling ProcedureFuture.get in client side.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-01-24 14:55:33 +08:00
Kevin 0b81c63b1b HBASE-21716 Add toStringCustomizedValues to TableDescriptor
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-01-24 11:32:39 +08:00
Tommy Li 8a7dd2de24 HBASE-21720 : metric to measure how actions are distributed to servers within a MultiAction
Signed-off-by: Sergey Shelukhin <sershe@apache.org>
2019-01-23 17:53:59 -08:00
Duo Zhang 6c57050af6 HBASE-21753 Support getting the locations for all the replicas of a region 2019-01-23 18:10:01 +08:00
huzheng 4eba6b3656 HBASE-21750 Most of KeyValueUtil#length can be replaced by cell#getSerializedSize for better performance because the latter one has been optimized 2019-01-22 20:21:49 +08:00
huzheng 1443694f89 HBASE-21734 Some optimization in FilterListWithOR 2019-01-22 11:50:19 +08:00
Duo Zhang ff272e8683 HBASE-21726 Add getAllRegionLocations method to AsyncTableRegionLocator 2019-01-19 11:14:40 +08:00
zhangduo 07484db1ff HBASE-19695 Handle disabled table for async client
Signed-off-by: tianjingyun <tianjy1990@gmail.com>
2019-01-17 18:34:30 +08:00
Duo Zhang ebf4fe3bb9 HBASE-21710 Add quota related methods to the Admin interface
Signed-off-by: Michael Stack <stack@apache.org>
2019-01-16 21:10:27 +08:00
Duo Zhang 479b8bef6d HBASE-21732 Should call toUpperCase before using Enum.valueOf in some methods for ColumnFamilyDescriptor
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-01-16 20:46:29 +08:00
Peter Somogyi c353d93f69 HBASE-21727 Simplify documentation around client timeout
Signed-off-by: Michael Stack <stack@apache.org>
2019-01-16 11:41:40 +01:00
Allan Yang b82ef56267 HBASE-20854 Wrong retries number in RpcRetryingCaller's log message 2019-01-16 11:40:49 +01:00
huzheng fdfbd9a21c HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case 2019-01-16 09:47:27 +08:00
Sakthi dc50b570c6 HBASE-21225: Having RPC & Space quota on a table/Namespace doesn't allow space quota to be removed using 'NONE'
Signed-off-by: Josh Elser <elserj@apache.org>
2019-01-14 18:01:33 -07:00
zhangduo e7113edddf HBASE-21705 Should treat meta table specially for some methods in AsyncAdmin 2019-01-13 13:41:01 +08:00
Duo Zhang a09dffd106 HBASE-21663 Add replica scan support 2019-01-11 14:19:17 +08:00
Duo Zhang 8b9bfa0df5 HBASE-21580 Support getting Hbck instance from AsyncConnection 2019-01-11 11:26:48 +08:00
meiyi 14fb3af5fb HBASE-21694 Add append_peer_exclude_tableCFs and remove_peer_exclude_tableCFs shell commands
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-01-09 22:22:47 +08:00
Sergey Shelukhin dfa86404dd HBASE-21614 RIT recovery with ServerCrashProcedure doesn't account for all regions
Signed-off-by: zhangduo <zhangduo@apache.org>
2019-01-09 20:03:30 +08:00
zhangduo d5fff9c256 HBASE-21682 Support getting from specific replica 2019-01-08 10:04:08 +08:00
meiyi c28e03e5df HBASE-21159 Add shell command to switch throttle on or off
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-01-08 09:39:22 +08:00
zhangduo 77ca660389 HBASE-17356 Add replica get support 2019-01-03 08:38:53 +08:00
Sakthi b396051153 HBASE-21631: list_quotas should print human readable values for LIMIT
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-12-22 22:05:32 +08:00
openinx 406f44219f HBASE-21620 Problem in scan query when using more than one column prefix filter in some cases
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Allan Yang <allan163@apache.org>
2018-12-21 15:21:34 -08:00
Guanghao Zhang 1dd137b148 HBASE-21618 Scan with the same startRow(inclusive=true) and stopRow(inclusive=false) returns one result 2018-12-21 09:50:23 +08:00
meiyi c494e9ff20 HBASE-21578 Fix wrong throttling exception for capacity unit
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-12-14 18:22:34 +08:00
zhangduo 9981e3f843 HBASE-21570 Add write buffer periodic flush support for AsyncBufferedMutator 2018-12-11 15:29:39 +08:00
Sakthi af72916c4c HBASE-21453 Convert ReadOnlyZKClient to DEBUG instead of INFO 2018-12-11 08:18:39 +01:00
Josh Elser 68bb66e35e HBASE-21524 Fix logging in ConnectionImplementation.isTableAvailable()
Signed-off-by: Michael Stack <stack@apache.org>
2018-11-29 11:28:19 -05:00
meiyi d590d6e472 HBASE-21034 Add new throttle type: read/write capacity unit
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-11-21 09:50:07 +08:00
Reid Chan 6ab0fbb1f1 HBASE-21255 [acl] Refactor TablePermission into three classes (Global, Namespace, Table)
Signed-off-by: Michael Stack <stack@apache.org>
2018-11-15 11:45:24 +08:00
Ben Lau 13b68abb58
HBASE-21439 RegionLoads aren't being used in RegionLoad cost functions
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
2018-11-08 14:08:52 -08:00
tianjingyun 857abfb082
HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-29 20:56:11 -07:00
Guanghao Zhang f71768308c HBASE-21385 HTable.delete request use rpc call directly instead of AsyncProcess
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-25 16:22:18 -07:00
jingyuntian 4a609db30c
HBASE-21269 Forward-port HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign 2018-10-18 06:22:22 -07:00
Sahil Aggarwal ad6ca7c95a
HBASE-20716: Changes the bytes[] conversion done in Bytes and ByteBufferUtils. Instead of doing check unsafe_aligned available everytime, choose the best converter at startup. 2018-10-17 21:04:09 -07:00
Michael Stack 6125872f48 HBASE-21303 [shell] clear_deadservers with no args fails 2018-10-12 11:19:40 -07:00
Nihal Jain 3d1a2dbe68
HBASE-21196 HTableMultiplexer clears the meta cache after every put operation
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-28 16:35:53 -07:00
Michael Stack 2836f75625
HBASE-21223 [amv2] Remove abort_procedure from shell
Signed-off-by: Balazs Meszaros <balazs.meszaros@cloudera.com>
2018-09-25 11:28:54 -07:00
Guangxu Cheng fd68e7593e
HBASE-20636 Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
2018-09-21 16:06:34 -07:00
Umesh Agashe 899fddb4e7
HBASE-21023 Added bypassProcedure() API to HbckService 2018-09-19 15:17:52 -07:00
Michael Stack 8876f12c0c
HBASE-21156 [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
Adds 'raw' assigns and unassigns methods to Hbck Service.

Fixes HbckService so it works when cluster is Kerberized.
2018-09-19 09:47:32 -07:00
Guangxu Cheng e1548d3349 HBASE-21179 Fix the number of actions in responseTooSlow log 2018-09-12 10:45:55 +08:00
Guangxu Cheng 1a7dba44ce HBASE-21158 Empty qualifier cell is always returned when using QualifierFilter 2018-09-10 21:51:48 +08:00
Toshihiro Suzuki 49501cd5b5 HBASE-21052 After restoring a snapshot, table.jsp page for the table gets stuck 2018-09-09 10:25:10 +09:00
Reid Chan 0409c54ba7 HBASE-21129 Clean up duplicate codes in #equals and #hashCode methods of Filter
Signed-off-by Ted Yu <yuzhihong@gmail.com>
2018-09-04 11:15:57 +08:00
Toshihiro Suzuki 9b7c530b6c HBASE-21084 When cloning a snapshot including a split parent region, the split parent region of the cloned table will be online - addendum 2018-08-28 15:34:52 +09:00
Umesh Agashe cdf5cfa396 HBASE-20941 Created and implemented HbckService in master
Added API setTableStateInMeta() to update table state only in Meta. This will be used by hbck2 tool.
2018-08-27 12:10:33 -07:00
subrat.mishra 25e988dd43 HBASE-21030: Changed javadoc for append operation.
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2018-08-26 23:45:24 +09:00
zhangduo f533f01a3a HBASE-20881 Introduce a region transition procedure to handle all the state transition for a region 2018-08-26 18:08:06 +08:00
liubangchen a7a281a644 HBASE-19008 Add missing equals or hashCode method(s) to stock Filter implementations
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-08-24 00:39:35 +08:00
jingyuntian bee8566f2c HBASE-20985 add two attributes when we do normalization
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-13 16:49:37 +08:00
Sakthi b2cefab0e1 HBASE-20885 Removed entry for RPC quota from hbase:quota when RPC quota is removed
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-08-03 11:06:56 -04:00
Michael Stack fc0c4660fa HBASE-20989 Minor, miscellaneous logging fixes
Signed-off-by: Zach York <zyork@amazon.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-08-01 11:20:26 -07:00
Reid Chan e14b60a539 HBASE-20886 [Auth] Support keytab login in hbase client
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-07-31 11:10:34 +08:00
Allan Yang ac4b789f5f HBASE-20867 RS may get killed while master restarts 2018-07-25 18:13:50 +08:00