Commit Graph

17851 Commits

Author SHA1 Message Date
stack 2ca0a105bc HBASE-24043 [Flakey Tests] TestAsyncRegionAdminApi, TestRegionMergeTransactionOnCluster fixes and debug
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java
 Edit of log about archiving that shows in middle of a table create;
 try to make it less disorientating.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
 Loosen assert. Compaction may have produced a single file only. Allow
 for this.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableGetMultiThreaded.java
 Make this test less furious given it is inline w/ a bunch of unit
 tests.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
 Add debug

hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestQuotaObserverChoreRegionReports.java
 Add wait on quota table to show up before moving forward; otherwise,
 attempt at quota setting fails.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
 Debug

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
 Remove asserts that expected regions to still have a presence in fs
 after merge when a catalogjanitor may have cleaned up parent dirs.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java
 Catch exception on way out and log it rather than let it fail test.

hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSnapshotScannerHDFSAclController.java
 Wait on acl table before proceeding.
2020-03-24 14:52:24 -07:00
Esteban Gutierrez 9a212ee487
HBASE-24041 [regression] Increase RESTServer buffer size back to 64k (#1339)
* HBASE-24041 [regression] Increase RESTServer buffer size back to 64k

* HBASE-24041 [regression] Increase RESTServer buffer size back to 64k
2020-03-24 15:12:02 -05:00
Nick Dimiduk aa53493ae8 HBASE-24002 shadedjars check does not propagate --hadoop-profile
This implementation is almost surely incorrect. Personality
initialization parses the `--hadoop-profile` argument and sets
`HADOOP_PROFILE`. That value is then used to build an `extras` value
that is passed  along to module initialization. I'm guessing that the
`extras` value need to be honored down in the shadedjars module. I'm
not clear on how to make that work (need to study the interfaces at
play here), so taking the more ham-handed approach of referring to
`HADOOP_PROFILE`. I'm not sure if this will even work, or if it will
only work because the `foo_yetus.sh` scripts happen to use a variable
of the same name.

Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: stack <stack@apache.org>
2020-03-24 09:31:40 -07:00
Reid Chan c3a704ea22
HBASE-24032 [RSGroup] Assign created tables to respective rsgroup automatically instead of manual operations (#1319)
Signed-off-by: binlijin <binlijin@gmail.com>
2020-03-24 13:33:00 +08:00
Reid Chan bdcfd6a4a8
[RSGroup] Forward-port HBASE-22658 to master branch and branch-2.x (#1326)
Signed-off-by: stack <stack@apache.org>
2020-03-24 10:34:12 +08:00
stack afc1746757 HBASE-24034 [Flakey Tests] A couple of fixes and cleanups
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupMajorCompactionTTL.java
 Remove spurious assert. Just before this it waits an arbitrary 10
 seconds. Compactions could have completed inside this time. The spirit
 of the test remains.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java
 Get log cleaner to go down promptly; its sticking around. See if this
 helps with TestMasterShutdown

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
 We get a rare NPE trying to sync. Make local copy of SyncFuture and see
 if that helps.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
 Compaction  may have completed when not expected; allow for it.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestBlockEvictionFromClient.java
 Add wait before testing. Compaction may not have completed. Let
 compaction complete before progressing and then test for empty cache.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
 Less resources.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestDefaultLoadBalancer.java
 Less resources.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
 Wait till online before we try and do compaction (else request is
 ignored)

hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
 Disable test that fails randomly w/ mockito complaint on some mac os
 x's.

TestMasterShutdown... fix NPE in RSRpcDispatcher... catch it and covert
to false and have master check for successful startup.
2020-03-23 16:26:32 -07:00
Nick Dimiduk af4fdcc556 HBASE-23980 Use enforcer plugin to print JVM info in maven output
Does what it says on the tin. Bound to `initialize` phase so that it
runs early in lifecycle. Uses `<inherited>false</inherited>` so that
the plugin will run only for the base pom's reactor stage and not for
any children.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-03-23 14:27:55 -07:00
stack 2f26d8a623 HBASE-24035 [Flakey Tests] Disable TestClusterScopeQuotaThrottle#testUserNamespaceClusterScopeQuota 2020-03-23 13:44:34 -07:00
Viraj Jasani 7081ccd53c
HBASE-23936 : Thrift support for Online SlowLog APIs (#1317)
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
2020-03-23 22:59:04 +05:30
Guanghao Zhang 8130858199
HBASE-23741 Data loss when WAL split to HFile enabled (#1254)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-23 14:38:47 +08:00
Duo Zhang eface74407
HBASE-23799 Make our core coprocessors use shaded protobuf (#1280)
Signed-off-by: stack <stack@apache.org>
2020-03-23 08:47:02 +08:00
Huaxiang Sun 542244056c
HBASE-23957 [flakey test] client.TestMultiParallel fails to read hbase-site.xml (#1310)
Signed-off-by: Nick Dimiduk ndimiduk@apache.org
Signed-off-by: stack <stack@apache.org>
2020-03-22 17:43:04 -07:00
Guanghao Zhang 2a60a61a73
HBASE-24033 Add ut for loading the corrupt recovered hfiles (#1322)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-22 22:56:30 +08:00
niuyulin d4ab645531
HBASE-23949 refactor loadBalancer implements for rsgroup balance by table to achieve overallbalanced (#1279)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-03-22 21:54:57 +08:00
Pankaj ca533a77cb
HBASE-23633 Find a way to handle the corrupt recovered hfiles (#1233)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-03-22 16:39:56 +08:00
Toshihiro Suzuki 9b4e75a7f6
HBASE-24030 Add necessary validations to HRegion.checkAndMutate() and HRegion.checkAndRowMutate() (#1315)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-03-22 11:55:52 +09:00
Toshihiro Suzuki b53fa3c51f
HBASE-24031 TestHRegion.testCheckAndMutate_WithFilters is flaky (#1316)
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2020-03-22 05:04:16 +09:00
Michael Stack 5592d0c191 HBASE-23984 [Flakey Tests] TestMasterAbortAndRSGotKilled fails in teardown (#1311)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 Change parameter name and add javadoc to make it more clear what the
 param actually is.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/AssignRegionHandler.java
 Move postOpenDeployTasks so if it fails to talk to the Master -- which
 can happen on cluster shutdown -- then we will do cleanup of state;
 without this the RS can get stuck and won't go down.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java
 Add handleException so CRH looks more like UnassignRegionHandler and
 AssignRegionHandler around exception handling. Add a bit of doc on
 why CRH.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/UnassignRegionHandler.java
 Right shift most of the body of process so can add in a finally
 that cleans up rs.getRegionsInTransitionInRS is on exception
 (otherwise outstanding entries can stop a RS going down on cluster
 shutdown)

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-20 15:26:58 -07:00
Guanghao Zhang 080d8645ed
HBASE-24023 Add 2.2.4 to download page (#1314)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-03-20 23:57:19 +08:00
Viraj Jasani 44b3ac6457
HBASE-23977 : Resolve flakes present in TestSlowLogRecorder (ADDENDUM) 2020-03-20 13:27:06 +05:30
Viraj Jasani 26b31e351c
HBASE-23977 : Resolve flakes present in TestSlowLogRecorder (#1286)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-03-19 15:35:49 +05:30
Huaxiang Sun 52a04e69c1
HBASE-23999 [flakey test] TestTableOutputFormatConnectionExhaust (#1302)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-03-18 21:51:04 -07:00
stack de967502ab HBASE-24017 Turn down flakey rerun rate on all but hot branches
On master branch run twice a day instead of every 4 hours.
2020-03-18 15:50:39 -07:00
stack add2d2ac57 HBASE-24016 Change nightly poll from cron @daily to pollSCM @daily; i.e. run nightly if a change ONLY 2020-03-18 14:39:53 -07:00
Jan Hentschel d7e70bb497
HBASE-23943 Updated Rubocop configuration to current version
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-03-18 11:09:36 +01:00
Michael Stack 3e1cf00c71 HBASE-23993 Use loopback for zk standalone server in minizkcluster (#1291)
hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 Have client and server use loopback instead of 'localhost'

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-03-17 20:26:06 -07:00
Duo Zhang 9190a6b59a
HBASE-23989 Consider 2.10.0 in our support matrix (#1290)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-03-18 09:58:16 +08:00
stack 9804f73c66 HBASE-23971 protoc warns: "no protobuf syntax specified" (#1278)
Remove overcommit in TestUsersOperationsWithSecureHadoop
(Noticed by ndimiduk)
2020-03-17 15:28:26 -07:00
Nick Dimiduk 2e1485f505 HBASE-24004 Include hadoop version in Nightly report name 2020-03-17 14:49:59 -07:00
Wei-Chiu Chuang 60de4c2e31
HBASE-22103. HDFS-13209 in Hadoop 3.3.0 breaks asyncwal. (#1284)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-17 14:36:56 +08:00
Duo Zhang 795e9887b5
HBASE-23986 Bump hadoop-two.version to 2.10.0 on master and branch-2 (#1289)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-03-17 08:05:56 +08:00
Nick Dimiduk 059c189451 HBASE-23829 Get `-PrunSmallTests` passing on JDK11
Signed-off-by: stack <stack@apache.org>
2020-03-16 15:21:37 -07:00
stack f66cbe1a40 HBASE-23992 Fix TestAdminShell and TestQuotasShell mistakenly broken by parent commit 2020-03-14 15:18:21 -07:00
stack eefeeceedf HBASE-23991 [Flakey Tests] Disable TestSecureExport.testVisibilityLabels 2020-03-14 14:05:36 -07:00
Duo Zhang ba3f02629a
HBASE-23987 NettyRpcClientConfigHelper will not share event loop by default which is incorrect (#1288)
Signed-off-by: stack <stack@apache.org>
2020-03-14 13:47:41 +08:00
xuqinya1 b9e74e0b0a
HBASE-23967 Improve the accuracy of the method sizeToString (#1273)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-03-13 19:41:48 +05:30
Jan Hentschel 1b163d98b9
HBASE-23847 Removed deprecated setStartRow from Scan (#1220)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-03-13 08:43:17 +08:00
Duo Zhang e5b0455c38 HBASE-23077 Addendum typo fix fingbugs -> findbugs 2020-03-13 08:36:39 +08:00
stack 02b6bbe7ce HBASE-23979 Disable TestSlowLogRecorder until parent addressed 2020-03-12 17:24:21 -07:00
Nick Dimiduk c088d2edc9 HBASE-23978 Dockerfiles reusing stale apt sources info
Signed-off-by: stack <stack@apache.org>
2020-03-12 17:09:06 -07:00
WenFeiYi abf0a22259
HBASE-23971 protoc warns: "no protobuf syntax specified" (#1278)
Signed-off-by: stack <stack@apache.org>
2020-03-12 08:32:22 -07:00
stack 26f3702735 HBASE-23974 [Flakey Tests] Allow that server may not yet be cleared from DeadServers in TestHBCKSCP 2020-03-11 23:04:16 -07:00
stack d42e9768d0 HBASE-23956 Use less resources running tests (#1266)
Addendum: TestNamespaceCommands was using up the limit on
nioeventloopgroup.... so create them per run of client.
2020-03-11 22:48:22 -07:00
stack 20658c5139 HBASE-23956 Use less resources running tests (#1266)
Addendum: Two more tests that had issues because parent cut
down the number of default RegionServers launched.
2020-03-11 21:03:10 -07:00
Duo Zhang f4c576582b HBASE-23077 Addendum remove findbugs annotations dependency from shaded client 2020-03-12 11:41:32 +08:00
stack 4b64558ebc HBASE-23956 Use less resources running tests (#1266)
Addendum: This test was depending on hard-coded number for
log cleaners; the parent patch made them less in test context.
2020-03-11 20:34:29 -07:00
Duo Zhang c5aa2bc121
HBASE-23077 move entirely to spotbugs (#1265)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-03-12 10:18:09 +08:00
stack cf121227aa HBASE-23930 Shell should attempt to format `timestamp` attributes as ISO-8601
Make display of Cell timestamp be ISO8601 format instead of pure milliseconds.
2020-03-11 17:15:15 -07:00
Minwoo Kang 6f1e8abc30
HBASE-23561 acquire lock before update regionsMap (#1263)
Addendum: tie add and delete of regionMap and encodedRegionsMap

Signed-off-by: stack <stack@apache.org>
2020-03-11 16:57:21 -07:00
stack 0b18a7dc26 HBASE-23929 Shell formatter for for meta table should pretty-print values of info:merge columns 2020-03-11 16:41:30 -07:00