Addendum:
For major compaction test, set hbase.hstore.compaction.min to a big number to
avoid kicking in minor compactions, which will pollute compaction state and
sometimes, cause major compaction cannot happen.
Co-authored-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.org>
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationTrackerZKImpl.java
Add debug for when assert fails (it fails on occasion locally)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestHDFSAclHelper.java
Move this inner class out standalone since it used now by two tests.
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSnapshotScannerHDFSAclController.java
Moved out testRestoreSnapshot and made methods in here static so could
be used by a new adacent test. Also made tablenames unique to methods
thinking that was root of original issue (wasn't but no harm in doing
this change) Moved out the inner class TestHDFSAclHelper.
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSnapshotScannerHDFSAclController2.java
New class that sets up same context as
TestSnapshotScannerHDFSAclController but just to run single
testRestoreSnapshot method.
hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/TestZKSecretWatcher.java
Some debug.
Signed-off-by: Yi Mei
JMXCacheBuster resets the metrics state at various points in time. These
events can potentially race with a master shutdown. When the master is
tearing down, metrics initialization can touch a lot of unsafe state,
for example invalidated FS objects. To avoid this, this patch makes
the getMetrics() a no-op when the master is either stopped or in the
process of shutting down. Additionally, getClusterId() when the server
is shutting down is made a no-op.
Simulating a test for this is a bit tricky but with the patch I don't
locally see the long stacktraces from the jira.
Signed-off-by: Michael Stack <stack@apache.org>
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientAsyncPrefetchScanner.java
Refactor to avoid NPE timing issue referencing lock during Construction.
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
Comment
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
Refactor. Catch NPE during startup and return it instead as failed initialization.
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java
Catch IndexOutOfBounds exception and convert to non-split request.
hbase-server/src/test/java/org/apache/hadoop/hbase/TestCachedClusterId.java
Make less furious. Make it less flakie.
hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerSideScanMetricsFromClientSide.java
Debug. Catch exception to log, then rethrow.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
Guess that waiting longer on compaction to succeed may help make this
less flakey.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
Be explicit about timestamping to avoid concurrent edit landing
server-side and messing up test expectation.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMasterRegistry.java
Add wait on meta before proceeding w/ test.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestScannersFromClientSide.java
Be explicit that edits are distinct.
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCacheRefCnt.java
Add @Ignore on RAM test... Fails sporadically.
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionMoveAndAbandon.java
Add wait for all RegionServers going down before proceeding; was
messing up RS accounting.
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/BalancerTestBase.java
Make balancer test sloppier; less restrictive; would fail on occasion
by being just outside test limits.
hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestQuotaObserverChoreRegionReports.java
Add wait on quota table coming up; helps make this less flakie.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
Be explicity about timestamps; see if helps w/ flakie failure.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java
Catch and ignore if issue in shutdown; don't care if after test.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java
Comment.
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
Add retry to see if helps w/ odd failure; grant hasn't propagated?
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLWithMultipleVersions.java
Explicit w/ timestamps so no accidental overlap of puts.
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
Hack to deal w/ BindException on startup.
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThrift2ServerCmdLine.java
Use loopback.
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandler.java
Disable flakie test.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
The PLAIN mechanism test added in the Shade authentication example has
different semantics than GSSAPI mechanism -- the client reports that the
handshake is done after the original challenge is computed. The javadoc
on SaslClient, however, tells us that we need to wait for a response
from the server before proceeding.
The client, best as I can see, does not receive any data from HBase;
however the application semantics (e.g. throw an exception on auth'n
error) do not work as we intend as a result of this bug.
Extra trace logging was also added to debug this, should a similar error
ever happen again with some other mechanism.
Closes#1260
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java
Edit of log about archiving that shows in middle of a table create;
try to make it less disorientating.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
Loosen assert. Compaction may have produced a single file only. Allow
for this.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableGetMultiThreaded.java
Make this test less furious given it is inline w/ a bunch of unit
tests.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
Add debug
hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestQuotaObserverChoreRegionReports.java
Add wait on quota table to show up before moving forward; otherwise,
attempt at quota setting fails.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
Debug
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
Remove asserts that expected regions to still have a presence in fs
after merge when a catalogjanitor may have cleaned up parent dirs.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java
Catch exception on way out and log it rather than let it fail test.
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSnapshotScannerHDFSAclController.java
Wait on acl table before proceeding.
hbase-rsgroup/src/test/java/org/apache/hadoop/hbase/rsgroup/TestRSGroupMajorCompactionTTL.java
Remove spurious assert. Just before this it waits an arbitrary 10
seconds. Compactions could have completed inside this time. The spirit
of the test remains.
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java
Get log cleaner to go down promptly; its sticking around. See if this
helps with TestMasterShutdown
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
We get a rare NPE trying to sync. Make local copy of SyncFuture and see
if that helps.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
Compaction may have completed when not expected; allow for it.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestBlockEvictionFromClient.java
Add wait before testing. Compaction may not have completed. Let
compaction complete before progressing and then test for empty cache.
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java
Less resources.
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestDefaultLoadBalancer.java
Less resources.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java
Wait till online before we try and do compaction (else request is
ignored)
hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestCanaryTool.java
Disable test that fails randomly w/ mockito complaint on some mac os
x's.
TestMasterShutdown... fix NPE in RSRpcDispatcher... catch it and covert
to false and have master check for successful startup.
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Change parameter name and add javadoc to make it more clear what the
param actually is.
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/AssignRegionHandler.java
Move postOpenDeployTasks so if it fails to talk to the Master -- which
can happen on cluster shutdown -- then we will do cleanup of state;
without this the RS can get stuck and won't go down.
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/CloseRegionHandler.java
Add handleException so CRH looks more like UnassignRegionHandler and
AssignRegionHandler around exception handling. Add a bit of doc on
why CRH.
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/UnassignRegionHandler.java
Right shift most of the body of process so can add in a finally
that cleans up rs.getRegionsInTransitionInRS is on exception
(otherwise outstanding entries can stop a RS going down on cluster
shutdown)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
Have client and server use loopback instead of 'localhost'
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
