* Use Reflection to access shaded Hadoop protobuf classes.
(cherry picked from commit a321e536989083ca3620bf2c53f12c07740bf5b0)
* Update to improve the code:
1. Added license.
2. Added more comments.
3. Wrap byte array instead of copy to make a ByteString.
4. Moved all reflection instantiation to static class loading time.
* Use LiteralByteString to wrap byte array instead of copying it.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
Split TestExportSnapshot and TestVerifyReplication to two smaller tests
rather than one big one that can take 13minutes+ when contention.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
hbase-server/src/test/java/org/apache/hadoop/hbase/TestClusterPortAssignment.java
Saw case where Master failed startup but it came out as an IOE so we
did not trip the retry logic.
hbase-server/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java
Add some debug and up timeouts. This test fails frequently for me
locally.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/locking/TestEntityLocks.java
Up the wait from 2x 200ms to 10x in case a pause on hardware or GC.
This test fails locally and up on jenkins.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestClearRegionBlockCache.java
Debug. Have assert say what bad count was.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
Fails on occasion. Found count is off by a few. Tricky to debug. HBASE-24129 to reenable.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
Debug. Add wait and check before moving to assert.
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
Check for null before shutting; can be null if failed start.
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
Add retry if client messes up connection. Fails for me locally.
Move the proto files only used by hbase-examples to hbase-examples, to show users how to make use of shaded hbase protos when implementing coprocessor.
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
proto files layout:
protobuf/client - client to server messages, client rpc service and protos, used in hbase-client exclusively;
protobuf/rest - hbase-rest messages;
protobuf/rpc - rpc and post-rpc tracing messages;
protobuf/server/coprocessor - coprocessor rpc services;
protobuf/server/coprocessor/example - coprocessors rpc services examples from hbase-examples;
protobuf/server/io - filesystem and hbase-server/io protos;
protobuf/server/maser - master rpc services and messages;
protobuf/server/region - region rpc services and messages (except client rpc service, which is in Client.proto);
protobuf/server/rsgroup - rsgroup protos;
protobuf/server/zookeeper - protos for zookeeper and ones used exclusively in hbase-zookeeper module;
protobuf/server - protos used across other server protos;
protobuf/test - protos used in tests;
protobuf/ - protos used across other protos, exclusive for hbase-mapreduce and hbase-backup, other protos.
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
After HBASE-23986, running on JDK8/Hadoop2 from the binary tarball
results in
```
Exception in thread "main" java.lang.NoClassDefFoundError: com/ctc/wstx/io/InputBootstrapper
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3062)
Caused by: java.lang.ClassNotFoundException: com.ctc.wstx.io.InputBootstrapper
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 2 more
```
The missing class is from `com.fasterxml.woodstox:woodstox-core`. This
and `org.codehaus.woodstox:stax2-api` are explicit dependencies of
`org.apache.hadoop:hadoop-common:jar:2.10.0`. Thus, include them in
the default jars provided under `lib`.
Signed-off-by: stack <stack@apache.org>
Addendum:
For major compaction test, set hbase.hstore.compaction.min to a big number to
avoid kicking in minor compactions, which will pollute compaction state and
sometimes, cause major compaction cannot happen.
Co-authored-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.org>
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationTrackerZKImpl.java
Add debug for when assert fails (it fails on occasion locally)
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestHDFSAclHelper.java
Move this inner class out standalone since it used now by two tests.
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSnapshotScannerHDFSAclController.java
Moved out testRestoreSnapshot and made methods in here static so could
be used by a new adacent test. Also made tablenames unique to methods
thinking that was root of original issue (wasn't but no harm in doing
this change) Moved out the inner class TestHDFSAclHelper.
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestSnapshotScannerHDFSAclController2.java
New class that sets up same context as
TestSnapshotScannerHDFSAclController but just to run single
testRestoreSnapshot method.
hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/TestZKSecretWatcher.java
Some debug.
Signed-off-by: Yi Mei
JMXCacheBuster resets the metrics state at various points in time. These
events can potentially race with a master shutdown. When the master is
tearing down, metrics initialization can touch a lot of unsafe state,
for example invalidated FS objects. To avoid this, this patch makes
the getMetrics() a no-op when the master is either stopped or in the
process of shutting down. Additionally, getClusterId() when the server
is shutting down is made a no-op.
Simulating a test for this is a bit tricky but with the patch I don't
locally see the long stacktraces from the jira.
Signed-off-by: Michael Stack <stack@apache.org>
hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientAsyncPrefetchScanner.java
Refactor to avoid NPE timing issue referencing lock during Construction.
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
Comment
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java
Refactor. Catch NPE during startup and return it instead as failed initialization.
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java
Catch IndexOutOfBounds exception and convert to non-split request.
hbase-server/src/test/java/org/apache/hadoop/hbase/TestCachedClusterId.java
Make less furious. Make it less flakie.
hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerSideScanMetricsFromClientSide.java
Debug. Catch exception to log, then rethrow.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi.java
Guess that waiting longer on compaction to succeed may help make this
less flakey.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
Be explicit about timestamping to avoid concurrent edit landing
server-side and messing up test expectation.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMasterRegistry.java
Add wait on meta before proceeding w/ test.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestScannersFromClientSide.java
Be explicit that edits are distinct.
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCacheRefCnt.java
Add @Ignore on RAM test... Fails sporadically.
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionMoveAndAbandon.java
Add wait for all RegionServers going down before proceeding; was
messing up RS accounting.
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/BalancerTestBase.java
Make balancer test sloppier; less restrictive; would fail on occasion
by being just outside test limits.
hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestQuotaObserverChoreRegionReports.java
Add wait on quota table coming up; helps make this less flakie.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
Be explicity about timestamps; see if helps w/ flakie failure.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java
Catch and ignore if issue in shutdown; don't care if after test.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java
Comment.
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
Add retry to see if helps w/ odd failure; grant hasn't propagated?
hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLWithMultipleVersions.java
Explicit w/ timestamps so no accidental overlap of puts.
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
Hack to deal w/ BindException on startup.
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThrift2ServerCmdLine.java
Use loopback.
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift2/TestThriftHBaseServiceHandler.java
Disable flakie test.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Andrew Purtell
Wei-Chiu Chuang
Michael Stack
huaxiangsun
Duo Zhang
Minwoo Kang
binlijin
Nick Dimiduk
stack
Peter Somogyi
Semen Komissarov
Viraj Jasani
meiyi
niuyulin
Jan Hentschel
申胜利
Bharath Vissapragada