Per comments in Jira, be explicit about what collector we
use. Existing code simply hard-codes HBASE_OPTS in
`conf/hbase-env.sh`. We now need to be a little more clever than this,
so moves the definition into `bin/hbase`. Also consolidates logic
around JVM version detection into a reusable function.
This change also changes how we set `HBASE_OPTS`. Before, we would
accept an operator's value, but always append our GC
prescription. After this change, we defer entirely to the operator's
choice, only applying our values when they've not specified their
intentions.
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
* Use Reflection to access shaded Hadoop protobuf classes.
(cherry picked from commit a321e536989083ca3620bf2c53f12c07740bf5b0)
* Update to improve the code:
1. Added license.
2. Added more comments.
3. Wrap byte array instead of copy to make a ByteString.
4. Moved all reflection instantiation to static class loading time.
* Use LiteralByteString to wrap byte array instead of copying it.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
(cherry picked from commit 72727ff9be)
(cherry picked from commit ae3de38bed)
Upgrade thrift from 0.12 to 0.13.
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
use the findPort logic even if the BindException is wrapped in an IOException
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Add being able to configure netty thread counts. Enable socket reuse
(should not have any impact).
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BlockingRpcConnection.java
Rename the threads we create in here so they are NOT named same was
threads created by Hadoop RPC.
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/DefaultNettyEventLoopConfig.java
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcClient.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java
Allow configuring eventloopgroup thread count (so can override for
tests)
hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/HttpProxyExample.java
Enable socket resuse.
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
Enable socket resuse and config for how many threads to use.
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
hbase-server/src/main/java/org/apache/hadoop/hbase/util/ModifyRegionUtils.java
Thread name edit; drop the redundant 'Thread' suffix.
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
Make closeable and shutdown executor when called.
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
Call close on HFileReplicator
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationBase.java
HDFS creates lots of threads. Use less of it so less threads overall.
hbase-server/src/test/resources/hbase-site.xml
hbase-server/src/test/resources/hdfs-site.xml
Constrain resources when running in test context.
hbase-server/src/test/resources/log4j.properties
Enable debug on netty to see netty configs in our log
pom.xml
Add system properties when we launch JVMs to constrain thread counts in
tests
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Down jdk8 forked jvm heap from 2800 to 2200 and the jdk11 heap from
3200 to 2200. Down the mvn size from 4G to 3.6G
Change how many puts done by TestMultiRespectsLimits because made
the test run the forked heap over 2.5G in size.
Signed-off-by: Sean Busbey <busbey@apache.org>
Pass --threads=2 to mvn when yetus runs so some parallelism
when dependencies allow.
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
- at DEBUG log messages about RegionCountSkewCostFunction region/server totals
- at DEBUG log messages about the decision to balance or not with total costs
- at TRACE log messages about region count on each server RegionCountSkewCostFunction sees
- at TRACE log message with the individual cost functions used in the decision to balance or not
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 2d78a286b6)
Split TestExportSnapshot and TestVerifyReplication to two smaller tests
rather than one big one that can take 13minutes+ when contention.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
hbase-server/src/test/java/org/apache/hadoop/hbase/TestClusterPortAssignment.java
Saw case where Master failed startup but it came out as an IOE so we
did not trip the retry logic.
hbase-server/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java
Add some debug and up timeouts. This test fails frequently for me
locally.
hbase-server/src/test/java/org/apache/hadoop/hbase/client/locking/TestEntityLocks.java
Up the wait from 2x 200ms to 10x in case a pause on hardware or GC.
This test fails locally and up on jenkins.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestClearRegionBlockCache.java
Debug. Have assert say what bad count was.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
Fails on occasion. Found count is off by a few. Tricky to debug. HBASE-24129 to reenable.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
Debug. Add wait and check before moving to assert.
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
Check for null before shutting; can be null if failed start.
hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
Add retry if client messes up connection. Fails for me locally.
Peter Somogyi
Nick Dimiduk
Joseph295
Wei-Chiu Chuang
Tamas Penzes
gkanade
stack
Viraj Jasani
BukrosSzabolcs
Gkkkk302
Istvan Toth
huaxiangsun
Reid Chan
Duo Zhang
Michael Stack
GeorryHuang
binlijin
EEPROM
Jianfei Jiang
Jan Hentschel
Pankaj
Bharath Vissapragada
Sean Busbey
Andrew Purtell