Commit Graph

17093 Commits

Author SHA1 Message Date
Michael Stack 6c69b65e4f
HBASE-24215 [Flakey Tests] [ERROR] TestSecureRESTServer java.lang.NoClassDefFoundError: com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate (#1553) 2020-04-20 21:51:16 -07:00
stack 8a1ffa6c43 HBASE-24220 Allow that zk NOTEMPTY multi exception is retryable by running in-series
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java
 Cleanup checkstyle warnings. Don't depend on hbase-client
 ScannerCallable.

hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
 Cut down on cluster resource usage.

hbase-server/src/test/java/org/apache/hadoop/hbase/master/AbstractTestDLS.java
 Debug

hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController3.java
 Debug

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
 Debug

hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 Debug

hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
 Allow that NONEMPTY is retryable by running in series.
2020-04-20 15:43:09 -07:00
Guangxu Cheng 2762ddc6a8 HBASE-23896 Snapshot owner cannot delete snapshot when ACL is enabled and Kerberos is not enabled (#1211)
Signed-off-by: binlijin <binlijin@gmail.com>
2020-04-20 10:26:21 +08:00
Reid Chan eed39202e1
HBASE-24174 Fix findbugs warning on ServiceAuthorizationManager for master branch (#1537) (#1542)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-18 13:36:19 +08:00
XinSun 935c23312f HBASE-24166 Duplicate implementation for acquireLock between CreateTableProcedure and its parent class (#1486)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-18 10:31:46 +08:00
Peter Somogyi 4646f7b52b HBASE-24203 Bump git version in Dockerfile; addendum (#1543)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-04-17 23:32:37 +02:00
Nick Dimiduk a2c78e99cb HBASE-24143 [JDK11] Switch default garbage collector from CMS
Per comments in Jira, be explicit about what collector we
use. Existing code simply hard-codes HBASE_OPTS in
`conf/hbase-env.sh`. We now need to be a little more clever than this,
so moves the definition into `bin/hbase`. Also consolidates logic
around JVM version detection into a reusable function.

This change also changes how we set `HBASE_OPTS`. Before, we would
accept an operator's value, but always append our GC
prescription. After this change, we defer entirely to the operator's
choice, only applying our values when they've not specified their
intentions.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-04-17 13:53:21 -07:00
Joseph295 c775adfc06 HBASE-24182 log when the region is set to closing status (#1509)
Signed-off-by: stack <stack@apache.org>
2020-04-17 12:21:12 -07:00
Wei-Chiu Chuang 98ecd584c4
HBASE-23833. The relocated hadoop-thirdparty protobuf breaks HBase asyncwal (#1301) (#1534)
* Use Reflection to access shaded Hadoop protobuf classes.

(cherry picked from commit a321e536989083ca3620bf2c53f12c07740bf5b0)

* Update to improve the code:

1. Added license.
2. Added more comments.
3. Wrap byte array instead of copy to make a ByteString.
4. Moved all reflection instantiation to static class loading time.

* Use LiteralByteString to wrap byte array instead of copying it.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
(cherry picked from commit 72727ff9be)
2020-04-17 12:08:02 -07:00
Michael Stack 34961d40b2
HBASE-24148: Upgrade Thrift to 0.13.0: 0.12.0 has outstanding CVEs. (#1519)
Co-authored-by: Tamas Penzes <tamaas@cloudera.com>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-04-17 08:12:43 -07:00
gkanade 361e81e1f8 MetricsTable#updateFlushTime is wrong (#1539)
Authored-by: Gaurav Kanade <gkanade@microsoft.com>
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
2020-04-17 19:24:52 +05:30
Peter Somogyi 55f65dc030 HBASE-24203 Bump git version in Dockerfile (#1538)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-17 13:07:40 +02:00
stack 3f43b9e349 HBASE-24158 [Flakey Tests] TestAsyncTableGetMultiThreaded
Addendum to address NPE
2020-04-16 08:02:54 -07:00
Viraj Jasani a0a769d96f
HBASE-24195 : Admin.getRegionServers() should return live servers exc… (#1523)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-16 20:16:30 +05:30
BukrosSzabolcs 8498fe7260
HBASE-24186: RegionMover ignores replicationId (#1512)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-16 19:56:52 +05:30
Viraj Jasani ca1f7a9499
HBASE-24194 : Refactor anonymous inner classes of BufferedEncodedSeeker to named inner classes (#1522)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
2020-04-16 19:17:44 +05:30
Gkkkk302 22912ff346
HBASE-24151 The master server aborted for IllegalThreadStateException (#1507)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-16 15:41:26 +05:30
Istvan Toth ec266a0743 HBASE-24197 TestHttpServer.testBindAddress failure with latest jetty (#1524)
use the findPort logic even if the BindException is wrapped in an IOException

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-04-16 11:52:35 +02:00
stack 360baf9dd6 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Addendum #2: The new test fails on origin.hbase.dir written by HTU up on
jenkins... has a 'tmp' in it. Fix.
2020-04-15 22:42:09 -07:00
Tamas Penzes 66a329e4b7 HBASE-24148: Upgrade Thrift to 0.13.0: 0.12.0 has outstanding CVEs.
Upgrade thrift from 0.12 to 0.13.
2020-04-15 22:40:57 -07:00
huaxiangsun efb5ff07ae
HBASE-24183 [flakey test] replication.TestAddToSerialReplicationPeer (#1514) (#1525)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-15 14:42:38 -07:00
stack 62c5bdcc08 HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Addendum: add test to check for '/tmp' references.
2020-04-15 10:55:08 -07:00
Reid Chan 4c83067e1e
HBASE-24112 [RSGroup] Support renaming rsgroup (#1520)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-15 19:04:40 +08:00
Michael Stack 62a488b946 HBASE-24126 Up the container nproc uplimit from 10000 to 12500 (#1504) 2020-04-14 17:36:06 -07:00
GeorryHuang f61aa0292d HBASE-23994: Add WebUI to Canary (#1292)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-04-14 16:09:47 -07:00
binlijin df3768cf2a HBASE-24181 Add region info when log meessages in HRegion. (#1506)
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-14 19:45:19 +08:00
EEPROM a66bd7a769 HBASE-24176 user_permission '.*' command failed to show all table permissions (#1499)
* user_permission failed when show table permissions with '.*' argument

Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2020-04-14 10:37:34 +08:00
Jianfei Jiang 2f4ae522c3 HBASE-24132 Upgrade to Apache ZooKeeper 3.5.7 (#1453)
* HBASE-24132 Upgrade to Apache ZooKeeper 3.5.7

* HBASE-24132 Upgrade to Apache ZooKeeper 3.5.7

Co-authored-by: 姜建飞 10222269 <jiang.jianfei@zte.com.cn>

Signed-off-by: Mate Szalay-Beko <szalay.beko.mate@gmail.com>
Signed-off-by: Norbert Kalmar <nkalmar@cloudera.com>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangdo@apache.org>
2020-04-13 13:48:47 -07:00
Joseph295 91361ca999
HBASE-24168 Align the internal.protobuf.version in hbase-protocol-shaded with hbase-thirdparty (#1489)
Co-authored-by: Joseph295 <johnhomsea@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-13 08:49:12 +08:00
Viraj Jasani 45622abe2f
HBASE-24140 : Move CandidateGenerator and their implementors out of StochasticLoadBalancer (#1458)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-12 19:21:06 +05:30
Jan Hentschel 8a117c9b8e
HBASE-23998. Update license for jetty-client. (#1297) (#1497)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>

Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-04-12 11:58:42 +02:00
stack 9067dfe83c HBASE-24175 [Flakey Tests] TestSecureExportSnapshot FileNotFoundException
Overwrite yarn configs that write /tmp dir by default.
2020-04-11 21:23:17 -07:00
stack cc4aaefa52 HBASE-24167 [Flakey Tests] TestHRegionWithInMemoryFlush#testWritesWhileScanning gets stuck MVCC 2020-04-10 18:30:08 -07:00
stack 3ccff5784e HBASE-24162 Move CHANGES.txt to CHANGES.md. Add RELEASENOTES.md too on branch-2, branch-2.3, and master 2020-04-10 14:41:02 -07:00
huaxiangsun 3e4dfa9b31
HBASE-24161 [flakey test] locking.TestEntityLocks.testEntityLockTimeout (#1477) (#1480)
Signed-off-by: stack <stack@apache.org>
2020-04-10 09:42:47 -07:00
huaxiangsun 39187f87dd
HBASE-24159 [flakey test] regionserver.TestRegionMergeTransactionOnCluster (#1472) (#1474)
Delete tables after each test cases to avoid polluting states.

Signed-off-by: stack <stack@apache.org>
2020-04-09 21:09:34 -07:00
Pankaj 823cf97495 HBASE-24074 ConcurrentModificationException occured in ReplicationSourceManager while refreshing the peer (#1471)
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
Signed-off-by: stack <stack@apache.org>
2020-04-09 16:42:22 -07:00
Bharath Vissapragada 91aa3ce97c HBASE-24156: Switch the default registry to ZK based
In branch-2 we want to retain the ZK based registry as the default.
2020-04-09 16:13:45 -07:00
Bharath Vissapragada db7ad07081 Revert "Switch the default registry to ZK based"
This reverts commit 558ee079fd because
it wasn't tagged with a jira ID. Will be re-applied.
2020-04-09 16:13:45 -07:00
Michael Stack 9eb49172d0
HBASE-24150 Allow module tests run in parallel (#1464)
Pass --threads=2 to mvn when yetus runs so some parallelism
when dependencies allow.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-09 14:41:41 -07:00
stack 2d11796380 HBASE-24158 [Flakey Tests] TestAsyncTableGetMultiThreaded 2020-04-09 11:03:22 -07:00
Reid Chan 5012241817 HBASE-24121 [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be (#1439)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2020-04-09 16:30:00 +08:00
Sean Busbey 611c62f52a HBASE-24138 log more details about balancer decisions for StochasticLoadBalancer (#1455)
- at DEBUG log messages about RegionCountSkewCostFunction region/server totals
- at DEBUG log messages about the decision to balance or not with total costs
- at TRACE log messages about region count on each server RegionCountSkewCostFunction sees
- at TRACE log message with the individual cost functions used in the decision to balance or not

Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 2d78a286b6)
2020-04-08 15:50:38 -05:00
huaxiangsun 6f00714ad3
HBASE-24120 Flakey Test: TestReplicationAdminWithClusters timeout (#1441) (#1456)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-08 12:23:50 -07:00
Andrew Purtell 3f511b0e93
HBASE-24099 Use a fair ReentrantReadWriteLock for the region close lock (#1407)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-04-08 11:40:42 -07:00
Michael Stack 5db62b422c
HBASE-24131 [Flakey Tests] TestExportSnapshot takes too long; up against 13min max (#1452)
Split TestExportSnapshot and TestVerifyReplication to two smaller tests
rather than one big one that can take 13minutes+ when contention.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-04-08 10:57:18 -07:00
Michael Stack ed54c35cf0
HBASE-24134 Down forked JVM heap size from 2800m to 2200m for jdk8 and jdk11 (#1451)
Down jdk8 forked jvm heap from 2800 to 2200 and the jdk11 heap from
3200 to 2200. Down the mvn size from 4G to 3.6G

Change how many puts done by TestMultiRespectsLimits because made
the test run the forked heap over 2.5G in size.

Signed-off-by: Sean Busbey <busbey@apache.org>
2020-04-08 10:51:03 -07:00
stack 953c326498 HBASE-24126 Up the container nproc uplimit from 10000 to 12500
Addendum number 3; set the yetus proclimit.
2020-04-07 10:22:24 -07:00
stack ba87ffed09 Revert "HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create new native thread."
Revert because applied with wrong JIRA number.

This reverts commit 9c60c5fddb.
2020-04-07 10:17:48 -07:00
Michael Stack 9da7f95fa7
HBASE-24128 [Flakey Tests] Add retry on thrift cmdline if client fails plus misc debug (#1442)
hbase-server/src/test/java/org/apache/hadoop/hbase/TestClusterPortAssignment.java
 Saw case where Master failed startup but it came out as an IOE so we
 did not trip the retry logic.

hbase-server/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java
 Add some debug and up timeouts. This test fails frequently for me
 locally.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/locking/TestEntityLocks.java
 Up the wait from 2x 200ms to 10x in case a pause on hardware or GC.
 This test fails locally and up on jenkins.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestClearRegionBlockCache.java
 Debug. Have assert say what bad count was.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactingToCellFlatMapMemStore.java
 Fails on occasion. Found count is off by a few. Tricky to debug. HBASE-24129 to reenable.

hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java
 Debug. Add wait and check before moving to assert.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftHttpServer.java
 Check for null before shutting; can be null if failed start.

hbase-thrift/src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerCmdLine.java
 Add retry if client messes up connection. Fails for me locally.
2020-04-07 09:28:05 -07:00