Commit Graph

17846 Commits

Author SHA1 Message Date
Viraj Jasani 15627bb722
HBASE-24327 : Flaky connection in TestMasterShutdown (ADDENDUM) 2020-05-16 12:29:16 +05:30
meiyi a73132c62b
HBASE-24364 [Chaos Monkey] Invalid data block encoding in ChangeEncodingAction (#1707)
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-05-15 11:30:44 +08:00
Guanghao Zhang bab4b1cf8c HBASE-23702 hbase.hstore.flusher.count setting to 0 breaks HMaster (#1703)
Co-authored-by: Ctest
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-05-15 09:21:53 +08:00
Andrew Purtell 94f36fdb8e
Amend HBASE-24350: Extending and Fixing HBaseTable level replication metrics (#1704)
- Rename WALEntryBatch#getWaEntriesWithSize -> getWalEntriesWithSize
2020-05-14 13:42:48 -07:00
Nick Dimiduk 7e295e767f HBASE-24271 Set values in `conf/hbase-site.xml` that enable running on `LocalFileSystem` out of the box
Simplify the new user experience shipping a configuration that enables
a fresh checkout or tarball distribution to run in standalone mode
without direct user configuration. This change restores the behavior
we had when running on Hadoop 2.8 and earlier.

Patch for master includes an update to the book. This change will be
omitted when backporting to earlier branches.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-14 13:18:43 -07:00
Nick Dimiduk 8148c5858b Revert "HBASE-24086 Disable output stream capability enforcement when running on LocalFileSystem"
This reverts commit 10cbb3fb73.
2020-05-14 13:18:43 -07:00
Nick Dimiduk a602a00b8d Revert "HBASE-24106 Update getting started documentation after HBASE-24086"
This reverts commit 7de861bb83.
2020-05-14 13:18:43 -07:00
stack 32e2682310 HBASE-24368 Let HBCKSCP clear 'Unknown Servers', even if RegionStateNode has RegionLocation == null
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
 Edit a log.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/HBCKServerCrashProcedure.java
 Add override of isMatchingRegionLocation. Allow 'null' as a pass in
 HBCKSCP.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
 Add a method for HBCKSCP to override and be less strict filtering
 assigns.

hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
 Some doc on what 'Unknown Servers' are.
2020-05-14 13:09:03 -07:00
Sandeep Pal 2e5a664233
HBASE-24350: Extending and Fixing HBaseTable level replication metrics (#1704)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-05-14 10:34:51 -07:00
bsglz 5e32e08782
HBASE-24164 Retain the ReadRequests and WriteRequests of region on we… (#1500)
Signed-off-by: stack <stack@apache.org>
2020-05-13 22:28:45 -07:00
Mingliang Liu a40a0322a7
HBASE-23969 Meta browser should show all `info` columns (#1485)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-13 11:50:53 -07:00
Viraj Jasani a782531633
HBASE-24327 : Flaky connection in TestMasterShutdown#testMasterShutdo… (#1690)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-13 02:38:47 +05:30
Michael Stack dc360f0f41
HBASE-24354 Make it so can make an hbase1 schema and hbase2 schema equate (#1696) 2020-05-12 12:31:11 -07:00
Sambit Mohapatra c4e892e3f1
HBASE-23832 Old config hbase.hstore.compactionThreshold is ignored
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2020-05-12 23:19:32 +05:30
Gkkkk302 9e73244082
HBASE-24355 Fix typos in the HStore#compact annotation (#1699)
Co-authored-by: guo.kangkang <guo.kangkang@immomo.com>

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-12 21:07:49 +05:30
Duo Zhang 8601416ee8
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for modules other than hbase-assembly (#1640)
Signed-off-by: stack <stack@apache.org>
2020-05-12 12:03:30 +08:00
stack bb3191c563 HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP Addendum2: make same fix for TestRaceBetweenSCPAndTRSP but in a second place (addendum 1 got first place). 2020-05-11 20:48:21 -07:00
stack 2c11d5a0a0 HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP
Addendum: make same fix for TestRaceBetweenSCPAndTRSP which does same
thing.
2020-05-11 14:06:56 -07:00
Joseph295 68229c9c99
HBASE-24341 The region should be removed from ConfigurationManager as a ConfigurationObserver when it is closed (#1675)
Co-authored-by: Joseph295 <johnhomsea@gmail.com>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2020-05-11 17:31:54 +05:30
Viraj Jasani 6729cafa1d
Revert "HBASE-24327 : Handle shutdown() if master cannot be contacted (#1684)" (#1691)
This reverts commit 34a74534fe.
2020-05-11 16:43:49 +05:30
Geoffrey Jacoby ca81283fe5
HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S… (#1655)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lars Hofhansl <larsh@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-11 12:54:22 +05:30
shahrs87 ff85daf1e2
[HBASE-24190] Make kerberos value of hbase.security.authentication property case insensitive (#1687)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-11 10:54:45 +05:30
Viraj Jasani 34a74534fe
HBASE-24327 : Handle shutdown() if master cannot be contacted (#1684)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-10 17:09:51 +05:30
Matt Foley f3c0a0e06d
HBASE-24318 Create-release scripts fixes and enhancements (#1643)
* narrow 'dry-run' limits so see svn activity up to just before check-in.
* Fix several typos and, in case of dry run, enable build step to chain from tag step by keeping the tagged git repo.
* Improve Maven settings.xml file, and names of variables related to it. Remove unnecessary use of "-Dmaven.repo.local", put it in settings.xml instead. Stop putting password literals in settings.xml. 
* stop deleting maven settings file, now that it doesn't contain password strings
* Merge 'tag' into release-build.sh as another stage. Also found and fixed a couple bugs in the current release-tag.sh.
* Delete redundant release-tag.sh script.
* Small changes to make dev-support/create-release tools less focused
on hbase project only, while retaining special behaviors for
hbase sub-projects which share the hbase dist and jira locations.
* Changed terminology of release publish steps, from <build|publish>
to <publish-dist|publish-release>. In fact, what was formerly called
"build" built the distribution tarballs AND published them to dist,
while "publish" built the  maven artifacts AND published them to Nexus.
The new terminology clarifies what's happening, and removes the appearance
of order dependendency.
* Fix publish-snapshot so it does same checks as publish-release.
* Factor out common maven usages, and move them to build-util.sh.
* Change default polarity of DRY_RUN to default to true. Change -n ("no-publish") to -f ("force publish") to actually publish.
* Fix problems in do-release.sh so it runs correctly outside of docker,
  including DRY_RUN being exported.
* Have do-release.sh set REPO (shared maven local repository) if doing all three stages.
* Cleaned up REPO directory creation.
* General cleanup of comments and usage.
*  fix all 'shellcheck' errors
* use ${BASH_SOURCE[0]} instead of $0 to determine script directory path
* smarter way to read version from pom with mvn
* do maven-gpg-plugin config settings in maven settings file correctly as documented
* fix gpg signing failure on Mac due to gpg-agent timeout
* fix various bugs to enable publish-dist, publish-snapshot, and publish-release to work correctly as individual steps and/or without docker
* improve log reporting from publish-release step
*  fix bug in argument to checkcompatibility.py: replace PACKAGE_VERSION with GIT_REF
* demote "PACKAGE_VERSION" to "package_version_name" and undocument it in favor of RELEASE_TAG. Still enable appropriate defaulting in case RELEASE_TAG is undefined.
* unify RELEASE_VERSION with VERSION, to remove ambiguity and allow it to be set when only running 'publish' step without 'tag'
* query confirm RELEASE_TAG
* emphasize that release-build.sh is called for a single action at a time, and should be called from do-release.sh.
* add '-s' option to do-release.sh
* suppress maven "Download from central:" messages
* Replace human wait for tag propagation

Signed-off-by: Cesar Delgado <cdelgado@apple.com>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-05-09 10:46:41 -05:00
stack 2774510e2b HBASE-24284 [h3/jdk11] REST server won't start Exclude transitive includes of jax-rs 1.x and then explicitly include jax-rs 2.x glassfish impl for REST context when hadoop3. (#1625)
Addendum to fix failing MR tests; e.g. TestExportSnapshot.
2020-05-08 13:08:57 -07:00
huaxiangsun 30f07e8187
HBASE-24256 When fixOverlap hits the max region limit, it is possible to include the same region in multiple merge request (#1584)
Signed-off-by: stack <stack@apache.org>
2020-05-08 10:16:42 -07:00
Reid Chan 0cd70ed89c
HBASE-24345 [ACL] renameRSGroup should require Admin level permission (#1683)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-08 18:56:05 +08:00
Jan Hentschel 03e5a1402d
HBASE-24305 Removed deprecations in ServerName (#1666)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-05-08 09:41:55 +02:00
Duo Zhang fddb2dd65c
HBASE-24310 Use Slf4jRequestLog for hbase-http (#1634)
Signed-off-by: stack <stack@apache.org>
2020-05-08 11:16:18 +08:00
stack fc283f7a68 HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP #1669
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-07 13:54:45 -07:00
stack d30268d0ee HBASE-24342 [Flakey Tests] Disable TestClusterPortAssignment.testClusterPortAssignment as it can't pass 100% of the time 2020-05-07 13:22:35 -07:00
Nick Dimiduk 0e81ab08b9 HBASE-24295 [Chaos Monkey] abstract logging through the class hierarchy ; ADDENDUM
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-07 13:20:15 -07:00
bsglz 2cafe81e9c
HBASE-24335 Support deleteall with ts but without column in shell mode (#1668)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-05-07 18:18:57 +01:00
niuyulin f4a446c3d2
HBASE-24328 skip duplicate GCMultipleMergedRegionsProcedure while previous finished (#1629)
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-05-06 10:06:49 -07:00
Michael Stack 6f059ecff0 HBASE-24331 [Flakey Test] TestJMXListener rmi port clash (#1657)
Add check that we can make jmx connector in setup. If we can't retry.
2020-05-06 09:29:57 -07:00
Jan Hentschel 4ac1719521
HBASE-24334 Removed duplicated dependency declaration of metrics-api (#1665)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-06 14:22:12 +02:00
Reid Chan c32d18d4fb
HBASE-24336 [Metrics] FSDataInputStream's localBytesRead is wrong (#1667)
Signed-off-by: binlijin <binlijin@gmail.com>
2020-05-06 19:24:05 +08:00
Mohammad Arshad 294c228c67
HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1631)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-06 15:12:20 +05:30
Duo Zhang c1cb22f0b3
HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry … (#1593)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-06 15:55:26 +08:00
Duo Zhang a9a1b9524d
HBASE-24304 Separate a hbase-asyncfs module (#1628)
Signed-off-by: stack <stack@apache.org>
2020-05-06 14:40:21 +08:00
huaxiangsun 045c909bdf
HBASE-24316 GCMulitpleMergedRegionsProcedure is not idempotent (#1660)
It addresses couple issues:
   1. Make sure deleteMergeQualifiers() does not delete the row if there is no columns with "merge" keyword.
   2. GCMulitpleMergedRegionsProcedure now acquire an exclusive lock on the child region.

Signed-off-by: stack <stack@apache.org>
2020-05-05 23:04:48 -07:00
Nick Dimiduk 07077a3950
HBASE-23976 Use nio for file manipulation in TestVerifyBucketCacheFile (#1342)
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-05-05 22:04:26 -07:00
Michael Stack 33eadb5bb1 HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws BindException (#1656)
Move the random free port generate back into hbasecommontestingutility
  from hbasetestingutility.

  Add a create simple kdc server utility that will start a kdc server and
  if a bindexception, create a new one on a new random port in hbase-common.

  Add new BoundSocketMaker helpful when trying to manufacture
  BindExceptions because of port clash.

  Change thrift and http kdc tests to use this new utility (removes
  code duplication around kdc server setup).
2020-05-05 21:36:59 -07:00
stack 24c995da97 HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash
Addendum; add back bit missing off end of test.
2020-05-05 21:16:33 -07:00
Yu Wang 3d96007c37
HBASE-24137:The max merge count of metafixer may be remind in hbase-site.xml (#1478) 2020-05-05 16:14:14 -07:00
Michael Stack 5488124be0 HBASE-24284 [h3/jdk11] REST server won't start Exclude transitive includes of jax-rs 1.x and then explicitly include jax-rs 2.x glassfish impl for REST context when hadoop3. (#1625) 2020-05-05 15:36:01 -07:00
Michael Stack 6613517933 HBASE-24303 Undo core of parent TestSecureRESTServer change; use fix over in HBASE-24280 instead (#1630) 2020-05-05 15:31:28 -07:00
Andrew Purtell 404849ab36
HBASE-24322 UnsafeAvailChecker should also check that required methods are available (#1650)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-05-05 12:07:56 -07:00
huaxiangsun a240ba8214
HBASE-24273 HBCK's "Orphan Regions on FileSystem" reports regions with referenced HFiles (#1613)
Signed-off-by: stack <stack@apache.org>
2020-05-05 11:10:02 -07:00
Duo Zhang 3340c0024e
HBASE-24314 Some classes still use log4j logger directly (#1637)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-05 20:03:55 +08:00