Commit Graph

17754 Commits

Author SHA1 Message Date
Nick Dimiduk 5fb9e518ef HBASE-24361 Make `RESTApiClusterManager` more resilient (#1701)
* sometimes API calls return with null/empty response bodies. thus,
  wrap all API calls in a retry loop.
* calls that submit work in the form of "commands" now retrieve the
  commandId from successful command submission, and track completion
  of that command before returning control to calling context.
* model CM's process state and use that model to guide state
  transitions more intelligently. this guards against, for example,
  the start command failing with an error message like "Role must be
  stopped".
* improvements to logging levels, avoid spamming logs with the
  side-effects of retries at this and higher contexts.
* include references to API documentation, such as it is.

Signed-off-by: stack <stack@apache.org>
2020-05-19 09:53:22 -07:00
niuyulin 8f4c255b38
HBASE-24386 TableSnapshotScanner support scan limit (#1724)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
2020-05-19 20:52:28 +05:30
Nick Dimiduk 0dae377f53 HBASE-24360 RollingBatchRestartRsAction loses track of dead servers
`RollingBatchRestartRsAction` doesn't handle failure cases when
tracking its list of dead servers. The original author believed that a
failure to restart would result in a retry. However, by removing the
dead server from the failed list, that state is lost, and retry never
occurs. Because this action doesn't ever look back to the current
state of the cluster, relying only on its local state for the current
action invocation, it never realizes the abandoned server is still
dead. Instead, be more careful to only remove the dead server from the
list when the `startRs` invocation claims to have been successful.

Signed-off-by: stack <stack@apache.org>
2020-05-18 13:01:11 -07:00
Sean Busbey 696894e2cc
HBASE-24393 release scripts should allow for creation of initial CHANGES/RELEASE on new major version.
closes #1728

Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-05-18 12:11:13 -05:00
Wellington Ramos Chevreuil 31cdbeba9c
HBASE-24313 [DOCS] Document ignoreTimestamps option added to HashTabl… (#1677)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-18 16:57:22 +01:00
ramkrish86 af8398a0ac
HBASE-24365 MetricsTableWrapperAggregateImpl runnable fails due to exception and never runs (#1729)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
2020-05-18 15:11:02 +05:30
binlijin 6ce1e140ee
HBASE-24377 MemStoreFlusher throw NullPointerException (#1721)
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-18 09:58:28 +08:00
Baiqiang Zhao 61efb6d5ad
HBASE-24381 The Size metrics in Master Webui is wrong if the size is 0 (#1723) 2020-05-18 09:14:36 +08:00
Viraj Jasani 15627bb722
HBASE-24327 : Flaky connection in TestMasterShutdown (ADDENDUM) 2020-05-16 12:29:16 +05:30
meiyi a73132c62b
HBASE-24364 [Chaos Monkey] Invalid data block encoding in ChangeEncodingAction (#1707)
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-05-15 11:30:44 +08:00
Guanghao Zhang bab4b1cf8c HBASE-23702 hbase.hstore.flusher.count setting to 0 breaks HMaster (#1703)
Co-authored-by: Ctest
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-05-15 09:21:53 +08:00
Andrew Purtell 94f36fdb8e
Amend HBASE-24350: Extending and Fixing HBaseTable level replication metrics (#1704)
- Rename WALEntryBatch#getWaEntriesWithSize -> getWalEntriesWithSize
2020-05-14 13:42:48 -07:00
Nick Dimiduk 7e295e767f HBASE-24271 Set values in `conf/hbase-site.xml` that enable running on `LocalFileSystem` out of the box
Simplify the new user experience shipping a configuration that enables
a fresh checkout or tarball distribution to run in standalone mode
without direct user configuration. This change restores the behavior
we had when running on Hadoop 2.8 and earlier.

Patch for master includes an update to the book. This change will be
omitted when backporting to earlier branches.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-14 13:18:43 -07:00
Nick Dimiduk 8148c5858b Revert "HBASE-24086 Disable output stream capability enforcement when running on LocalFileSystem"
This reverts commit 10cbb3fb73.
2020-05-14 13:18:43 -07:00
Nick Dimiduk a602a00b8d Revert "HBASE-24106 Update getting started documentation after HBASE-24086"
This reverts commit 7de861bb83.
2020-05-14 13:18:43 -07:00
stack 32e2682310 HBASE-24368 Let HBCKSCP clear 'Unknown Servers', even if RegionStateNode has RegionLocation == null
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
 Edit a log.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/HBCKServerCrashProcedure.java
 Add override of isMatchingRegionLocation. Allow 'null' as a pass in
 HBCKSCP.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
 Add a method for HBCKSCP to override and be less strict filtering
 assigns.

hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
 Some doc on what 'Unknown Servers' are.
2020-05-14 13:09:03 -07:00
Sandeep Pal 2e5a664233
HBASE-24350: Extending and Fixing HBaseTable level replication metrics (#1704)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-05-14 10:34:51 -07:00
bsglz 5e32e08782
HBASE-24164 Retain the ReadRequests and WriteRequests of region on we… (#1500)
Signed-off-by: stack <stack@apache.org>
2020-05-13 22:28:45 -07:00
Mingliang Liu a40a0322a7
HBASE-23969 Meta browser should show all `info` columns (#1485)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-13 11:50:53 -07:00
Viraj Jasani a782531633
HBASE-24327 : Flaky connection in TestMasterShutdown#testMasterShutdo… (#1690)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-13 02:38:47 +05:30
Michael Stack dc360f0f41
HBASE-24354 Make it so can make an hbase1 schema and hbase2 schema equate (#1696) 2020-05-12 12:31:11 -07:00
Sambit Mohapatra c4e892e3f1
HBASE-23832 Old config hbase.hstore.compactionThreshold is ignored
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2020-05-12 23:19:32 +05:30
Gkkkk302 9e73244082
HBASE-24355 Fix typos in the HStore#compact annotation (#1699)
Co-authored-by: guo.kangkang <guo.kangkang@immomo.com>

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-12 21:07:49 +05:30
Duo Zhang 8601416ee8
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for modules other than hbase-assembly (#1640)
Signed-off-by: stack <stack@apache.org>
2020-05-12 12:03:30 +08:00
stack bb3191c563 HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP Addendum2: make same fix for TestRaceBetweenSCPAndTRSP but in a second place (addendum 1 got first place). 2020-05-11 20:48:21 -07:00
stack 2c11d5a0a0 HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP
Addendum: make same fix for TestRaceBetweenSCPAndTRSP which does same
thing.
2020-05-11 14:06:56 -07:00
Joseph295 68229c9c99
HBASE-24341 The region should be removed from ConfigurationManager as a ConfigurationObserver when it is closed (#1675)
Co-authored-by: Joseph295 <johnhomsea@gmail.com>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2020-05-11 17:31:54 +05:30
Viraj Jasani 6729cafa1d
Revert "HBASE-24327 : Handle shutdown() if master cannot be contacted (#1684)" (#1691)
This reverts commit 34a74534fe.
2020-05-11 16:43:49 +05:30
Geoffrey Jacoby ca81283fe5
HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S… (#1655)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lars Hofhansl <larsh@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-11 12:54:22 +05:30
shahrs87 ff85daf1e2
[HBASE-24190] Make kerberos value of hbase.security.authentication property case insensitive (#1687)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-11 10:54:45 +05:30
Viraj Jasani 34a74534fe
HBASE-24327 : Handle shutdown() if master cannot be contacted (#1684)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-10 17:09:51 +05:30
Matt Foley f3c0a0e06d
HBASE-24318 Create-release scripts fixes and enhancements (#1643)
* narrow 'dry-run' limits so see svn activity up to just before check-in.
* Fix several typos and, in case of dry run, enable build step to chain from tag step by keeping the tagged git repo.
* Improve Maven settings.xml file, and names of variables related to it. Remove unnecessary use of "-Dmaven.repo.local", put it in settings.xml instead. Stop putting password literals in settings.xml. 
* stop deleting maven settings file, now that it doesn't contain password strings
* Merge 'tag' into release-build.sh as another stage. Also found and fixed a couple bugs in the current release-tag.sh.
* Delete redundant release-tag.sh script.
* Small changes to make dev-support/create-release tools less focused
on hbase project only, while retaining special behaviors for
hbase sub-projects which share the hbase dist and jira locations.
* Changed terminology of release publish steps, from <build|publish>
to <publish-dist|publish-release>. In fact, what was formerly called
"build" built the distribution tarballs AND published them to dist,
while "publish" built the  maven artifacts AND published them to Nexus.
The new terminology clarifies what's happening, and removes the appearance
of order dependendency.
* Fix publish-snapshot so it does same checks as publish-release.
* Factor out common maven usages, and move them to build-util.sh.
* Change default polarity of DRY_RUN to default to true. Change -n ("no-publish") to -f ("force publish") to actually publish.
* Fix problems in do-release.sh so it runs correctly outside of docker,
  including DRY_RUN being exported.
* Have do-release.sh set REPO (shared maven local repository) if doing all three stages.
* Cleaned up REPO directory creation.
* General cleanup of comments and usage.
*  fix all 'shellcheck' errors
* use ${BASH_SOURCE[0]} instead of $0 to determine script directory path
* smarter way to read version from pom with mvn
* do maven-gpg-plugin config settings in maven settings file correctly as documented
* fix gpg signing failure on Mac due to gpg-agent timeout
* fix various bugs to enable publish-dist, publish-snapshot, and publish-release to work correctly as individual steps and/or without docker
* improve log reporting from publish-release step
*  fix bug in argument to checkcompatibility.py: replace PACKAGE_VERSION with GIT_REF
* demote "PACKAGE_VERSION" to "package_version_name" and undocument it in favor of RELEASE_TAG. Still enable appropriate defaulting in case RELEASE_TAG is undefined.
* unify RELEASE_VERSION with VERSION, to remove ambiguity and allow it to be set when only running 'publish' step without 'tag'
* query confirm RELEASE_TAG
* emphasize that release-build.sh is called for a single action at a time, and should be called from do-release.sh.
* add '-s' option to do-release.sh
* suppress maven "Download from central:" messages
* Replace human wait for tag propagation

Signed-off-by: Cesar Delgado <cdelgado@apple.com>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-05-09 10:46:41 -05:00
stack 2774510e2b HBASE-24284 [h3/jdk11] REST server won't start Exclude transitive includes of jax-rs 1.x and then explicitly include jax-rs 2.x glassfish impl for REST context when hadoop3. (#1625)
Addendum to fix failing MR tests; e.g. TestExportSnapshot.
2020-05-08 13:08:57 -07:00
huaxiangsun 30f07e8187
HBASE-24256 When fixOverlap hits the max region limit, it is possible to include the same region in multiple merge request (#1584)
Signed-off-by: stack <stack@apache.org>
2020-05-08 10:16:42 -07:00
Reid Chan 0cd70ed89c
HBASE-24345 [ACL] renameRSGroup should require Admin level permission (#1683)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-08 18:56:05 +08:00
Jan Hentschel 03e5a1402d
HBASE-24305 Removed deprecations in ServerName (#1666)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-05-08 09:41:55 +02:00
Duo Zhang fddb2dd65c
HBASE-24310 Use Slf4jRequestLog for hbase-http (#1634)
Signed-off-by: stack <stack@apache.org>
2020-05-08 11:16:18 +08:00
stack fc283f7a68 HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP #1669
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-07 13:54:45 -07:00
stack d30268d0ee HBASE-24342 [Flakey Tests] Disable TestClusterPortAssignment.testClusterPortAssignment as it can't pass 100% of the time 2020-05-07 13:22:35 -07:00
Nick Dimiduk 0e81ab08b9 HBASE-24295 [Chaos Monkey] abstract logging through the class hierarchy ; ADDENDUM
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-07 13:20:15 -07:00
bsglz 2cafe81e9c
HBASE-24335 Support deleteall with ts but without column in shell mode (#1668)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-05-07 18:18:57 +01:00
niuyulin f4a446c3d2
HBASE-24328 skip duplicate GCMultipleMergedRegionsProcedure while previous finished (#1629)
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-05-06 10:06:49 -07:00
Michael Stack 6f059ecff0 HBASE-24331 [Flakey Test] TestJMXListener rmi port clash (#1657)
Add check that we can make jmx connector in setup. If we can't retry.
2020-05-06 09:29:57 -07:00
Jan Hentschel 4ac1719521
HBASE-24334 Removed duplicated dependency declaration of metrics-api (#1665)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-06 14:22:12 +02:00
Reid Chan c32d18d4fb
HBASE-24336 [Metrics] FSDataInputStream's localBytesRead is wrong (#1667)
Signed-off-by: binlijin <binlijin@gmail.com>
2020-05-06 19:24:05 +08:00
Mohammad Arshad 294c228c67
HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1631)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-06 15:12:20 +05:30
Duo Zhang c1cb22f0b3
HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry … (#1593)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-06 15:55:26 +08:00
Duo Zhang a9a1b9524d
HBASE-24304 Separate a hbase-asyncfs module (#1628)
Signed-off-by: stack <stack@apache.org>
2020-05-06 14:40:21 +08:00
huaxiangsun 045c909bdf
HBASE-24316 GCMulitpleMergedRegionsProcedure is not idempotent (#1660)
It addresses couple issues:
   1. Make sure deleteMergeQualifiers() does not delete the row if there is no columns with "merge" keyword.
   2. GCMulitpleMergedRegionsProcedure now acquire an exclusive lock on the child region.

Signed-off-by: stack <stack@apache.org>
2020-05-05 23:04:48 -07:00
Nick Dimiduk 07077a3950
HBASE-23976 Use nio for file manipulation in TestVerifyBucketCacheFile (#1342)
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-05-05 22:04:26 -07:00