Commit Graph

17922 Commits

Author SHA1 Message Date
Zheng Wang 29c59c0ba8 HBASE-24423 No need to get lock in canSplit because hasReferences will get lock too 2020-05-25 22:52:38 -07:00
WenFeiYi 285e59af25
HBASE-24427 HStore.add log format error (#1773)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-25 18:28:12 +05:30
Viraj Jasani 301659dcc7
HBASE-24426 : Provide regionName in warning during failure to move region from CLOSING to CLOSED (#1772)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-25 17:33:15 +05:30
Guangxu Cheng 92673342ec
HBASE-24417 update copyright notices year to 2020 (#1760)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com> 
Signed-off-by: Clara
2020-05-25 12:58:29 +08:00
wenbang 006e4d5e46
HBASE-24401 Cell size limit check on append should consider 0 or less value to disable the check (#1742)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-05-25 09:58:01 +08:00
niuyulin 1be583f021
HBASE-24387 TableSnapshotInputFormatImpl support row limit on each InputSplit (#1731)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-05-25 09:55:48 +08:00
huaxiangsun eaa2762a4e
HBASE-24369 Provide more information about merged child regions in Hbck Overlaps section, which cannot be fixed immediately (#1756)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Clara Xiong
Signed-off-by: stack <stack@apache.org>
2020-05-24 17:21:04 -07:00
Bharath Vissapragada 2382f90679
HBASE-24422: Remove hbase-native-client code from the master branch (#1766)
There is not much value in having this code in the master branch.
This code has it's dedicated git repo at
https://github.com/apache/hbase-native-client.

It was already removed from branch-1 (HBASE-19419) and branch-2
(HBASE-19176)

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-05-24 16:19:58 -07:00
Duo Zhang c303f9d329
HBASE-24408 Introduce a general 'local region' to store data on master (#1753)
Signed-off-by: stack <stack@apache.org>
2020-05-23 15:59:51 +08:00
Sean Busbey a9fefd7f53
HBASE-24297 release scripts should be able to use a custom git repo
* adds a optional -r [repo] arg
* if the passed repo is on the local filesystem, creates a container mount
* when cloning a local repo configure git to share objects with the local repo instead of copying
* when cloning a local repo in a container configure the clone to have a remote that will work back on the host.

closes #1725

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-05-22 22:24:12 -05:00
Sean Busbey 8ff8e70edf
HBASE-24296 install yetus as a part of building the rm docker image.
* non-zero exit for usage
* in non-docker mode prompt for yetus install location
* make sure we exit if YETUS_HOME does not actually point at a yetus install.

closes #1726

Signed-off-by: Matt Foley <mattf@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-05-22 22:24:03 -05:00
huaxiangsun 2b894dc100
HBASE-24379 CatalogJanitor misreports region holes when there are actually over laps. (#1741)
Signed-off-by: stack <stack@apache.org>
2020-05-22 16:55:54 -07:00
bsglz 0ce85fe63c
HBASE-24407 Correct the comment of clusterRegionLocationMocks in TestStochasticLoadBalancer (#1750) 2020-05-22 14:14:59 -07:00
Viraj Jasani e1d7dc82ea
HBASE-24405 : Document hbase:slowlog related operations (#1747)
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-05-22 18:19:57 +05:30
Wellington Ramos Chevreuil 9d9f07be36
HBASE-24413 HBASE-22259 Removed deprecated getTimeStampOfLastShippedOp method from ReplicationLoadSource, but there were still references to this method on ruby admin.rb (#1754)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
2020-05-22 10:46:33 +01:00
huaxiangsun 80b64ef4dc
HBASE-24376 MergeNormalizer is merging non-adjacent regions and causing region overlaps/holes. (#1734)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-05-21 10:08:43 -07:00
huaxiangsun a8724e8120
HBASE-24370 Avoid aggressive MergeRegion and GCMultipleMergedRegionsProcedure (#1719)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-20 14:18:58 -07:00
Viraj Jasani 86a2692dc4
HBASE-23938 : System table hbase:slowlog to store complete slow/large… (#1681)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
2020-05-20 15:10:29 +05:30
Nick Dimiduk 5fb9e518ef HBASE-24361 Make `RESTApiClusterManager` more resilient (#1701)
* sometimes API calls return with null/empty response bodies. thus,
  wrap all API calls in a retry loop.
* calls that submit work in the form of "commands" now retrieve the
  commandId from successful command submission, and track completion
  of that command before returning control to calling context.
* model CM's process state and use that model to guide state
  transitions more intelligently. this guards against, for example,
  the start command failing with an error message like "Role must be
  stopped".
* improvements to logging levels, avoid spamming logs with the
  side-effects of retries at this and higher contexts.
* include references to API documentation, such as it is.

Signed-off-by: stack <stack@apache.org>
2020-05-19 09:53:22 -07:00
niuyulin 8f4c255b38
HBASE-24386 TableSnapshotScanner support scan limit (#1724)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
2020-05-19 20:52:28 +05:30
Nick Dimiduk 0dae377f53 HBASE-24360 RollingBatchRestartRsAction loses track of dead servers
`RollingBatchRestartRsAction` doesn't handle failure cases when
tracking its list of dead servers. The original author believed that a
failure to restart would result in a retry. However, by removing the
dead server from the failed list, that state is lost, and retry never
occurs. Because this action doesn't ever look back to the current
state of the cluster, relying only on its local state for the current
action invocation, it never realizes the abandoned server is still
dead. Instead, be more careful to only remove the dead server from the
list when the `startRs` invocation claims to have been successful.

Signed-off-by: stack <stack@apache.org>
2020-05-18 13:01:11 -07:00
Sean Busbey 696894e2cc
HBASE-24393 release scripts should allow for creation of initial CHANGES/RELEASE on new major version.
closes #1728

Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-05-18 12:11:13 -05:00
Wellington Ramos Chevreuil 31cdbeba9c
HBASE-24313 [DOCS] Document ignoreTimestamps option added to HashTabl… (#1677)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-05-18 16:57:22 +01:00
ramkrish86 af8398a0ac
HBASE-24365 MetricsTableWrapperAggregateImpl runnable fails due to exception and never runs (#1729)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
2020-05-18 15:11:02 +05:30
binlijin 6ce1e140ee
HBASE-24377 MemStoreFlusher throw NullPointerException (#1721)
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-18 09:58:28 +08:00
Baiqiang Zhao 61efb6d5ad
HBASE-24381 The Size metrics in Master Webui is wrong if the size is 0 (#1723) 2020-05-18 09:14:36 +08:00
Viraj Jasani 15627bb722
HBASE-24327 : Flaky connection in TestMasterShutdown (ADDENDUM) 2020-05-16 12:29:16 +05:30
meiyi a73132c62b
HBASE-24364 [Chaos Monkey] Invalid data block encoding in ChangeEncodingAction (#1707)
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-05-15 11:30:44 +08:00
Guanghao Zhang bab4b1cf8c HBASE-23702 hbase.hstore.flusher.count setting to 0 breaks HMaster (#1703)
Co-authored-by: Ctest
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-05-15 09:21:53 +08:00
Andrew Purtell 94f36fdb8e
Amend HBASE-24350: Extending and Fixing HBaseTable level replication metrics (#1704)
- Rename WALEntryBatch#getWaEntriesWithSize -> getWalEntriesWithSize
2020-05-14 13:42:48 -07:00
Nick Dimiduk 7e295e767f HBASE-24271 Set values in `conf/hbase-site.xml` that enable running on `LocalFileSystem` out of the box
Simplify the new user experience shipping a configuration that enables
a fresh checkout or tarball distribution to run in standalone mode
without direct user configuration. This change restores the behavior
we had when running on Hadoop 2.8 and earlier.

Patch for master includes an update to the book. This change will be
omitted when backporting to earlier branches.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-14 13:18:43 -07:00
Nick Dimiduk 8148c5858b Revert "HBASE-24086 Disable output stream capability enforcement when running on LocalFileSystem"
This reverts commit 10cbb3fb73.
2020-05-14 13:18:43 -07:00
Nick Dimiduk a602a00b8d Revert "HBASE-24106 Update getting started documentation after HBASE-24086"
This reverts commit 7de861bb83.
2020-05-14 13:18:43 -07:00
stack 32e2682310 HBASE-24368 Let HBCKSCP clear 'Unknown Servers', even if RegionStateNode has RegionLocation == null
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
 Edit a log.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/HBCKServerCrashProcedure.java
 Add override of isMatchingRegionLocation. Allow 'null' as a pass in
 HBCKSCP.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
 Add a method for HBCKSCP to override and be less strict filtering
 assigns.

hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
 Some doc on what 'Unknown Servers' are.
2020-05-14 13:09:03 -07:00
Sandeep Pal 2e5a664233
HBASE-24350: Extending and Fixing HBaseTable level replication metrics (#1704)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-05-14 10:34:51 -07:00
bsglz 5e32e08782
HBASE-24164 Retain the ReadRequests and WriteRequests of region on we… (#1500)
Signed-off-by: stack <stack@apache.org>
2020-05-13 22:28:45 -07:00
Mingliang Liu a40a0322a7
HBASE-23969 Meta browser should show all `info` columns (#1485)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-13 11:50:53 -07:00
Viraj Jasani a782531633
HBASE-24327 : Flaky connection in TestMasterShutdown#testMasterShutdo… (#1690)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-13 02:38:47 +05:30
Michael Stack dc360f0f41
HBASE-24354 Make it so can make an hbase1 schema and hbase2 schema equate (#1696) 2020-05-12 12:31:11 -07:00
Sambit Mohapatra c4e892e3f1
HBASE-23832 Old config hbase.hstore.compactionThreshold is ignored
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2020-05-12 23:19:32 +05:30
Gkkkk302 9e73244082
HBASE-24355 Fix typos in the HStore#compact annotation (#1699)
Co-authored-by: guo.kangkang <guo.kangkang@immomo.com>

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-12 21:07:49 +05:30
Duo Zhang 8601416ee8
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for modules other than hbase-assembly (#1640)
Signed-off-by: stack <stack@apache.org>
2020-05-12 12:03:30 +08:00
stack bb3191c563 HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP Addendum2: make same fix for TestRaceBetweenSCPAndTRSP but in a second place (addendum 1 got first place). 2020-05-11 20:48:21 -07:00
stack 2c11d5a0a0 HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP
Addendum: make same fix for TestRaceBetweenSCPAndTRSP which does same
thing.
2020-05-11 14:06:56 -07:00
Joseph295 68229c9c99
HBASE-24341 The region should be removed from ConfigurationManager as a ConfigurationObserver when it is closed (#1675)
Co-authored-by: Joseph295 <johnhomsea@gmail.com>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2020-05-11 17:31:54 +05:30
Viraj Jasani 6729cafa1d
Revert "HBASE-24327 : Handle shutdown() if master cannot be contacted (#1684)" (#1691)
This reverts commit 34a74534fe.
2020-05-11 16:43:49 +05:30
Geoffrey Jacoby ca81283fe5
HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S… (#1655)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lars Hofhansl <larsh@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-11 12:54:22 +05:30
shahrs87 ff85daf1e2
[HBASE-24190] Make kerberos value of hbase.security.authentication property case insensitive (#1687)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-11 10:54:45 +05:30
Viraj Jasani 34a74534fe
HBASE-24327 : Handle shutdown() if master cannot be contacted (#1684)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-10 17:09:51 +05:30
Matt Foley f3c0a0e06d
HBASE-24318 Create-release scripts fixes and enhancements (#1643)
* narrow 'dry-run' limits so see svn activity up to just before check-in.
* Fix several typos and, in case of dry run, enable build step to chain from tag step by keeping the tagged git repo.
* Improve Maven settings.xml file, and names of variables related to it. Remove unnecessary use of "-Dmaven.repo.local", put it in settings.xml instead. Stop putting password literals in settings.xml. 
* stop deleting maven settings file, now that it doesn't contain password strings
* Merge 'tag' into release-build.sh as another stage. Also found and fixed a couple bugs in the current release-tag.sh.
* Delete redundant release-tag.sh script.
* Small changes to make dev-support/create-release tools less focused
on hbase project only, while retaining special behaviors for
hbase sub-projects which share the hbase dist and jira locations.
* Changed terminology of release publish steps, from <build|publish>
to <publish-dist|publish-release>. In fact, what was formerly called
"build" built the distribution tarballs AND published them to dist,
while "publish" built the  maven artifacts AND published them to Nexus.
The new terminology clarifies what's happening, and removes the appearance
of order dependendency.
* Fix publish-snapshot so it does same checks as publish-release.
* Factor out common maven usages, and move them to build-util.sh.
* Change default polarity of DRY_RUN to default to true. Change -n ("no-publish") to -f ("force publish") to actually publish.
* Fix problems in do-release.sh so it runs correctly outside of docker,
  including DRY_RUN being exported.
* Have do-release.sh set REPO (shared maven local repository) if doing all three stages.
* Cleaned up REPO directory creation.
* General cleanup of comments and usage.
*  fix all 'shellcheck' errors
* use ${BASH_SOURCE[0]} instead of $0 to determine script directory path
* smarter way to read version from pom with mvn
* do maven-gpg-plugin config settings in maven settings file correctly as documented
* fix gpg signing failure on Mac due to gpg-agent timeout
* fix various bugs to enable publish-dist, publish-snapshot, and publish-release to work correctly as individual steps and/or without docker
* improve log reporting from publish-release step
*  fix bug in argument to checkcompatibility.py: replace PACKAGE_VERSION with GIT_REF
* demote "PACKAGE_VERSION" to "package_version_name" and undocument it in favor of RELEASE_TAG. Still enable appropriate defaulting in case RELEASE_TAG is undefined.
* unify RELEASE_VERSION with VERSION, to remove ambiguity and allow it to be set when only running 'publish' step without 'tag'
* query confirm RELEASE_TAG
* emphasize that release-build.sh is called for a single action at a time, and should be called from do-release.sh.
* add '-s' option to do-release.sh
* suppress maven "Download from central:" messages
* Replace human wait for tag propagation

Signed-off-by: Cesar Delgado <cdelgado@apple.com>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-05-09 10:46:41 -05:00