Commit Graph

1579 Commits

Author SHA1 Message Date
Andrew Purtell b67f896954
HBASE-24367 ScheduledChore log elapsed timespan in a human-friendly format (#1749)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-09 09:57:03 -07:00
XinSun 67fa367bb6 HBASE-24359 Optionally ignore edits for deleted CFs for replication (#1855)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-08 13:14:50 +08:00
Nick Dimiduk 4884773676 HBASE-24418 Consolidate Normalizer implementations
Simplify our Normalizer story to have just a single, configurable
implementation.

* fold the features of `MergeNormalizer` into
  `SimpleRegionNormalizer`, removing the intermediate abstract class.
* configuration keys for merge-only features now share a common
  structure.
* add configuration to selectively disable normalizer split/merge
  operations.
* `RegionNormalizer` now extends `Configurable` instead of creating a
  new instance of `HBaseConfiguration` or snooping one off of other
  fields.
* avoid the extra RPCs by using `MasterServices` instead of
  `MasterRpcServices`.
* boost test coverage of all the various flags and feature
  combinations.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-06-03 16:43:40 -07:00
niuyulin 571ef7e811 HBASE-24412 Canary support check only one column family per RegionTask (#1817)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-03 09:16:11 +08:00
Bharath Vissapragada d7f388518c
HBASE-24477: Move ConfigurationObserver and related classes to hbase-common
This utility is useful for any module that wants to detect
dynamic config changes. Having it to hbase-common makes it
accessible to all the other modules.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-31 14:14:30 -07:00
bsglz 22efe67758
HBASE-24451 Remove the HasThread because the related bug had been fixed since jdk7 (#1797)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-29 17:46:03 +05:30
Duo Zhang 58eed9a4bb HBASE-24408 Introduce a general 'local region' to store data on master (#1753)
Signed-off-by: stack <stack@apache.org>
2020-05-23 16:40:39 +08:00
Viraj Jasani 2383cc4242
HBASE-24405 : Document hbase:slowlog related operations (#1747)
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-05-22 20:58:04 +05:30
Viraj Jasani 6f5e5e4828
HBASE-23938 : System table hbase:slowlog to store complete slow/large… (#1681)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
2020-05-20 15:30:30 +05:30
Mingliang Liu 5893c4f393
HBASE-23969 Meta browser should show all `info` columns (#1710)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-18 11:48:36 -07:00
Nick Dimiduk 1c32a8ba2e HBASE-24271 Set values in `conf/hbase-site.xml` that enable running on `LocalFileSystem` out of the box
Simplify the new user experience shipping a configuration that enables
a fresh checkout or tarball distribution to run in standalone mode
without direct user configuration. This change restores the behavior
we had when running on Hadoop 2.8 and earlier.

Patch for master includes an update to the book. This change will be
omitted when backporting to earlier branches.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-14 14:05:40 -07:00
Nick Dimiduk 2ab394ddb1 Revert "HBASE-24086 Disable output stream capability enforcement when running on LocalFileSystem"
This reverts commit 4e31ada43b.
2020-05-14 14:05:40 -07:00
shahrs87 941cef71c2
HBASE-24190 : Make kerberos value of hbase.security.authentication property case insensitive (#1687)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-14 11:46:33 +05:30
Viraj Jasani b78f4367f7
Revert "Make kerberos value of hbase.security.authentication property case insensitive (#1687)"
This reverts commit c03ec837e7.
2020-05-14 11:45:23 +05:30
Duo Zhang dc2146069c
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for … (#1697)
Signed-off-by: stack <stack@apache.org>
2020-05-13 17:59:21 +08:00
Sambit Mohapatra d85caaa195
HBASE-23832 Old config hbase.hstore.compactionThreshold is ignored (#1698)
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2020-05-12 23:20:48 +05:30
shahrs87 c03ec837e7
Make kerberos value of hbase.security.authentication property case insensitive (#1687)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-11 11:10:29 +05:30
Duo Zhang 7f4683bafa HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry … (#1593)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-06 16:11:28 +08:00
Michael Stack 528e9d6454
HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws BindException (#1656)
Move the random free port generate back into hbasecommontestingutility
  from hbasetestingutility.

  Add a create simple kdc server utility that will start a kdc server and
  if a bindexception, create a new one on a new random port in hbase-common.

  Add new BoundSocketMaker helpful when trying to manufacture
  BindExceptions because of port clash.

  Change thrift and http kdc tests to use this new utility (removes
  code duplication around kdc server setup).
2020-05-05 21:08:09 -07:00
Andrew Purtell 5dc10dbba5
HBASE-24322 UnsafeAvailChecker should also check that required methods are available (#1650)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-05-05 12:11:37 -07:00
Duo Zhang 1f556b3ad9 HBASE-24281 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:25:41 +08:00
Duo Zhang 379b48bb5d Revert "HBASE-24181 Remove some stale hadoop.version properties in the pom for sub modules (#1602)"
This reverts commit dc9608ec0b.
2020-04-29 18:25:23 +08:00
Duo Zhang dc9608ec0b HBASE-24181 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:08:33 +08:00
Duo Zhang 922921ee5f HBASE-24249 Move code in FSHDFSUtils to FSUtils and mark related clas… (#1586)
Signed-off-by: stack <stack@apache.org>
2020-04-29 11:31:32 +08:00
Viraj Jasani b40381857a
HBASE-24024 : Reject multi() requests with rows higher than threshold (#1560)
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-04-22 12:51:49 +05:30
Mark Robert Miller 8a7aa89506
HBASE-24185: Junit tests do not behave well with System.exit or Runtime.halt or JVM exits in general. (#1540)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-20 22:20:08 -07:00
Viraj Jasani ca1f7a9499
HBASE-24194 : Refactor anonymous inner classes of BufferedEncodedSeeker to named inner classes (#1522)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
2020-04-16 19:17:44 +05:30
stack ba87ffed09 Revert "HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create new native thread."
Revert because applied with wrong JIRA number.

This reverts commit 9c60c5fddb.
2020-04-07 10:17:48 -07:00
stack 9c60c5fddb HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create new native thread.
Up the nproc in hbase-personality. Add test that prints out env.
2020-04-07 07:37:48 -07:00
Nick Dimiduk 4e31ada43b HBASE-24086 Disable output stream capability enforcement when running on LocalFileSystem
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-03 09:24:29 -07:00
Duo Zhang 7d842ca886
HBASE-24050 Deprecated PBType on all 2.x branches (#1354)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-03-27 10:43:14 +08:00
Duo Zhang 30eba2c24e HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0 (#1335)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-03-26 18:10:03 +08:00
Nick Dimiduk ffb2359146
HBASE-24013 Bump branch-2 version to 2.4.0-SNAPSHOT (#1309)
Increment version in poms with

```
$ mvn org.codehaus.mojo:versions-maven-plugin:2.7:set -DnewVersion=2.4.0-SNAPSHOT -DgenerateBackupPoms=false
```

Verified no dangling references with

```
$ find . -iname '*pom.xml' -exec grep -n '2.3.0-SNAPSHOT' {} +
```

Verified build with

```
$ JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home mvn clean package -DskipTests
$ JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home mvn clean package -DskipTests -Dhadoop.profile=3.0
```

Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-03-19 08:01:43 -07:00
Nick Dimiduk 33cb9fc932 HBASE-23829 Get `-PrunSmallTests` passing on JDK11
Signed-off-by: stack <stack@apache.org>
2020-03-17 09:01:02 -07:00
Duo Zhang 7eeb6a0815 HBASE-23077 move entirely to spotbugs (#1265)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-03-12 11:42:23 +08:00
Aman Poonia c9c22fedaa HBASE-22285 A normalizer which merges small size regions with adjacent regions (#978)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-03-10 15:47:18 -07:00
Michael Stack a6700bc6b5
HBASE-23851 Log networks and bind addresses when multicast publisher/listener enabled (#1173)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-03-10 10:55:44 -07:00
Anoop Sam John 9ff3fe11c4
HBASE-23788 ROW_INDEX_V1 encoder should consider the secondary index size with the encoded data size tracking (#1241)
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2020-03-06 06:15:15 +05:30
Viraj Jasani 6366b73134
HBASE-22978 : Online slow response log (#1228)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-03-01 00:32:35 +05:30
Bharath Vissapragada 5848b7d835 HBASE-23804: Fix default master addr hostname in master registry (#1137)
Master rpc server end point doesn't bind to localhost's
IP address by default. Instead, it looks up the hostname and
binds to the endpoint to which it resolves. MasterRegistry should
do the same when building the default server end point to talk to.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
(cherry picked from commit c34dbc3c29)
2020-02-27 10:33:42 -08:00
Bharath Vissapragada bd13416275 HBASE-23731: De-flake TestFromClientSide (#1091)
There were a couple of issues.

- There was a leak of a file descriptor for hbck lock file. This
was contributing to all the "ConnectionRefused" stack traces since
it was trying to renew lease for an already expired mini dfs cluster.
This issue was there for a while, just that we noticed it now.

- After upgrade to JUnit 4.13, it looks like the behavior for test
timeouts has changed. Earlier the timeout seems to have applied for
each parameterized run, but now it looks like it is applied across
all the runs.

This patch fixes both the issues.

Signed-off-by: Stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
(cherry picked from commit 5c88672d54)
2020-02-27 10:33:42 -08:00
Bharath Vissapragada 69e3e0e2ef HBASE-23647: Make MasterRegistry the default impl. (#1039)
Signed-off-by: Stack <stack@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 229b8aaaf3)
2020-02-27 10:33:42 -08:00
Bharath Vissapragada be30d43a6c HBASE-23305: Master based registry implementation (#954)
Implements a master based registry for clients.

 - Supports hedged RPCs (fan out configured via configs).
 - Parameterized existing client tests to run with multiple registry combinations.
 - Added unit-test coverage for the new registry implementation.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 62da419b23)
2020-02-27 10:33:42 -08:00
stack 8e11a5caa0 HBASE-23789 [Flakey Tests] ERROR [Time-limited test] balancer.HeterogeneousRegionCountCostFunction(199): cannot read rules file located at ' /tmp/hbase-balancer.rules ' 2020-02-04 16:37:56 -08:00
Mark Robert Miller e385fd97e0 HBASE-23783: Address tests writing and reading SSL/Security files in a common location. (#1116)
This is causing me issues with parallel test runs.

Also allow setting the surefire reports and temp directories via command line.

Signed-off-by: stack <stack@apache.org>
2020-02-04 14:40:52 -08:00
Michael Stack 12f4e0977c
HBASE-23780 Edit of test classifications (#1109)
These classifications come of running at various fork counts.. A test
may complete quick if low fork count but if it is accessing disk, it
will run much slower if fork count is high. This edit accommodates
some of this phenomenon.


Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-02-03 09:16:47 -08:00
Bharath Vissapragada 4de06915b8 HBASE-23764: Switch to IP address for ZK ensemble #1106
Tests showed that using IP address is much faster for mini cluster
and a lot less flakier. See the jira for more details.
2020-02-01 15:13:54 -08:00
stack 62875a229a Revert "Revert "HBASE-23705 Add CellComparator to HFileContext (#1062)""
Revert of a revert, i.e., putting this patch back.

This reverts commit f4b0ad9f6a.
2020-01-27 07:40:56 -08:00
stack f4b0ad9f6a Revert "HBASE-23705 Add CellComparator to HFileContext (#1062)"
This reverts commit f817293fb4.
2020-01-26 20:49:03 -08:00
Jan Hentschel bfa4b0c4c1
HBASE-23686 Revert binary incompatible change in ByteRangeUtils and removed reflections in CommonFSUtils
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-01-24 20:28:01 +01:00
Michael Stack f817293fb4 HBASE-23705 Add CellComparator to HFileContext (#1062)
Codecs don't have access to what CellComparator to use.  Backfill.

M hbase-common/src/main/java/org/apache/hadoop/hbase/CellComparator.java
 Adds a new compareRows with default implementation that takes a ByteBuffer.
 Needed by the index in a block encoder implementation.

M hbase-common/src/main/java/org/apache/hadoop/hbase/CellComparatorImpl.java
 Adds implementation for meta of new compareRows method. Adds utility
 method for figuring comparator based off tablename.

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/AbstractDataBlockEncoder.java
M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java
M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/RowIndexCodecV1.java
M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/RowIndexSeekerV1.java
 Comparator is in context. Remove redundant handling.

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java
 Comparator is in context. Remove redundant handling. Clean javadoc.

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDecodingContext.java
 Clean javadoc.

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/RowIndexEncoderV1.java
 Cache context so can use it to get comparator to use later.

M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileContext.java
 Cache cellcomparator to use. Javdoc on diff between HFileContext and
 HFileInfo.
M hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileContextBuilder.java
 Add CellComparator

M hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterImpl.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileWriter.java
 Remove comparator caching. Get from context instead.

M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/FixedFileTrailer.java
 Skip a reflection if we can.

M hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileInfo.java
 Javadoc. Removed unused filed.

Signed-off-by: Anoop Sam John <anoopsamjohn@apacher.org>
Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-01-22 20:38:24 -08:00
Michael Stack ac852c2b66
HBASE-23055 Alter hbase:meta (#1043)
Make hbase:meta region schema dynamic.

Patch has been under development a good while and its focus has changed
a few times so its bloated with fixup from older versions.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
 Shut down access to internals and removed unused methods.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
 Cleanup/refactor section on replica-handling.

M hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
 Get hbase:meta schema from filesystem rather than from hard-coding.
2020-01-21 13:17:27 -08:00
Jan Hentschel c16569996b
HBASE-23625 Reduced number of Checkstyle violations in hbase-common
Signed-off-by: stack <stack@apache.org>
2020-01-20 22:24:01 +01:00
Duo Zhang 70c69ba765 HBASE-23680 RegionProcedureStore missing cleaning of hfile archive (#1022)
Signed-off-by: stack <stack@apache.org>
2020-01-18 20:52:29 +08:00
Josh Elser 1ea389a834 HBASE-23701 Try to converge automated checks around Category
We have a compile-time guarantee to either have a category array
of length zero or one because the category annotation is non-repeatable.

Also tries to clean up the checking for unit-test annotations.

Closes #1057

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-01-17 12:49:50 -05:00
Josh Elser bef1eb33f4 HBASE-23347 Allow custom authentication methods for RPCs
Decouple the HBase internals such that someone can implement
their own SASL-based authentication mechanism and plug it into
HBase RegionServers/Masters.

Comes with a design doc in dev-support/design-docs and an example in
hbase-examples known as "Shade" which uses a flat-password file
for authenticating users.

Closes #884

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-01-16 11:04:08 -05:00
Josh Elser 0935ba4e16 HBASE-23695 Fail gracefully if no category is present
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

Closes #1052
2020-01-16 10:24:53 -05:00
Bharath Vissapragada 31b4fbdee5 HBASE-23665: Split unit tests from TestTableName into a separate test-only class. (#1032)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-01-15 13:56:51 -08:00
stack da782e4233 Revert "HBASE-23055 Alter hbase:meta"
This reverts commit d64b0e3612.
2020-01-11 09:21:15 -08:00
stack d64b0e3612 HBASE-23055 Alter hbase:meta
Make it so hbase:meta can be altered. TableState for hbase:meta
    was hardcoded ENABLED. Make it dynamic. State is now kept in
    current active Master. It is transient so falls back to default
    if Master crashes. Add to registry a getMetaTableState
    which reads mirrored state from zookeeper (NOT from Master and
    defaults ENABLED if no implementation or error fetching state).
    hbase:meta schema will be bootstrapped from the filesystem.
    Changes to filesystem schema are atomic so we should be ok if
    Master fails mid-edit (TBD). Undoes a bunch of guards that
    prevented our being able to edit hbase:meta.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     Add fetching of hbase:meta table state from registry. Adds cache of
     tablestates w/ a ttl of 1 second (adjustable).

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
      Add querying registry for hbase:meta table state.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
      Add querying of mirrored table state for hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta. Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-01-09 09:52:21 -08:00
Jan Hentschel 4f7de9e27e
HBASE-23645 Fixed remaining Checkstyle violations in hbase-common tests
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-07 09:07:56 +01:00
Nick Dimiduk f98085e44e HBASE-23333 Include Call.toShortString() in sendCall exceptions 2020-01-03 12:23:37 -08:00
Guanghao Zhang a693a8fd95 HBASE-23286 Improve MTTR: Split WAL to HFile (#820)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-01-03 16:39:06 +08:00
Duo Zhang bee7f4e08c HBASE-23624 Add a tool to dump the procedure info in HFile (#975)
Signed-off-by: stack <stack@apache.org>
2020-01-02 11:32:29 +08:00
Viraj Jasani 17652a7b32
HBASE-23590 : Update maxStoreFileRefCount to maxCompactedStoreFileRefCount for auto region recovery based on old reader references
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-01-01 22:50:37 +05:30
Jan Hentschel 71a0c30caf
HBASE-23626 Reduced number of Checkstyle violations in tests in hbase-common
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2019-12-29 22:42:41 +01:00
Jan Hentschel f3e11d7691
HBASE-23622 Reduced the number of Checkstyle violations in hbase-common
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2019-12-28 20:05:50 +01:00
Jan Hentschel 2382d009a6
HBASE-23621 Reduced the number of Checkstyle violations in tests of hbase-common
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2019-12-28 18:51:33 +01:00
belugabehr d6eec74273 HBASE-23556: Minor ChoreService Cleanup (#927) 2019-12-10 16:21:11 -08:00
chenxu14 46c090ce45 HBASE-23293 [REPLICATION] make ship edits timeout configurable (#882)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2019-11-28 11:01:18 +08:00
Nick Dimiduk d742112ed9
HBASE-23259: Populate master address end points in cluster/rs configs (#807) (#857)
All the clients need to know the master RPC end points while using master
based registry for creating cluster connections. This patch amends the
test cluster utility to populate these configs in the base configuration
object used to spin up the cluster.

The config key added here ("hbase.master.addrs") is used in the subsequent
patches for HBASE-18095.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2019-11-21 09:43:53 -08:00
Sean Busbey baf8849050 HBASE-19450 Addendum Limit logging of chore execution time at INFO to once per 5 minutes.
* Ensure MovingAverage related classes are IA.Private
* Move trace logging into MovingAverage class

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-11-16 09:40:20 -06:00
Reid Chan f5db86ca8f HBASE-19450 Add log about average execution time for ScheduledChore
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-11-14 22:07:02 -06:00
Mingliang Liu d22e6f55d8 HBASE-23283 Provide clear and consistent logging about the period of enabled chores
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-11-14 20:47:01 -06:00
Geoffrey Jacoby da06aa7f1f HBASE-23251 - Add Column Family and Table Names to HFileContext and use in HFileWriterImpl logging (#796)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-11-11 16:22:09 +08:00
Viraj Jasani 29c27e3d04
HBASE-23212 Dynamically reload configs for Region Recovery chore (#802)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-11-06 15:54:57 -08:00
chenxu14 26663585de HBASE-23184 The HeapAllocation in WebUI is not accurate (#730)
Signed-off-by: stack <stack@apache.org>
2019-10-31 17:57:52 +08:00
Michael Stack 471538ca9e HBASE-23221 Polish the WAL interface after HBASE-23181 (#774)
Removes the closeRegion flag added by HBASE-23181 and instead
relies on reading meta WALEdit content. Modified how qualifier is
written when the meta WALEdit is for a RegionEventDescriptor
so the 'type' is added to the qualifer so can figure type
w/o having to deserialize protobuf value content: e.g.
HBASE::REGION_EVENT::REGION_CLOSE

Added doc on WALEdit and tried to formalize the 'meta' WALEdit
type and how it works. Needs complete redo in part as suggested
by HBASE-8457. Meantime, some doc and cleanup.

Also changed the LogRoller constructor to remove redundant param.
Because of constructor change, need to change also
TestFailedAppendAndSync, TestWALLockup, TestAsyncFSWAL &
WALPerformanceEvaluation.java

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-10-30 22:10:51 -07:00
Duo Zhang 3dba799837 HBASE-23181 Blocked WAL archive: "LogRoller: Failed to schedule flush of XXXX, because it is not online on us" (#753)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-10-26 20:47:29 +08:00
Viraj Jasani 33e8156ebc HBASE-22460 : Reopen regions with very high Store Ref Counts (#750)
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2019-10-24 21:20:24 +05:30
chenxu14 b0b7e5f5b8 HBASE-23107 Avoid temp byte array creation when doing cacheDataOnWrite (#678)
Signed-off-by: huzheng <openinx@gmail.com>
Signed-off-by: stack <stack@apache.org>
2019-10-17 09:47:14 +08:00
Michael Stack d8ad2a0bb5 HBASE-23129 Move core to use hbase-thirdparty-3.1.1 (#695)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-10-11 14:32:19 +02:00
haxiaolin fff0f33c5a HBASE-22767 System table RIT STUCK if their RSGroup has no highest version RSes
Signed-off-by: stack <stack@apache.org>
2019-10-08 16:32:07 -07:00
Istvan Toth e3078aa5aa HBASE-23119 ArrayIndexOutOfBoundsException in PrivateCellUtil#qualifierStartsWith (#688)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-10-04 15:14:30 +02:00
stack 61bc57f525 Revert " HBASE-23055 Alter hbase:meta (#655)"
"Too radical for branch-2"

This reverts commit ff217d4269.
2019-10-01 08:34:09 -07:00
Michael Stack ff217d4269
HBASE-23055 Alter hbase:meta (#655)
Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.


   Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2019-09-30 09:27:41 -07:00
chenxu14 be932487e8 HBASE-22965 RS Crash due to DBE reference to an reused ByteBuff (#603)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 09:20:35 +08:00
Sanjeet Nishad 6def152819 HBASE-23038 Provide consistent and clear logging about disabling chores
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit b45c0d0efa)
2019-09-27 20:13:18 -05:00
Viraj Jasani faed3e457d HBASE-23024 Replace initcause with Constructor arg (#627)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-17 13:40:07 +02:00
Viraj Jasani 63568854b6 HBASE-22760 : Pause/Resume/Query Snapshot Auto Cleanup Activity (#618) 2019-09-13 10:37:22 +05:30
chenxu14 4282d70212 HBASE-22802 Avoid temp ByteBuffer allocation in FileIOEngine#read (#583) 2019-09-09 17:44:49 +08:00
Josh Elser 2feca0abde HBASE-22701 Disable the DynamicClassLoader when it fails to initialize
Signed-off-by: Ankit Singhal <ankitsinghal59@gmail.com>
2019-09-04 18:22:21 -04:00
康智冬 576e688cec HBASE-22962 Fix typo in javadoc description (#569)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-01 20:16:05 +08:00
linkaline 84b4a61e88 HBASE-22881 Fix non-daemon threads in hbase server implementation (#512) (#536)
Signed-off-by: stack <stack@apache.org>
2019-08-28 09:48:27 -07:00
ravowlga123 c60748cba1 HBASE-22893 Change the comment in HBaseClassTestRule to reflect change in default test timeouts (#545)
Signed-off-by: Sakthi <sakthi@apache.org>
2019-08-27 01:36:26 -07:00
openinx 53522f36cc HBASE-22810 Initialize an separate ThreadPoolExecutor for taking/restoring snapshot (addendum - use the old config key) (#517) 2019-08-22 11:53:46 +08:00
Huon Wilson 8cbe20c382 HBASE-22841 Add more factory functions to TimeRange
These functions make it easier to possible to use
`org.apache.hadoop.hbase.client.Table.CheckAndMutateBuilder#timeRange`
with more interesting ranges, without being forced to use the
deprecated constructors.

Signed-off-by: huzheng <openinx@gmail.com>
2019-08-17 10:12:48 +08:00
openinx 6f9db4660a HBASE-22810 Initialize an separate ThreadPoolExecutor for taking/restoring snapshot (#486) 2019-08-15 11:22:08 +08:00
stack 225c6cd5bc HBASE-22777 Add a multi-region merge (for fixing overlaps)
Makes MergeTableRegionsProcedure do more than just two regions at a
time. Compatible as MTRP was done considering one day it'd do more than
two at a time.

Changes hardcoded assumption that merge parent regions are named
mergeA and mergeB in a column on the resultant region. Instead
can have N columns on the merged region, one for each parent
merged. Column qualifiers all being with 'merge'.

Most of code below is undoing the assumption that there are two
parents on a merge only.
2019-08-07 16:58:06 -07:00
Jan Hentschel 3971773189
HBASE-22790 Added deprecation versions for fields in ByteBuffAllocator
Signed-off-by: stack <stack@apache.org>
2019-08-05 09:01:03 +02:00
Braavos 89e2cc6281
HBASE-22677 Add unit tests for org.apache.hadoop.hbase.util.ByteRangeUtils and Classes
Co-authored-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-05 03:32:32 +02:00
stack 4565e9f43b HBASE-22723 Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta
Refactor of CatalogJanitor so it generates a
Report on the state of hbase:meta when it runs. Also
refactor so CJ runs even if RIT (previous it would
punt on running if RIT) so it can generate a 'Report'
on the interval regardless. If RIT, it just doesn't
go on to do the merge/split GC as it used to.

If report finds an issue, dump as a WARN message
to the master log.

Follow-on is to make the Report actionable/available
for the Master to pull when it goes to draw the hbck
UI page (could also consider shipping the Report as
part of ClusterMetrics?)

Adds new, fatter Visitor to CJ, one that generates
Report on each run keeping around more findings as
it runs.

Moved some methods around so class reads better;
previous methods were randomly ordered in the class.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Make a few handy methods public.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add utility as defaults on the Inteface; i.e. is this the first region
 in table, is it last, does a passed region come next, or does passed
 region overlap this region (added tests for this new stuff).

M hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
 Bugfix... handle case where buffer passed is null.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Lots of change, reorg., but mostly adding consistency checking
 to the visitor used scanning hbase:meta on a period and the
 generation of a Report on what the scan has found traversing
 hbase:meta. Added a main so could try the CatalogJanitor against
 a running cluster.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitorCluster.java
 Fat ugly test for CatalogJanitor consistency checking.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
 Add tests for new functionality in RI.

M hbase-shell/src/main/ruby/hbase/table.rb
 Bug fix for case where meta has a null regioninfo; scan was aborting.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-07-29 16:07:10 -07:00
Wellington Chevreuil f160dd3c10 HBASE-21773 rowcounter utility should respond to pleas for help
Signed-off-by: stack <stack@apache.org>
2019-07-26 21:29:48 +01:00
Viraj Jasani 3a72ccefbd
HBASE-22648 Snapshot TTL (#371)
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotDescriptionUtils.java
2019-07-22 15:08:20 -07:00
openinx 6fb03e63b6 HBASE-22663 The HeapAllocationRatio in WebUI is not accurate because all of the heap allocation will happen in another separated allocator named HEAP (#365) 2019-07-20 23:36:21 +08:00
毛蛤丝 dfebcbec38 HBASE-22700:incorrect timeout in recommended ZooKeeper configuration
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-07-17 22:12:07 +08:00
syedmurtazahassan 955e643e1d
HBASE-22586 Javadoc Warnings related to @param tag
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-07-13 16:56:53 +02:00
Peter Somogyi ed44b60cb1 HBASE-22651 ErrorProne issue in TestByteBufferArray
Signed-off-by: Michael Stack <stack@apache.org>
2019-07-02 16:37:46 +02:00
syedmurtazahassan 8fa7800389 HBASE-22572 Javadoc Warnings: @link reference not found (#306)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: stack <stack@apache.org>
2019-07-01 21:18:11 -07:00
openinx 345f80ea5f HBASE-20060 Add details of off heap memstore into book. (#334) 2019-06-27 16:54:48 +08:00
huzheng 6a3aa789d8 HBASE-22612 Address the final overview reviewing comments of HBASE-21879 2019-06-26 15:40:23 +08:00
openinx af0e23c359 HBASE-22547 Align the config keys and add document for offheap read in HBase Book. (#301) 2019-06-25 22:25:30 +08:00
openinx d34c4a0481 HBASE-22598 Deprecated the hbase.ipc.server.reservoir.initial.buffer.size & hbase.ipc.server.reservoir.initial.max for HBase2.x compatibility (#318) 2019-06-25 22:22:29 +08:00
openinx 3199175932 HBASE-22491 Separate the heap HFileBlock and offheap HFileBlock because the heap block won't need refCnt and save into prevBlocks list before shipping (#268) 2019-06-25 22:22:16 +08:00
huzheng 5ff317fe7b HBASE-22504 Optimize the MultiByteBuff#get(ByteBuffer, offset, len) 2019-06-25 22:22:04 +08:00
openinx 4dd728dfff HBASE-22483 It's better to use 65KB as the default buffer size in ByteBuffAllocator (#279) 2019-06-25 22:20:58 +08:00
huzheng 9e5fc2b379 HBASE-22435 Add a UT to address the HFileBlock#heapSize() in TestHeapSize 2019-06-25 22:14:57 +08:00
huzheng b00360731a HBASE-22412 Improve the metrics in ByteBuffAllocator 2019-06-25 22:14:50 +08:00
huzheng d7009d258c HBASE-21921 Notify users if the ByteBufAllocator is always allocating ByteBuffers from heap which means the increacing GC pressure 2019-06-25 22:14:36 +08:00
huzheng 39a6a357ff HBASE-21937 Make the Compression#decompress can accept ByteBuff as input 2019-06-25 22:11:51 +08:00
huzheng fc460136e0 HBASE-21957 Unify refCount of BucketEntry and refCount of hbase.nio.ByteBuff into one 2019-06-25 22:07:16 +08:00
huzheng bd8260c07e HBASE-22159 ByteBufferIOEngine should support write off-heap ByteBuff to the bufferArray 2019-06-25 21:28:02 +08:00
huzheng 081dc03e12 HBASE-22127 Ensure that the block cached in the LRUBlockCache offheap is allocated from heap 2019-06-25 21:23:54 +08:00
huzheng 0d294d23a9 HBASE-22005 Use ByteBuff's refcnt to track the life cycle of data block 2019-06-25 21:16:48 +08:00
huzheng c19592a971 HBASE-21916 Abstract an ByteBuffAllocator to allocate/free ByteBuffer in ByteBufferPool 2019-06-25 21:12:37 +08:00
Duo Zhang 974e1086d7 HBASE-22617 Recovered WAL directories not getting cleaned up (#330)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-06-25 17:32:39 +08:00
Sean Busbey 23a2f8abab HBASE-22615 Make TestChoreService more robust to timing
* phrase fudge factor "deltas" in terms of the original period
* increase the delta allowed for chore timing from 5% to 20%
* improve some assertions

Closes #328

Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Sakthi <sakthivel.azhaku@gmail.com>
(cherry picked from commit 9aee88e03a)
2019-06-22 00:29:04 -05:00
Mingliang Liu bc737d1ed9
HBASE-21284 Default limits for PressureAwareCompactionThroughputController are too low
This is a forward port HBASE-21000 from branch-1.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-06-20 12:44:14 -07:00
Jan Hentschel 6cff25cc23
HBASE-22344 Documented the deprecation of public and limited private APIs (#208)
Signed-off-by: stack <stack@apache.org>
2019-06-17 22:29:51 +02:00
Murtaza Hassan a2b22a6807
HBASE-22565 Javadoc Warnings: @see cannot be used in inline documentation 2019-06-13 23:14:58 +02:00
Sakthi 38139350a3 HBASE-22458: TestClassFinder fails when run on JDK11
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 0445186f1a)
2019-06-13 09:02:20 -05:00
Sakthi 35d3612f66 HBASE-22534 TestCellUtil fails when run on JDK11
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 9b23e1d00c)
2019-06-04 14:51:20 -05:00
Duo Zhang 1a2632b796 HBASE-22316 Record the stack trace for current thread in FutureUtils.get 2019-05-30 11:27:43 +08:00
Wellington Chevreuil ea41133e44 HBASE-22496 UnsafeAccess.unsafeCopy should not copy more than UNSAFE_COPY_THRESHOLD on each iteration
Change-Id: I259bc54a0a5b3474d3c455639c3e9cb1e95c8438

Signed-off-by: huzheng <openinx@gmail.com>
2019-05-30 10:21:43 +08:00
Artem Ervits e9e680f3d1 HBASE-22469 replace md5 checksum in saveVersion script with sha512
Signed-off-by: Josh Elser <elserj@apache.org>
2019-05-28 15:06:37 -04:00
Sean Busbey 16f18d5c2d HBASE-22148 Provide a LimitedPrivate(COPROC) API for setting Cell timestamp without copying.
Signed-off-by: Josh Elser <elserj@apache.org>
2019-05-28 12:18:48 -05:00
Andrew Purtell 2c55bd9344
HBASE-22449 https everywhere in Maven metadata (#247) 2019-05-21 12:38:42 -07:00
OrDTesters b585dc42ba HBASE-21467 Fix flaky test TestCoprocessorClassLoader.testCleanupOldJars 2019-05-06 09:45:51 -07:00
Peter Somogyi 32462d5690 HBASE-22340 Corrupt KeyValue is silently ignored (#207) 2019-05-02 11:22:05 +02:00
Sakthi 018dadc3c1 HBASE-22054: Space Quota: Compaction is not working for super user in case of NO_WRITES_COMPACTIONS
Signed-off-by: Josh Elser <elserj@apache.org>
2019-04-30 13:41:08 -04:00
Sean Busbey 4862a596ef HBASE-22083 move eclipse settings into a profile.
Signed-off-by: stack <stack@apache.org>

 Conflicts:
	hbase-backup/pom.xml
	hbase-hadoop-compat/pom.xml
	hbase-protocol/pom.xml
2019-04-25 14:38:38 -05:00
Duo Zhang b117d5b4c2 HBASE-22307 Deprecated Preemptive Fail Fast 2019-04-25 14:20:49 +08:00
李小保 1bd5b5cf7b HBASE-22250 The same constants used in many places should be placed in constant classes
Signed-off-by: stack <stack@apache.org>
2019-04-23 21:24:46 -07:00
zouxiangwen 4138fce370 HBASE-22292 PreemptiveFastFailInterceptor clean repeatedFailuresMap issue
Signed-off-by: stack <stack@apache.org>
2019-04-23 08:02:48 -07:00
Jan Hentschel c40e6e2339 HBASE-22231 Removed unused and '*' import 2019-04-23 12:53:52 +02:00
Ben Manes cadf321d53
HBASE-15560 W-TinyLFU based BlockCache
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-04-17 11:10:55 -07:00
Andrew Purtell f3532e4c26
HBASE-22235 OperationStatus.{SUCCESS|FAILURE|NOT_RUN} are not visible to 3rd party coprocessors 2019-04-16 13:19:56 -07:00
Vladimir Rodionov ae4bfabeaa HBASE-21688 Address WAL filesystem issues
Amending-Author: Josh Elser <elserj@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2019-04-03 13:55:48 -04:00
subrat.mishra a01c8e1dd6 HBASE-22053 Changed zookeeper URL from http://hadoop.apache.org to https://zookeeper.apache.org Replaced version 3.1.1 with 3.3.3 as 3.1.1 is deprecated.
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2019-03-28 00:31:24 +09:00
Geoffrey Jacoby 2c58dc503b
HBASE-22032 KeyValue validation should check for null byte array
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-common/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
2019-03-15 12:45:46 -07:00
Sean Busbey d2296a4bad HBASE-22044 Addendum for checkstyle. 2019-03-13 13:43:39 -05:00
Sean Busbey 09f9fc7c3e HBASE-22044 Deprecate ByteBufferUtils; it will be IA.Private in HBase 3.0
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-03-13 13:39:44 -05:00
zhangduo 31b3716e84 HBASE-22001 Polish the Admin interface
Signed-off-by: stack <stack@apache.org>
2019-03-11 14:43:12 +08:00
Sean Busbey 32631f18f5 HBASE-21999 ADDENDUM unknown revisions are okay; make sure we indicate them.
(cherry picked from commit a7bbff170a)
2019-03-07 13:38:55 -06:00
ramkrishna 82618c439d HBASE-21874 Bucket cache on Persistent memory
Signed-off-by: Anoop John <anoop.hbase@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit f67f207965)
2019-03-07 08:53:31 -06:00
Sean Busbey a5c26653f9 Revert "HBASE-21874 Bucket cache on Persistent memory"
This reverts commit f67f207965.

bad signed off by line
2019-03-07 08:52:13 -06:00
ramkrishna f67f207965 HBASE-21874 Bucket cache on Persistent memory
Signed-off-by: Anoop John <anoop.hbase@gmail.com>, Sean Busbey
<busbey@apache.org>
2019-03-07 10:52:18 +05:30
stack 41be1fbd8b HBASE-21999 [DEBUG] Exit if git returns empty revision! 2019-03-06 16:45:55 -08:00
Reid Chan 6de61d6bee HBASE-21481 [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-03-01 23:42:11 +08:00
Duo Zhang 761aef6d9d HBASE-20587 Replace Jackson with shaded thirdparty gson
Signed-off-by: Michael Stack <stack@apache.org>
2019-02-22 16:40:45 +08:00
zhangduo f198311556 HBASE-21928 Deprecated HConstants.META_QOS
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-02-20 14:16:54 +08:00
Duo Zhang dec5bf3087 HBASE-21900 Infinite loop in AsyncMetaRegionLocator if we can not get the location for meta
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-14 19:50:23 +08:00
Peter Somogyi a33cda4b03 HBASE-21727 Simplify documentation around client timeout - ADDENDUM
Add back HBaseConfiguration#getInt with deprecation

Signed-off-by: zhangduo <zhangduo@apache.org>
2019-02-05 18:01:00 +08:00
zhangduo 54a8cc87c6 HBASE-21829 Use FutureUtils.addListener instead of calling whenComplete directly 2019-02-03 15:29:20 +08:00
Guanghao Zhang 16665b6e93 HBASE-21799 Update branch-2 version to 2.3.0-SNAPSHOT 2019-01-29 21:53:21 +08:00
Wellington Chevreuil aae52f787c HBASE-20215 - Rename CollectionUtils to ConcurrentMapUtils 2019-01-24 20:55:55 -08:00
zhangduo 67ba050fce HBASE-21761 Align the methods in RegionLocator and AsyncTableRegionLocator
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-01-24 15:21:26 +08:00
Allan Yang d80556ec85 HBASE-21754 ReportRegionStateTransitionRequest should be executed in priority executor 2019-01-23 21:08:13 +08:00
huzheng 4eba6b3656 HBASE-21750 Most of KeyValueUtil#length can be replaced by cell#getSerializedSize for better performance because the latter one has been optimized 2019-01-22 20:21:49 +08:00
Duo Zhang ff272e8683 HBASE-21726 Add getAllRegionLocations method to AsyncTableRegionLocator 2019-01-19 11:14:40 +08:00
Peter Somogyi c353d93f69 HBASE-21727 Simplify documentation around client timeout
Signed-off-by: Michael Stack <stack@apache.org>
2019-01-16 11:41:40 +01:00
huzheng fdfbd9a21c HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case 2019-01-16 09:47:27 +08:00
Jingyun Tian f02ac310d2 HBASE-21588 Procedure v2 wal splitting implementation 2019-01-08 17:26:58 +08:00
zhangduo 77ca660389 HBASE-17356 Add replica get support 2019-01-03 08:38:53 +08:00
Ted Yu fe27709e0c HBASE-20136 TestKeyValue misses ClassRule and Category annotations 2018-12-23 10:17:55 -08:00
huzheng 46ba4591a4 HBASE-21401 Sanity check when constructing the KeyValue 2018-12-21 18:50:27 +08:00
Reid Chan 6ab0fbb1f1 HBASE-21255 [acl] Refactor TablePermission into three classes (Global, Namespace, Table)
Signed-off-by: Michael Stack <stack@apache.org>
2018-11-15 11:45:24 +08:00
huzheng a765859131 HBASE-21473 RowIndexSeekerV1 may return cell with extra two \x00\x00 bytes which has no tags 2018-11-14 23:13:04 +08:00
huzheng 98ad5ca6e5 Revert "HBASE-21401 Sanity check in BaseDecoder#parseCell"
This reverts commit b6d32e8a10.
2018-11-12 11:42:04 +08:00
huzheng b6d32e8a10 HBASE-21401 Sanity check in BaseDecoder#parseCell 2018-11-08 20:14:03 +08:00
zhangduo 4f22397ad4 HBASE-21351 The force update thread may have race with PE worker when the procedure is rolling back 2018-11-03 08:24:25 +08:00
Allan Yang ca1b5503e1
HBASE-21292 IdLock.getLockEntry() may hang if interrupted 2018-10-18 14:41:42 -07:00
Sahil Aggarwal ad6ca7c95a
HBASE-20716: Changes the bytes[] conversion done in Bytes and ByteBufferUtils. Instead of doing check unsafe_aligned available everytime, choose the best converter at startup. 2018-10-17 21:04:09 -07:00
Michael Stack c67f7f14e2
HBASE-21320 [canary] Cleanup of usage and add commentary
Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
2018-10-16 22:19:34 -07:00
Zephyr Guo 2b1716fd8e HBASE-21256 Improve IntegrationTestBigLinkedList for testing huge data
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-10-12 11:00:03 +08:00
Mike Drob 302b77697a
HBASE-20764 build broken when latest commit is gpg signed
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-10-08 19:50:39 -05:00
zhangduo 51854a41fc HBASE-21227 Implement exponential retrying backoff for Assign/UnassignRegionHandler introduced in HBASE-21217 2018-09-26 20:54:06 +08:00
Zach York 21fafbaf53 HBASE-20734 Colocate recovered edits directory with hbase.wal.dir
Amending-Author: Reid Chan <reidchan@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-09-26 11:49:21 +08:00
Mingliang Liu 0f514ab75a
HBASE-21164 reportForDuty should do backoff rather than retry
Remove unused methods from Sleeper (its ok, its @Private).
Remove notion of startTime from Sleeper handling (it is is unused).
Allow passing in how long to sleep so can maintain externally.
In HRS, use a RetryCounter to calculate backoff sleep time for when
reportForDuty is failing against a struggling Master.
2018-09-25 11:32:15 -07:00
Chia-Ping Tsai df5310fc1e HBASE-21208 Bytes#toShort doesn't work without unsafe
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-09-25 10:27:08 +08:00
Michael Stack 6616fd2e0f HBASE-21191 Add a holding-pattern if no assign for meta or namespace (Can happen if masterprocwals have been cleared).
Add a check for hbase:meta being online before we go to read it.
If not online, move into a holding-pattern until rectified, probably
by external operator.

Incorporates bulk of patch made by Allan Yang over on HBASE-21035.

M hbase-common/src/main/java/org/apache/hadoop/hbase/util/RetryCounterFactory.java

 Add a Constructor for case where retries are for ever.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Move stuff around so that the first hbase:meta read is the AM#loadMeta.
 Previously, checking table state and/or favored nodes could end up
 trying to read a meta that was not onlined holding up master startup.
 Do similar for the namespace table. Adds new methods isMeta and
 isNamespace which check that the regions/tables are online.. if not,
 we wait logging with a back-off that assigns need to be run.

Signed-off-by: Allan Yang <allan163@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-09-16 21:43:48 -07:00
Tyler Mi d40348e8cd HBASE-21098 Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-09-11 16:31:56 -07:00
Allan Yang 737ac48473
HBASE-21083 Introduce a mechanism to bypass the execution of a stuck procedure 2018-08-28 20:19:55 -07:00
Umesh Agashe cdf5cfa396 HBASE-20941 Created and implemented HbckService in master
Added API setTableStateInMeta() to update table state only in Meta. This will be used by hbck2 tool.
2018-08-27 12:10:33 -07:00
brandboat fb9aa8a95a HBASE-18201 add UT and docs for DataBlockEncodingTool
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-08-10 11:16:36 +08:00
Reid Chan e14b60a539 HBASE-20886 [Auth] Support keytab login in hbase client
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-07-31 11:10:34 +08:00
Alex Leblang ab18440107
HBASE-19369 Switch to Builder Pattern In WAL
This patch switches to the builder pattern by adding a helper method.
It also checks to ensure that the pattern is available (i.e. that
HBase is running on a hadoop version that supports it).

Amending-Author: Mike Drob <mdrob@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-07-27 23:42:53 -05:00
zhangduo 51b0d0edd0 HBASE-20939 There will be race when we call suspendIfNotReady and then throw ProcedureSuspendedException 2018-07-27 21:25:22 +08:00
Xu Cang 1e167e4e2e HBASE-20928 Rewrite calculation of midpoint - addemdum
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-25 10:01:56 -07:00
Saurabh Singh 097ae55708 HBASE-20928 Rewrite calculation of midpoint in binarySearch functions to prevent overflow
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-24 08:17:16 -07:00