Commit Graph

1451 Commits

Author SHA1 Message Date
Viraj Jasani 859bf3ea9a
HBASE-24968 : Move META_COMPARATOR to subclass MetaCellComparator (#2332)
* Break subclass referencing of MetaCellComparator from superclass CellComparatorImpl
  static initializer by moving META_COMPARATOR to subclass MetaCellComparator

Closes #2329

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-08-31 19:51:00 +05:30
Nick Dimiduk 9c85f3641f HBASE-24583 Normalizer can't actually merge empty regions...
when neighbor is larger than average size

* add `testMergeEmptyRegions` to explicitly cover different
  interleaving of 0-sized regions.
* fix bug where merging a 0-size region is skipped due to large
  neighbor.
* remove unused `splitPoint` from `SplitNormalizationPlan`.
* generate `toString`, `hashCode`, and `equals` methods from Apache
  Commons Lang3 template on `SplitNormalizationPlan` and
  `MergeNormalizationPlan`.
* simplify test to use equality matching over `*NormalizationPlan`
  instances as plain pojos.
* test make use of this handy `TableNameTestRule`.
* fix line-length issues in `TestSimpleRegionNormalizer`

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
Signed-off-by: Aman Poonia <aman.poonia.29@gmail.com>
2020-08-12 10:11:40 -07:00
Viraj Jasani 8ccf643fdc
HBASE-24750 : All ExecutorService should use guava ThreadFactoryBuilder (#2214)
Closes #2196

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
Signed-off-by: niuyulin <nyl353@163.com>
2020-08-12 15:57:53 +05:30
bsglz ebf493f075
HBASE-24694 Support flush a single column family of table (#2218)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-08-10 10:36:13 +01:00
Duo Zhang ef7b9eb36e HBASE-23157 WAL unflushed seqId tracking may wrong when Durability.ASYNC_WAL is used (#762)
Signed-off-by: stack <stack@apache.org>
2020-08-10 10:54:30 +08:00
niuyulin e099aa5e20 HBASE-24659 Calculate FIXED_OVERHEAD automatically (#2018)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>

SIgned-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-08-06 14:52:20 -07:00
Sean Busbey 0806349ada
HBASE-24805 HBaseTestingUtility.getConnection should be threadsafe
* refactor how we use connection to rely on the access method
* refactor initialization and cleanup of the shared connection
* incompatibly change HCTU's Configuration member variable to be final so it can be safely accessed from multiple threads.

Closes #2180

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 86ebbdd8a2)
2020-08-02 00:51:03 -05:00
stack edb4cd534c HBASE-24632 Enable procedure-based log splitting as default in hbase3 Add deprecation of 'classic' zk-based WAL splitter.
Also fix three bugs:

 * We were trying to delete non-empty directory; weren't doing
 accounting for meta WALs where meta had moved off the server
 (successfully)
 * We were deleting split WALs rather than archiving them.
 * We were not handling corrupt files.

Deprecations and removal of tests of old system.
2020-07-27 08:23:29 -07:00
Viraj Jasani 6cb51cc0f0
HBASE-24757 : ReplicationSink should limit row count in batch mutation based on hbase.rpc.rows.warning.threshold (#2136)
Closes #2127

Signed-off-by: stack <stack@apache.org>
2020-07-24 20:32:40 +05:30
Viraj Jasani ce4e692699
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses) (#2110)
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-22 11:36:19 +05:30
Peter Somogyi e0c917b647 HBASE-24710 Incorrect checksum calculation in saveVersion.sh (#2056)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-07-21 10:41:38 +02:00
stack 620470607e Revert "HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses)"
Causes TestAdminShell and TestThriftHBaseServiceHandler to fail 100% of the time in branch-2.

This reverts commit 8ae3480e70.
2020-07-20 14:50:03 -07:00
Viraj Jasani 8ae3480e70
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses)
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-17 19:57:05 +05:30
bsglz ec3beaf5a2
HBASE-24664 Some changing of split region by overall region size rather than only one store size (#2054)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-17 10:47:01 +01:00
Bo Cui 44d351915a
HBASE-24693 regioninfo#isLast() should point to EMPTY_END_ROW (readability improvement)
Closes #2051

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-14 15:43:17 +05:30
XinSun 5fcffae5db
HBASE-24431 RSGroupInfo add configuration map to store something extra (#2031)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-08 14:39:41 +08:00
pengmq1 2a12fd283e HBASE-24289 Heterogeneous Storage for Date Tiered Compaction (#1730)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-30 15:19:30 +08:00
Duo Zhang ed6104418c HBASE-24646 Set the log level for ScheduledChore to INFO in HBTU (#1987)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-06-28 23:27:42 +08:00
Bharath Vissapragada a6c887012e
HBASE-24603: Make Zookeeper sync() call synchronous (#1945) (#1975)
Writing a test for this is tricky. There is enough coverage for
functional tests. Only concern is performance, but there is enough
logging for it to detect timed out/badly performing sync calls.

Additionally, this patch decouples the ZK event processing into it's
own thread rather than doing it in the EventThread's context. That
avoids deadlocks and stalls of the event thread.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 84e246f9b1)
2020-06-25 16:41:11 -07:00
Nick Dimiduk 3db4f00f24 HBASE-22504 Addendum: restore findCommonPrefix
Address incompatibility issue raised in 2.3.0RC0 vote thread.
2020-06-25 11:17:22 -07:00
Michael Stack c4042f4927
HBASE-23055 Alter hbase:meta (#1956)
Addendum to fix illegal removal of unused constant w/o
a deprecation cycle.
2020-06-22 20:20:42 -07:00
Anoop Sam John ed7dc9ed4c HBASE-24189 WALSplit recreates region dirs for deleted table with recovered edits data. 2020-06-11 21:28:32 +05:30
Andrew Purtell b67f896954
HBASE-24367 ScheduledChore log elapsed timespan in a human-friendly format (#1749)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-09 09:57:03 -07:00
XinSun 67fa367bb6 HBASE-24359 Optionally ignore edits for deleted CFs for replication (#1855)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-08 13:14:50 +08:00
Nick Dimiduk 4884773676 HBASE-24418 Consolidate Normalizer implementations
Simplify our Normalizer story to have just a single, configurable
implementation.

* fold the features of `MergeNormalizer` into
  `SimpleRegionNormalizer`, removing the intermediate abstract class.
* configuration keys for merge-only features now share a common
  structure.
* add configuration to selectively disable normalizer split/merge
  operations.
* `RegionNormalizer` now extends `Configurable` instead of creating a
  new instance of `HBaseConfiguration` or snooping one off of other
  fields.
* avoid the extra RPCs by using `MasterServices` instead of
  `MasterRpcServices`.
* boost test coverage of all the various flags and feature
  combinations.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-06-03 16:43:40 -07:00
niuyulin 571ef7e811 HBASE-24412 Canary support check only one column family per RegionTask (#1817)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-03 09:16:11 +08:00
Bharath Vissapragada d7f388518c
HBASE-24477: Move ConfigurationObserver and related classes to hbase-common
This utility is useful for any module that wants to detect
dynamic config changes. Having it to hbase-common makes it
accessible to all the other modules.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-31 14:14:30 -07:00
bsglz 22efe67758
HBASE-24451 Remove the HasThread because the related bug had been fixed since jdk7 (#1797)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-29 17:46:03 +05:30
Duo Zhang 58eed9a4bb HBASE-24408 Introduce a general 'local region' to store data on master (#1753)
Signed-off-by: stack <stack@apache.org>
2020-05-23 16:40:39 +08:00
Viraj Jasani 2383cc4242
HBASE-24405 : Document hbase:slowlog related operations (#1747)
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-05-22 20:58:04 +05:30
Viraj Jasani 6f5e5e4828
HBASE-23938 : System table hbase:slowlog to store complete slow/large… (#1681)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
2020-05-20 15:30:30 +05:30
Mingliang Liu 5893c4f393
HBASE-23969 Meta browser should show all `info` columns (#1710)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-18 11:48:36 -07:00
Nick Dimiduk 1c32a8ba2e HBASE-24271 Set values in `conf/hbase-site.xml` that enable running on `LocalFileSystem` out of the box
Simplify the new user experience shipping a configuration that enables
a fresh checkout or tarball distribution to run in standalone mode
without direct user configuration. This change restores the behavior
we had when running on Hadoop 2.8 and earlier.

Patch for master includes an update to the book. This change will be
omitted when backporting to earlier branches.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-14 14:05:40 -07:00
Nick Dimiduk 2ab394ddb1 Revert "HBASE-24086 Disable output stream capability enforcement when running on LocalFileSystem"
This reverts commit 4e31ada43b.
2020-05-14 14:05:40 -07:00
shahrs87 941cef71c2
HBASE-24190 : Make kerberos value of hbase.security.authentication property case insensitive (#1687)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-14 11:46:33 +05:30
Viraj Jasani b78f4367f7
Revert "Make kerberos value of hbase.security.authentication property case insensitive (#1687)"
This reverts commit c03ec837e7.
2020-05-14 11:45:23 +05:30
Duo Zhang dc2146069c
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for … (#1697)
Signed-off-by: stack <stack@apache.org>
2020-05-13 17:59:21 +08:00
Sambit Mohapatra d85caaa195
HBASE-23832 Old config hbase.hstore.compactionThreshold is ignored (#1698)
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2020-05-12 23:20:48 +05:30
shahrs87 c03ec837e7
Make kerberos value of hbase.security.authentication property case insensitive (#1687)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-11 11:10:29 +05:30
Duo Zhang 7f4683bafa HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry … (#1593)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-06 16:11:28 +08:00
Michael Stack 528e9d6454
HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws BindException (#1656)
Move the random free port generate back into hbasecommontestingutility
  from hbasetestingutility.

  Add a create simple kdc server utility that will start a kdc server and
  if a bindexception, create a new one on a new random port in hbase-common.

  Add new BoundSocketMaker helpful when trying to manufacture
  BindExceptions because of port clash.

  Change thrift and http kdc tests to use this new utility (removes
  code duplication around kdc server setup).
2020-05-05 21:08:09 -07:00
Andrew Purtell 5dc10dbba5
HBASE-24322 UnsafeAvailChecker should also check that required methods are available (#1650)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-05-05 12:11:37 -07:00
Duo Zhang 1f556b3ad9 HBASE-24281 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:25:41 +08:00
Duo Zhang 379b48bb5d Revert "HBASE-24181 Remove some stale hadoop.version properties in the pom for sub modules (#1602)"
This reverts commit dc9608ec0b.
2020-04-29 18:25:23 +08:00
Duo Zhang dc9608ec0b HBASE-24181 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:08:33 +08:00
Duo Zhang 922921ee5f HBASE-24249 Move code in FSHDFSUtils to FSUtils and mark related clas… (#1586)
Signed-off-by: stack <stack@apache.org>
2020-04-29 11:31:32 +08:00
Viraj Jasani b40381857a
HBASE-24024 : Reject multi() requests with rows higher than threshold (#1560)
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-04-22 12:51:49 +05:30
Mark Robert Miller 8a7aa89506
HBASE-24185: Junit tests do not behave well with System.exit or Runtime.halt or JVM exits in general. (#1540)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-04-20 22:20:08 -07:00
Viraj Jasani ca1f7a9499
HBASE-24194 : Refactor anonymous inner classes of BufferedEncodedSeeker to named inner classes (#1522)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
2020-04-16 19:17:44 +05:30
stack ba87ffed09 Revert "HBASE-24072 Nightlies reporting OutOfMemoryError: unable to create new native thread."
Revert because applied with wrong JIRA number.

This reverts commit 9c60c5fddb.
2020-04-07 10:17:48 -07:00