1493 Commits

Author SHA1 Message Date
Viraj Jasani
35d59dc570
HBASE-24528 : BalancerDecision queue implementation in HMaster with Admin API
* Admin API getLogEntries() for ring buffer use-cases: so far, provides balancerDecision and slowLogResponse
* Refactor RPC call for similar use-cases
* Single RPC API getLogEntries() for both Master.proto and Admin.proto

Closes #2261

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-09-16 20:56:54 +05:30
XinSun
ca96f96461
HBASE-25014 ScheduledChore is never triggered when initalDelay > 1.5*period (#2395)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-16 09:00:21 +08:00
XinSun
c2721c8235
HBASE-24759 Refuse to update configuration of default group (#2126)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-09-04 08:19:39 +08:00
Viraj Jasani
1c568ec95d
HBASE-24968 : Move META_COMPARATOR to subclass MetaCellComparator
* Break subclass referencing of MetaCellComparator from superclass CellComparatorImpl
  static initializer by moving META_COMPARATOR to subclass MetaCellComparator

Closes #2329

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-08-30 22:03:57 +05:30
Nick Dimiduk
2e8120a63f HBASE-24583 Normalizer can't actually merge empty regions...
when neighbor is larger than average size

* add `testMergeEmptyRegions` to explicitly cover different
  interleaving of 0-sized regions.
* fix bug where merging a 0-size region is skipped due to large
  neighbor.
* remove unused `splitPoint` from `SplitNormalizationPlan`.
* generate `toString`, `hashCode`, and `equals` methods from Apache
  Commons Lang3 template on `SplitNormalizationPlan` and
  `MergeNormalizationPlan`.
* simplify test to use equality matching over `*NormalizationPlan`
  instances as plain pojos.
* test make use of this handy `TableNameTestRule`.
* fix line-length issues in `TestSimpleRegionNormalizer`

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
Signed-off-by: Aman Poonia <aman.poonia.29@gmail.com>
2020-08-12 10:04:45 -07:00
Viraj Jasani
ea130249ae
HBASE-24750 : Adding default UncaughtExceptionHandler for Thread factories (ADDENDUM)
Closes #2231
2020-08-11 17:18:47 +05:30
Duo Zhang
726756a8b9
HBASE-23157 WAL unflushed seqId tracking may wrong when Durability.ASYNC_WAL is used (#762)
Signed-off-by: stack <stack@apache.org>
2020-08-10 09:11:31 +08:00
Viraj Jasani
0b604d921a
HBASE-24750 : All ExecutorService should use guava ThreadFactoryBuilder
Closes #2196

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
2020-08-07 20:24:36 +05:30
bsglz
485e0d2fa4
HBASE-24694 Support flush a single column family of table (#2179)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-08-07 11:28:53 +01:00
niuyulin
f710d2d654
HBASE-24659 Calculate FIXED_OVERHEAD automatically (#2018)
Co-authored-by: niuyulin <niuyulin@xiaomi.com>

SIgned-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-08-06 14:50:32 -07:00
Sean Busbey
86ebbdd8a2
HBASE-24805 HBaseTestingUtility.getConnection should be threadsafe
* refactor how we use connection and async connection to rely on their access methods
* refactor initialization and cleanup of the shared connection
* incompatibly change HCTU's Configuration member variable to be final so it can be safely accessed from multiple threads.

Closes #2180

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-01 23:19:07 -05:00
Michael Stack
345b77a4ca
HBASE-24632 Enable procedure-based log splitting as default in hbase3 Add deprecation of 'classic' zk-based WAL splitter. (#2156)
Also fix three bugs:

 * We were trying to delete non-empty directory; weren't doing
 accounting for meta WALs where meta had moved off the server
 (successfully)
 * We were deleting split WALs rather than archiving them.
 * We were not handling corrupt files.

Deprecations and removal of tests of old system.

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-07-29 07:37:28 -07:00
Viraj Jasani
09e7ccd42d
HBASE-24757 : ReplicationSink should limit row count in batch mutation based on hbase.rpc.rows.warning.threshold
Closes #2127

Signed-off-by: stack <stack@apache.org>
2020-07-24 13:16:02 +05:30
Duo Zhang
0b85729da4
HBASE-24762 Purge protobuf java 2.5.0 dependency (#2128)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-07-24 11:48:35 +08:00
Viraj Jasani
f35c5eaadd
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses) (#2109)
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-21 20:24:13 +05:30
bsglz
fccd03ffa2
HBASE-24555 Correct the description of hbase.hregion.max.filesize (#1895)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-21 11:04:42 +01:00
Peter Somogyi
ea96743dd2
HBASE-24710 Incorrect checksum calculation in saveVersion.sh (#2056)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-07-21 10:40:22 +02:00
Viraj Jasani
f2c087caeb
Revert "HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses)"
This reverts commit 16f306b4a17e0f3ab8e3381bde9b2c4c7f742ef9.

TestAdminShell and TestThriftHBaseServiceHandler are failing consistently
2020-07-21 13:50:27 +05:30
Viraj Jasani
16f306b4a1
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses)
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-17 19:16:22 +05:30
Bo Cui
d132c4b53d
HBASE-24693 regioninfo#isLast() should point to EMPTY_END_ROW (readability improvement)
Closes #2051

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-14 15:41:44 +05:30
bsglz
3bd54217a0
HBASE-24664 Some changing of split region by overall region size rath… (#2011)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-07-13 10:00:48 +01:00
pengmq1
be57e40f36
HBASE-24289 Heterogeneous Storage for Date Tiered Compaction (#1730)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-30 15:10:04 +08:00
Duo Zhang
46bf8944fd
HBASE-24646 Set the log level for ScheduledChore to INFO in HBTU (#1987)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-06-28 23:26:05 +08:00
Michael Stack
0197438564
HBASE-24638 Edit doc on (offheap) memory management (#1978) 2020-06-26 09:11:04 -07:00
Bharath Vissapragada
84e246f9b1
HBASE-24603: Make Zookeeper sync() call synchronous (#1945)
Writing a test for this is tricky. There is enough coverage for
functional tests. Only concern is performance, but there is enough
logging for it to detect timed out/badly performing sync calls.

Additionally, this patch decouples the ZK event processing into it's
own thread rather than doing it in the EventThread's context. That
avoids deadlocks and stalls of the event thread.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-25 13:27:38 -07:00
XinSun
4b89ee24f3
HBASE-24431 RSGroupInfo add configuration map to store something extra (#1782)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-24 18:46:09 +08:00
Michael Stack
500a1df379 HBASE-23055 Alter hbase:meta (#1956)
Addendum to fix illegal removal of unused constant w/o
a deprecation cycle.
2020-06-22 20:25:15 -07:00
Anoop Sam John
4005e42a62
HBASE-24189 WALSplit recreates region dirs for deleted table with recovered edits data. 2020-06-11 21:27:21 +05:30
Andrew Purtell
474d200daa
HBASE-24367 ScheduledChore log elapsed timespan in a human-friendly format (#1749)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-09 09:55:20 -07:00
XinSun
59031ab661
HBASE-24359 Optionally ignore edits for deleted CFs for replication (#1722)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-04 17:52:52 +08:00
niuyulin
f71f1cdfa0
HBASE-24412 Canary support check only one column family per RegionTask (#1817)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-03 09:15:10 +08:00
Nick Dimiduk
cb5c0d2782 HBASE-24418 Consolidate Normalizer implementations
Simplify our Normalizer story to have just a single, configurable
implementation.

* fold the features of `MergeNormalizer` into
  `SimpleRegionNormalizer`, removing the intermediate abstract class.
* configuration keys for merge-only features now share a common
  structure.
* add configuration to selectively disable normalizer split/merge
  operations.
* `RegionNormalizer` now extends `Configurable` instead of creating a
  new instance of `HBaseConfiguration` or snooping one off of other
  fields.
* avoid the extra RPCs by using `MasterServices` instead of
  `MasterRpcServices`.
* boost test coverage of all the various flags and feature
  combinations.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
2020-06-02 11:38:39 -07:00
Bharath Vissapragada
4f49a96258
HBASE-24477: Move ConfigurationObserver and related classes to hbase-common (#1815)
This utility is useful for any module that wants to detect
dynamic config changes. Having it to hbase-common makes it
accessible to all the other modules.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-31 11:18:16 -07:00
bsglz
7b7e25a50b
HBASE-24451 Remove the HasThread because the related bug had been fixed since jdk7 (#1797)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-29 17:33:18 +05:30
Bharath Vissapragada
fe1fc25fba
HBASE-24449: Generate version.h include file during compilation. (#1794)
This is needed for the hbase-native-client to compile. Also ships
the includes in the assembly tar.

Signed-off-by: Marc <phrocker@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2020-05-28 15:26:53 -07:00
Duo Zhang
c303f9d329
HBASE-24408 Introduce a general 'local region' to store data on master (#1753)
Signed-off-by: stack <stack@apache.org>
2020-05-23 15:59:51 +08:00
Viraj Jasani
e1d7dc82ea
HBASE-24405 : Document hbase:slowlog related operations (#1747)
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-05-22 18:19:57 +05:30
Viraj Jasani
86a2692dc4
HBASE-23938 : System table hbase:slowlog to store complete slow/large… (#1681)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
2020-05-20 15:10:29 +05:30
Nick Dimiduk
7e295e767f HBASE-24271 Set values in conf/hbase-site.xml that enable running on LocalFileSystem out of the box
Simplify the new user experience shipping a configuration that enables
a fresh checkout or tarball distribution to run in standalone mode
without direct user configuration. This change restores the behavior
we had when running on Hadoop 2.8 and earlier.

Patch for master includes an update to the book. This change will be
omitted when backporting to earlier branches.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-05-14 13:18:43 -07:00
Nick Dimiduk
8148c5858b Revert "HBASE-24086 Disable output stream capability enforcement when running on LocalFileSystem"
This reverts commit 10cbb3fb7310e88de3814396bd7766303c2e65aa.
2020-05-14 13:18:43 -07:00
Mingliang Liu
a40a0322a7
HBASE-23969 Meta browser should show all info columns (#1485)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-13 11:50:53 -07:00
Sambit Mohapatra
c4e892e3f1
HBASE-23832 Old config hbase.hstore.compactionThreshold is ignored
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2020-05-12 23:19:32 +05:30
Duo Zhang
8601416ee8
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for modules other than hbase-assembly (#1640)
Signed-off-by: stack <stack@apache.org>
2020-05-12 12:03:30 +08:00
shahrs87
ff85daf1e2
[HBASE-24190] Make kerberos value of hbase.security.authentication property case insensitive (#1687)
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-05-11 10:54:45 +05:30
Jan Hentschel
03e5a1402d
HBASE-24305 Removed deprecations in ServerName (#1666)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-05-08 09:41:55 +02:00
Duo Zhang
c1cb22f0b3
HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry … (#1593)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-05-06 15:55:26 +08:00
Michael Stack
33eadb5bb1 HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws BindException (#1656)
Move the random free port generate back into hbasecommontestingutility
  from hbasetestingutility.

  Add a create simple kdc server utility that will start a kdc server and
  if a bindexception, create a new one on a new random port in hbase-common.

  Add new BoundSocketMaker helpful when trying to manufacture
  BindExceptions because of port clash.

  Change thrift and http kdc tests to use this new utility (removes
  code duplication around kdc server setup).
2020-05-05 21:36:59 -07:00
Yu Wang
3d96007c37
HBASE-24137:The max merge count of metafixer may be remind in hbase-site.xml (#1478) 2020-05-05 16:14:14 -07:00
Andrew Purtell
404849ab36
HBASE-24322 UnsafeAvailChecker should also check that required methods are available (#1650)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-05-05 12:07:56 -07:00
Duo Zhang
a2e373dc32 HBASE-24281 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:24:52 +08:00