Commit Graph

17494 Commits

Author SHA1 Message Date
Toshihiro Suzuki ecbed33092
HBASE-23146 Support CheckAndMutate with multiple conditions (#1114)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-02-26 08:09:04 +09:00
Jan Hentschel 9f223c2236
HBASE-23781 Removed deprecated createTableDescriptor(String) from HBaseTestingUtility
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-02-25 21:10:09 +01:00
Nick Dimiduk 3e21dc73de HBASE-23874 Move Jira-attached file precommit definition from script in Jenkins config to dev-support (#1191) (ADDENDUM)
Addendum -- set new script executable
2020-02-24 10:07:23 -08:00
Wei-Chiu Chuang 69d683afcb
HBASE-23755. [OpenTracing] Declare HTrace is unusable in the user doc (#1196)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-02-24 23:08:13 +05:30
Nick Dimiduk a5916c8e8e
HBASE-23874 Move Jira-attached file precommit definition from script in Jenkins config to dev-support (#1191)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-02-24 09:18:31 -08:00
Guanghao Zhang 418234356f HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled (addendum) 2020-02-24 22:11:41 +08:00
Michael 400b7ce954 HBASE-23854 replaced deprecated code in Example Scala Code section
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-02-21 15:06:03 -06:00
Sean Busbey 0b2eb3594b Revert "replaced deprecated code in Example Scala Code section"
This reverts commit 4381b18b87.
2020-02-21 15:05:16 -06:00
Michael 4381b18b87 replaced deprecated code in Example Scala Code section
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-02-21 15:03:38 -06:00
Minwoo Kang 3ab0135c0e
HBASE-23561 Look up of Region in Master by encoded region name is O(n) (#1193)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-02-21 16:40:45 +00:00
Bharath Vissapragada fe0d49bc52 HBASE-23331: Documentation for HBASE-18095 2020-02-19 20:10:17 -08:00
Bharath Vissapragada c34dbc3c29 HBASE-23804: Fix default master addr hostname in master registry (#1137)
Master rpc server end point doesn't bind to localhost's
IP address by default. Instead, it looks up the hostname and
binds to the endpoint to which it resolves. MasterRegistry should
do the same when building the default server end point to talk to.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-02-19 20:10:17 -08:00
Bharath Vissapragada 02bd0eca53 HBASE-23752: Fix remaining test failures from nightly runs (#1102)
TestFromClientSideWithCoprocessor: Initialization bug causing parameterized
runs to fail.
TestCustomSaslAuthenticationProvider: Test config had to be fixed because
it was written pre-master registry implementation.
TestSnapshotScannerHDFSAclController: Cluster restart did not reset the
cached connection state.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-02-19 20:10:17 -08:00
Bharath Vissapragada 5c88672d54 HBASE-23731: De-flake TestFromClientSide (#1091)
There were a couple of issues.

- There was a leak of a file descriptor for hbck lock file. This
was contributing to all the "ConnectionRefused" stack traces since
it was trying to renew lease for an already expired mini dfs cluster.
This issue was there for a while, just that we noticed it now.

- After upgrade to JUnit 4.13, it looks like the behavior for test
timeouts has changed. Earlier the timeout seems to have applied for
each parameterized run, but now it looks like it is applied across
all the runs.

This patch fixes both the issues.

Signed-off-by: Stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-02-19 20:10:17 -08:00
Bharath Vissapragada 229b8aaaf3 HBASE-23647: Make MasterRegistry the default impl. (#1039)
Signed-off-by: Stack <stack@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-02-19 20:10:17 -08:00
Bharath Vissapragada d8b3f55518 HBASE-23330: Fix delegation token fetch with MasterRegistry (#1084)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-02-19 20:10:17 -08:00
Bharath Vissapragada 1aa6a4efb9 HBASE-23305: Master based registry implementation (#954)
Implements a master based registry for clients.

 - Supports hedged RPCs (fan out configured via configs).
 - Parameterized existing client tests to run with multiple registry combinations.
 - Added unit-test coverage for the new registry implementation.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-02-19 20:10:17 -08:00
Bharath Vissapragada 7162c02c0d HBASE-23648: Re-use underlying connection registry in RawAsyncHBaseAdmin (#994)
* HBASE-23648: Re-use underlying connection registry in RawAsyncHBaseAdmin

No need to create and close a new registry on demand. Audited other
usages of getRegistry() and the code looks fine.

* Fix checkstyle issues in RawAsyncHBaseAdmin
2020-02-19 20:10:17 -08:00
Bharath Vissapragada 0a1c3b2055 HBASE-23604: Clarify AsyncRegistry usage in the code. (#957)
* HBASE-23604: Cleanup AsyncRegistry interface

- Cleans up the method names to make more sense and adds a little
more javadocs for context. In future patches we can revisit
the name of the actual class to make it more self explanatory.

- Does AsyncRegistry -> ConnectionRegistry rename.
"async" ness of the registry is kind of implicit based on
the interface contents and need not be reflected in the name.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-02-19 20:10:17 -08:00
Bharath Vissapragada 023078ffa4 HBASE-23304: RPCs needed for client meta information lookup (#904)
* HBASE-23304: RPCs needed for client meta information lookup

This patch implements the RPCs needed for the meta information
lookup during connection init. New tests added to cover the RPC
code paths. HBASE-23305 builds on this to implement the client
side logic.

Fixed a bunch of checkstyle nits around the places the patch
touches.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-02-19 20:10:17 -08:00
Bharath Vissapragada 8126566b27 HBASE-23281: Track meta region locations in masters (#830)
* HBASE-23281: Track meta region changes on masters

This patch adds a simple cache that tracks the meta region replica
locations. It keeps an eye on the region movements so that the
cached locations are not stale.

This information is used for servicing client RPCs for connections
that use master based registry (HBASE-18095). The RPC end points
will be added in a separate patch.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-02-19 20:10:17 -08:00
Bharath Vissapragada 3d298b087a HBASE-23275: Track active master's address in ActiveMasterManager (#812)
* HBASE-23275: Track active master's address in ActiveMasterManager

Currently we just track whether an active master exists.
It helps to also track the address of the active master in
all the masters to help serve the client RPC requests to
know which master is active.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-02-19 20:10:17 -08:00
stack 3168cf210c HBASE-23877 [Flakey Test] TestStochasticLoadBalancerRegionReplicaHighReplication "Two or more region replicas are hosted on the same host after balance" 2020-02-19 16:09:11 -08:00
stack 435b79335e HBASE-23872 [Flakey Test] TestGenerateDelegationToken; Master not initialized after 200000ms 2020-02-19 15:48:33 -08:00
Vladimir Rodionov b8194b4902 HBASE-22749 Distributed MOB compactions
- MOB compaction is now handled in-line with per-region compaction on region
  servers
- regions with mob data store per-hfile metadata about which mob hfiles are
  referenced
- admin requested major compaction will also rewrite MOB files; periodic RS
  initiated major compaction will not
- periodically a chore in the master will initiate a major compaction that
  will rewrite MOB values to ensure it happens. controlled by
  'hbase.mob.compaction.chore.period'. default is weekly
- control how many RS the chore requests major compaction on in parallel
  with 'hbase.mob.major.compaction.region.batch.size'. default is as
  parallel as possible.
- periodic chore in master will scan backing hfiles from regions to get the
  set of referenced mob hfiles and archive those that are no longer
  referenced. control period with 'hbase.master.mob.cleaner.period'
- Optionally, RS that are compacting mob files can limit write
  amplification by not rewriting values from mob hfiles over a certain size
  limit. opt-in by setting 'hbase.mob.compaction.type' to 'optimized'.
  control threshold by 'hbase.mob.compactions.max.file.size'.
  default is 1GiB
- Should smoothly integrate with existing MOB users via rolling upgrade.
  will delay old MOB file cleanup until per-region compaction has managed
  to compact each region at least once so that used mob hfile metadata can
  be gathered.
2020-02-19 16:06:38 -06:00
Guanghao Zhang 53afc1dc10
HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled (#1182)
Signed-off-by: binlijin <binlijin@gmail.com>
2020-02-19 18:20:12 +08:00
stack 156ababa4e HBASE-23867 [Flakey Test] TestStochasticLoadBalancerRegionReplicaSameHosts#testRegionReplicationOnMidClusterSameHosts 2020-02-18 22:40:34 -08:00
stack be4b6d6824 HBASE-23866 More test classifications
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncMetaRegionLocator.java
 Add wait on no region in transition in setup before proceeding.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncResultScannerCursor.java
 Make test large. Takes longer than 50seconds.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableRSCrashPublish.java
 Make large because it fails when networking is weird... when VPN or
 something is up. See HBASE-23850. When large, doesn't get run
 when you do mvn test.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncTableScanAll.java
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestScannersFromClientSide2.java
hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestMultiRowRangeFilter.java
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancerRegionReplica.java
 Make it large. Takes too long.

hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestSeparateClientZKCluster.java
 Use data dir instead of /tmp.

M hbase-server/src/test/java/org/apache/hadoop/hbase/http/TestInfoServersACL.java
 Wait till mbean is up before proceeding w/ test.

hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/SecureTestCluster.java
 Test for null CLUSTER in tearDown.
2020-02-18 21:11:21 -08:00
stack 86e76c2bd9 HBASE-23865 Up flakey history from 5 to 10 2020-02-18 16:32:36 -08:00
GeorryHuang c4fb1cd234
HBASE-23859 Modify "Block locality" of RegionServer Web UI to human readable percentage (#1176)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-02-18 10:06:47 +08:00
Duo Zhang 0140d3f3fe
HBASE-23857 Add 2.1.9 to download page (#1177)
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-02-18 09:51:58 +08:00
stack 0c18fa983b HBASE-23838 Adding debug logging to a few ExportSnapshot tests
ADDENDUM 2: More logging to debug.
2020-02-17 17:19:25 -08:00
GeorryHuang e11dbf36a5
HBASE-23855: Change bytes size to human readable size for Server Metrics of RegionServer Web UI (#1175)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-02-17 09:14:53 +08:00
Jan Hentschel 68f4fd1ee2
HBASE-23844 Removed deprecated Scan(byte[], Filter) from Scan
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-02-16 21:51:43 +01:00
Jan Hentschel ae781e569c
HBASE-23837 Removed deprecated isLegalFamilyName(byte[]) from HColumnDescriptor
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-02-15 21:19:51 +01:00
stack 0e96638898 HBASE-23838 Adding debug logging to a few ExportSnapshot tests
ADDENDUM to fix failing TestExportSnapshot
2020-02-14 16:09:51 -08:00
Sakthi 5f61df4792
HBASE-22827 Expose multi-region merge in shell and Admin API (#1138)
mergeRegionsAsync admin API with two regions as parameters deprecated since 2.3.0 and removed from 4.0
merge_region shell command now supports multiple regions merge since 2.3.0 & 3.0.0

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-02-13 14:40:46 -08:00
Nick Dimiduk 533302adde
HBASE-23808 [Flakey Test] TestMasterShutdown#testMasterShutdownBefore… (#1141)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-02-13 14:02:15 -08:00
stack b7ef225609 HBASE-23838 Adding debug logging to a few ExportSnapshot tests
* hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
 Add logging of temp output dir and if tmp dir is being used at all.

* hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
 Don't create new Configurations. Use current. Set it into the launched
 ExportSnapshot too. Log as we make progress through stages so easier to
 find location of exception source.

* hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV1NoCluster.java
 Add makeQualified though looks to be redundant.

* hbase-server/src/test/java/org/apache/hadoop/hbase/http/TestInfoServersACL.java
 Might fix a local fail where hbase doesn't seem to be up when we query
 jmx. Wait on meta being online.

M hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionStateStore.java
 Run all on the one cluster instance. See if helps w/ odd failure when
 run locally.
2020-02-13 13:50:43 -08:00
Michael Stack 488c2b2483 HBASE-23816 [Flakey Test] TestExportSnapshotNoCluster.testSnapshotV2WithRefsExportFileSystemState(TestExportSnapshotNoCluster.java:91) Wrong FS! ADDENDUM: Break test in two to see if it makes it more reliable. (#1162)
ADDENDUM that breaks test in two to see if it makes it more reliable.
2020-02-12 09:38:36 -08:00
Pankaj Kumar 4c55c06ff4 Adding Pankaj to the developer list 2020-02-12 19:55:22 +05:30
maoling 12f98bc50e
HBASE-20623: Introduce the helper method "getCellBuilder()" to Mutation (#1121)
Signed-off-by: stack <stack@apache.org>
2020-02-10 10:03:35 -08:00
stack a6774ca7f7 HBASE-23812 [Flakey Test] TestReplicator#testReplicatorWithErrors: AssertionError: We did not replicate enough rows expected:<10> but was:<7> 2020-02-10 09:45:43 -08:00
meiyi 29a36568b1
HBASE-23824 TestSnapshotScannerHDFSAclController is flakey (#1157)
Signed-off-by: stack <stack@apache.org>
2020-02-10 09:40:05 -08:00
stack cdc98e906c HBASE-23816 [Flakey Test] TestExportSnapshotNoCluster.testSnapshotV2WithRefsExportFileSystemState(TestExportSnapshotNoCluster.java:91) Wrong FS! 2020-02-10 09:06:50 -08:00
stack 93c04e4f60 HBASE-23815 [Flakey Test] AbstractTestAsyncTableRegionReplicasRead family of tests fails with no breadcrumbs on why 2020-02-10 09:04:36 -08:00
Wellington Ramos Chevreuil 069df4d791
HBASE-23803 [DOC] Fix the maths on the section explaining call queue tuning options (#1128)
Signed-off-by: Anoop Sam John <anoopsamjohn@apacher.org>
2020-02-10 16:55:52 +00:00
stack a4aa183f45 HBASE-23814 Add null checks and logging to misc set of tests
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Zach York <zyork@apache.org>

* TestFullLogReconstruction log the server we've chosen to expire and then note where we starting counting rows
* TestAsyncTableScanException use a define for row counts
* TestRawAsyncTableLimitedScanWithFilter check connection was made before closing it in tearDown
* TestLogsCleaner use single mod time. Make it for sure less than now in case test runs all in the same millisecond (would cause test fail)
* TestReplicationBase test table is non-null before closing in tearDown
2020-02-10 08:51:56 -08:00
GeorryHuang 35989c1968
HBASE-23822:Fix typo in procedures.jsp (#1155)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-02-10 09:48:01 +08:00
binlijin 24823ecfc9
HBASE-23682 Fix NPE when disable DeadServerMetricRegionChore (#1026)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-02-08 15:20:23 -08:00