Commit Graph

7304 Commits

Author SHA1 Message Date
satanson 9e18049957 HBASE-22870 reflection fails to access a private nested class
Signed-off-by Reid Chan <reidchan@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
2019-08-18 13:25:41 +08:00
Duo Zhang befbcbaca8 HBASE-22871 Move the DirScanPool out and do not use static field (#504)
Signed-off-by: Zheng Hu <openinx@gmail.com>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-08-17 23:57:45 +08:00
Huon Wilson a4e3136614 HBASE-22841 Add more factory functions to TimeRange
These functions make it easier to possible to use
`org.apache.hadoop.hbase.client.Table.CheckAndMutateBuilder#timeRange`
with more interesting ranges, without being forced to use the
deprecated constructors.

Signed-off-by: huzheng <openinx@gmail.com>
2019-08-17 09:48:05 +08:00
Daisuke Kobayashi 69debb69e1 HBASE-22860 Master's webui returns NPE/HTTP 500 under maintenance mode
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-08-16 11:02:34 +01:00
Guanghao Zhang 386b6e3aca HBASE-22858 Add HBCK Report to master's header.jsp (#497)
Signed-off-by: Stack <stack@apache.org>
Signed-off-by: Sakthi <sakthi@apache.org>
2019-08-16 14:40:22 +08:00
Itsuki Toyota 6e273e8fdc HBASE-22833 MultiRowRangeFilter should provide a method for creating… (#493)
* HBASE-22833: MultiRowRangeFilter should provide a method for creating a filter which is functionally equivalent to multiple prefix filters

* Delete superfluous comments

* Add description for MultiRowRangeFilter constructor

* Add null check for rowKeyPrefixes

* Fix checkstyle

Signed-off-by: huzheng <openinx@gmail.com>
2019-08-16 10:03:28 +08:00
Caroline Zhou 0481b04ca3
HBASE-22823 Mark Canary as Public/Evolving
Signed-off-by: Xu Cang <xucang@apache.org>
2019-08-15 14:22:21 -07:00
openinx f49a86ae4f HBASE-22810 Initialize an separate ThreadPoolExecutor for taking/restoring snapshot (#486) 2019-08-15 11:14:15 +08:00
Duo Zhang e1f91a5415 HBASE-22848 Set version as 2.1.6 in branch-2.1 in prep for first RC of 2.1.6 2019-08-14 11:48:14 +08:00
Andrew Purtell e27ef92e3f
HBASE-22828 Log a region close journal (#471)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-08-13 15:44:03 -07:00
Ankit Singhal 9ee32d8f64 HBASE-22840 Fix backport of HBASE-21325 2019-08-12 21:02:58 -07:00
syedmurtazahassan c9d02f9535
HBASE-22725 Remove all remaining Javadoc warnings
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-11 22:24:23 +02:00
Guanghao Zhang aaa9d93715 HBASE-22824 Show filesystem path for the orphans regions on filesystem (#469)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-08-09 15:25:48 +08:00
Sakthi 2a707ad334 HBASE-22803 Modify config value range to enable turning off of the hbck chore (#466)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-08-08 23:42:48 -07:00
stack ea08907e08 HBASE-22777 Add a multi-region merge (for fixing overlaps)
Makes MergeTableRegionsProcedure do more than just two regions at a
time. Compatible as MTRP was done considering one day it'd do more than
two at a time.

Changes hardcoded assumption that merge parent regions are named
mergeA and mergeB in a column on the resultant region. Instead
can have N columns on the merged region, one for each parent
merged. Column qualifiers all being with 'merge'.

Most of code below is undoing the assumption that there are two
parents on a merge only.
2019-08-08 14:35:07 -07:00
Guanghao Zhang a5d981c3d2 HBASE-22808 HBCK Report showed the offline regions which belong to disabled table (#465)
Signed-off-by: Stack <stack@apache.org>
2019-08-08 17:17:27 +08:00
Guanghao Zhang 5cec62f0f9 HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem (#461)
Signed-off-by: Sakthi <sakthi@apache.org>
2019-08-08 08:29:37 +08:00
syedmurtazahassan d4d58f3250 HBASE-22812 InterfaceAudience annotation in CatalogJanitor uses fully-qualified name (#462)
Signed-off-by: Stack <stack@apache.org>
Signed-off-by: Sakthi <sakthi@apache.org>
2019-08-07 10:56:07 -07:00
Wellington Chevreuil 0d50ce1b11 HBASE-22417 DeleteTableProcedure.deleteFromMeta method should remove table from Master's table descriptors cache
Signed-off-by: stack <stack@apache.org>
2019-08-07 15:16:57 +01:00
Sergey Shelukhin 78c4e4996a HBASE-22115 HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe
Signed-off-by: Andrew Purtell <apurtell@apache.org>, stack <stack@apache.org>
2019-08-07 21:14:31 +08:00
Wellington Chevreuil 46d0e5a05d HBASE-22731 ReplicationSource and HBaseInterClusterReplicationEndpoint log messages should include a target Peer identifier
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-08-07 11:09:04 +01:00
stack f3b205d396 HBASE-22805 Fix broken unit test, TestCatalogJanitorCluster on branch-2.1 and branch-2.0 2019-08-06 13:38:49 -07:00
Peter Somogyi c71285f0ae HBASE-22771 [HBCK2] fixMeta method and server-side support; ADDENDUM (#457)
Remove HRegion.java.rej file
2019-08-06 08:10:56 -07:00
sreenivasulureddy 935d271f77 HBASE-22793 RPC server connection is logging user as NULL principal
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Pankaj Kumar <pankaj.kr@huawei.com>
2019-08-06 14:50:10 +02:00
Andor Molnár 9a6494f02f HBASE-22759 Extended grant and revoke audit events with caller info - ADDENDUM
Added remote address to grant/revoke audit log messages
2019-08-06 11:44:13 +02:00
stack 94e35d8cdf HBASE-22771 [HBCK2] fixMeta method and server-side support; ADDENDUM 2019-08-05 22:56:21 -07:00
stack db440d3099 HBASE-22771 [HBCK2] fixMeta method and server-side support
This is a first cut at this patch. Implements hold fixing only
currently.

Add a fixMeta method to Hbck Interface.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Bug fix. If hole is on end of last table, I wasn't seeing it.

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
 Add a general meta fixer class. Explains up top why this stuff doesn't
 belong inside MetaTableAccessor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 Break out the filesystem messing so don't have to copy it nor do more
 than is needed doing fixup for Region holes.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java
 Change behavious slightly. If directory exists, don't fail as we did
 but try and keep going and create .regioninfo file if missing (or
 overwrite if in place). This should make it idempotent. Can rerun
 command. Lets see if any repercussions in test suite.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
 Add test.

Signed-off-by: Zheng Hu <openinx@gmail.com>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-08-05 22:51:53 -07:00
Duo Zhang fe2409971c HBASE-22539 WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used (#437)
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-08-05 16:52:07 +08:00
Guanghao Zhang e68b16a6c1 HBASE-22737 Add a new admin method and shell cmd to trigger the hbck chore to run (#425)
Signed-off-by: stack <stack@apache.org>
2019-08-01 10:43:57 +08:00
Biju Nair c8ab562338 HBASE-22363 Remove hardcoded number of read cache block buckets
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-07-31 09:26:32 +02:00
stack 9799e0b6b2 HBASE-22741 Show catalogjanitor consistency complaints in new 'HBCK Report' page
Signed-off-by: huzheng <openinx@gmail.com>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-07-30 11:32:46 -07:00
stack 4587b39e63 Revert "Revert "HBASE-22723 Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta""
This reverts commit bf36bdf2b6.
2019-07-30 09:21:47 -07:00
openinx 892aa832ad HBASE-22758 Remove the unneccesary info cf deletion in DeleteTableProcedure#deleteFromMeta (#424) 2019-07-30 21:48:46 +08:00
Andor Molnár 29f8e195ef HBASE-22759. Extended grant and revoke audit events with caller info (#427) 2019-07-30 12:50:57 +02:00
stack efead31eb2 HBASE-22751 table.jsp fails if ugly regions in table 2019-07-30 10:30:27 +02:00
Peter Somogyi bf36bdf2b6 Revert "HBASE-22723 Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta"
This reverts commit 65a6b270dd.
2019-07-30 10:12:27 +02:00
Peter Somogyi dcc2a2b8d8 Revert "HBASE-22751 table.jsp fails if ugly regions in table"
This reverts commit d45b2d4469.
2019-07-30 10:12:03 +02:00
stack 65a6b270dd HBASE-22723 Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta
Refactor of CatalogJanitor so it generates a
Report on the state of hbase:meta when it runs. Also
refactor so CJ runs even if RIT (previous it would
punt on running if RIT) so it can generate a 'Report'
on the interval regardless. If RIT, it just doesn't
go on to do the merge/split GC as it used to.

If report finds an issue, dump as a WARN message
to the master log.

Follow-on is to make the Report actionable/available
for the Master to pull when it goes to draw the hbck
UI page (could also consider shipping the Report as
part of ClusterMetrics?)

Adds new, fatter Visitor to CJ, one that generates
Report on each run keeping around more findings as
it runs.

Moved some methods around so class reads better;
previous methods were randomly ordered in the class.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Make a few handy methods public.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add utility as defaults on the Inteface; i.e. is this the first region
 in table, is it last, does a passed region come next, or does passed
 region overlap this region (added tests for this new stuff).

M hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
 Bugfix... handle case where buffer passed is null.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Lots of change, reorg., but mostly adding consistency checking
 to the visitor used scanning hbase:meta on a period and the
 generation of a Report on what the scan has found traversing
 hbase:meta. Added a main so could try the CatalogJanitor against
 a running cluster.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitorCluster.java
 Fat ugly test for CatalogJanitor consistency checking.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
 Add tests for new functionality in RI.

M hbase-shell/src/main/ruby/hbase/table.rb
 Bug fix for case where meta has a null regioninfo; scan was aborting.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-07-29 16:21:05 -07:00
stack d45b2d4469 HBASE-22751 table.jsp fails if ugly regions in table 2019-07-29 15:15:32 -07:00
Guanghao Zhang e54c4018e7 HBASE-22709 Add a chore thread in master to do hbck checking (#404)
Signed-off-by: stack <stack@apache.org>
2019-07-27 18:55:22 +08:00
Guanghao Zhang f8b592e9f8 HBASE-22742 [HBCK2] Add more log for hbck operations at master side
Signed-off-by: stack <stack@apache.org>
2019-07-27 12:41:54 +08:00
stack fd267168d8 HBASE-22382 Refactor tests in TestFromClientSide; ADDENDUM to fix tests 2019-07-26 13:42:45 -07:00
linkaline 100a625324 HBASE-22733 TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor is flakey (#408)
Signed-off-by: stack <stack@apache.org>
2019-07-25 11:16:16 -07:00
Andor Molnár 884843d085 HBASE-22382 Refactor tests in TestFromClientSide (#385)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-07-25 10:41:31 +08:00
Guanghao Zhang 56417284bf HBASE-22721 Refactor HBaseFsck: move the inner class out
Signed-off-by: stack <stack@apache.org>
2019-07-25 08:32:19 +08:00
Jeongdae Kim d88ee0447a
HBASE-22715 All scan requests should be handled by scan handler threads in RWQueueRpcExecutor (#393)
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcScheduler.java
2019-07-24 11:56:48 -07:00
syedmurtazahassan 6a46e071c5
HBASE-22603 Javadoc Warnings related to @link tag
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-07-23 14:46:24 +02:00
syedmurtazahassan 49fcd92bdf HBASE-22610 [BucketCache] Rename hbase.offheapcache.minblocksize
Co-authored-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-07-23 12:05:51 +08:00
Toshihiro Suzuki 03f9fc3aa8 HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+
Signed-off-by: stack <stack@apache.org>
2019-07-23 10:22:54 +09:00
Guanghao Zhang 727ebbddac HBASE-22527 [hbck2] Add a master web ui to show the problematic regions 2019-07-18 08:11:06 -07:00