7675 Commits

Author SHA1 Message Date
Pankaj
cb5651b1e6 HBASE-22806 Recreating a deleted column family brings back the deleted cells (#530)
Signed-off-by: stack <stack@apache.org>
2019-08-24 18:19:20 -07:00
Tai-Ying Lee
da87ebb481
HBASE-20509 putting List in HashSet directly without using addAll function to improve performance
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-24 00:03:04 +02:00
openinx
4275c5f374 HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table (#513)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-08-22 20:20:21 +08:00
openinx
53522f36cc HBASE-22810 Initialize an separate ThreadPoolExecutor for taking/restoring snapshot (addendum - use the old config key) (#517) 2019-08-22 11:53:46 +08:00
Viraj Jasani
e28a7c29c5 HBASE-22863 Cleanup transitive Jackson1 vulnerable dependencies(forward-port HBASE-22728) (#505)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-08-21 22:16:52 +08:00
shahrs87
0a1950f43c HBASE-22601 Misconfigured addition of peers leads to cluster shutdown. (#500)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-08-21 21:56:04 +08:00
Andrew Purtell
22814f8138
Revert "HBASE-22823 Mark Canary as Public/Evolving"
This reverts commit 4b60eaca17b1665ffaa3c8452f4681dbaa52194c.

See HBASE-22874
2019-08-20 09:46:10 -07:00
stack
e9874a6abd HBASE-22882 TestFlushSnapshotFromClient#testConcurrentSnapshottingAttempts is flakey (was written flakey)
Addendum; just remove the test altogether

 Signed-off-by:  Nick Dimiduk <ndimiduk@apache.org>
2019-08-19 14:27:06 -07:00
stack
b7c7b4a570 HBASE-22882 TestFlushSnapshotFromClient#testConcurrentSnapshottingAttempts is flakey (was written flakey) 2019-08-19 12:58:43 -07:00
satanson
29ed4157b3 HBASE-22870 reflection fails to access a private nested class
Signed-off-by Reid Chan <reidchan@apache.org>
2019-08-18 10:30:04 +08:00
Duo Zhang
c295fd7c6a HBASE-22871 Move the DirScanPool out and do not use static field (#504)
Signed-off-by: Zheng Hu <openinx@gmail.com>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-08-17 23:46:38 +08:00
Huon Wilson
8cbe20c382 HBASE-22841 Add more factory functions to TimeRange
These functions make it easier to possible to use
`org.apache.hadoop.hbase.client.Table.CheckAndMutateBuilder#timeRange`
with more interesting ranges, without being forced to use the
deprecated constructors.

Signed-off-by: huzheng <openinx@gmail.com>
2019-08-17 10:12:48 +08:00
Daisuke Kobayashi
f9e20a6189 HBASE-22860 Master's webui returns NPE/HTTP 500 under maintenance mode
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-08-16 10:35:20 +01:00
Guanghao Zhang
43603ec3b9 HBASE-22858 Add HBCK Report to master's header.jsp (#497)
Signed-off-by: Stack <stack@apache.org>
Signed-off-by: Sakthi <sakthi@apache.org>
2019-08-16 14:37:11 +08:00
meiyi
3ff39386f4 HBASE-22842 Tmp directory should not be deleted when master restart used for user scan snapshot feature (#485) 2019-08-16 10:48:26 +08:00
Itsuki Toyota
dfe7e27e25 HBASE-22833 MultiRowRangeFilter should provide a method for creating… (#493)
* HBASE-22833: MultiRowRangeFilter should provide a method for creating a filter which is functionally equivalent to multiple prefix filters

* Delete superfluous comments

* Add description for MultiRowRangeFilter constructor

* Add null check for rowKeyPrefixes

* Fix checkstyle

Signed-off-by: huzheng <openinx@gmail.com>
2019-08-16 10:06:44 +08:00
Caroline Zhou
4b60eaca17
HBASE-22823 Mark Canary as Public/Evolving
Signed-off-by: Xu Cang <xucang@apache.org>
2019-08-15 14:20:41 -07:00
Guanghao Zhang
76a68bd27c HBASE-22857 Fix the failed ut TestHRegion and TestHRegionWithInMemoryFlush (#495)
Signed-off-by: Peter Somogyi psomogyi@apache.org
Signed-off-by: stack stack@apache.org
2019-08-15 13:19:25 -07:00
openinx
6f9db4660a HBASE-22810 Initialize an separate ThreadPoolExecutor for taking/restoring snapshot (#486) 2019-08-15 11:22:08 +08:00
Andrew Purtell
3233f6730f
HBASE-22828 Log a region close journal (#471)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-08-13 15:43:51 -07:00
syedmurtazahassan
c5b29265a3
HBASE-22725 Remove all remaining Javadoc warnings
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-11 22:02:53 +02:00
Geoffrey Jacoby
7f0ab84214
HBASE-22623 - Add RegionObserver coprocessor hook for preWALAppend (#390)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-08-09 14:29:41 -07:00
Daisuke Kobayashi
e4157f0fe6 HBASE-22543 Revisit HBASE-21207 to make all values fully sortable
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-08-09 14:30:15 +01:00
Guanghao Zhang
9f6c37bcc3 HBASE-22824 Show filesystem path for the orphans regions on filesystem (#469)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-08-09 15:07:48 +08:00
Sakthi
65d6390e50 HBASE-22803 Modify config value range to enable turning off of the hbck chore (#466)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-08-08 17:46:00 -07:00
Guanghao Zhang
796cbf9bc3 HBASE-22808 HBCK Report showed the offline regions which belong to disabled table (#465)
Signed-off-by: Stack <stack@apache.org>
2019-08-08 17:00:21 +08:00
meiyi
e70a5a687c HBASE-22776 Rename config names in user scan snapshot feature (#440) 2019-08-08 16:18:58 +08:00
Guanghao Zhang
ee40d47b3a HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem (#461)
Signed-off-by: Sakthi <sakthi@apache.org>
2019-08-08 08:18:40 +08:00
stack
225c6cd5bc HBASE-22777 Add a multi-region merge (for fixing overlaps)
Makes MergeTableRegionsProcedure do more than just two regions at a
time. Compatible as MTRP was done considering one day it'd do more than
two at a time.

Changes hardcoded assumption that merge parent regions are named
mergeA and mergeB in a column on the resultant region. Instead
can have N columns on the merged region, one for each parent
merged. Column qualifiers all being with 'merge'.

Most of code below is undoing the assumption that there are two
parents on a merge only.
2019-08-07 16:58:06 -07:00
syedmurtazahassan
d39736a274 HBASE-22812 InterfaceAudience annotation in CatalogJanitor uses fully-qualified name (#462)
Signed-off-by: Stack <stack@apache.org>
Signed-off-by: Sakthi <sakthi@apache.org>
2019-08-07 10:40:43 -07:00
Wellington Chevreuil
e13ed336fe HBASE-22417 DeleteTableProcedure.deleteFromMeta method should remove table from Master's table descriptors cache
Signed-off-by: stack <stack@apache.org>
2019-08-07 12:39:01 +01:00
Wellington Chevreuil
afb32a3361 HBASE-22731 ReplicationSource and HBaseInterClusterReplicationEndpoint log messages should include a target Peer identifier
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-08-07 10:52:30 +01:00
Andor Molnár
cc4b5103de HBASE-22759 Add user info to AUDITLOG events when doing grant/revoke 2019-08-07 11:07:25 +02:00
sreenivasulureddy
be84e36bee HBASE-22793 RPC server connection is logging user as NULL principal
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Pankaj Kumar <pankaj.kr@huawei.com>
2019-08-06 14:43:55 +02:00
stack
5fa8668202 HBASE-22771 [HBCK2] fixMeta method and server-side support
This is a first cut at this patch. Implements hold fixing only
currently.

Add a fixMeta method to Hbck Interface.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Bug fix. If hole is on end of last table, I wasn't seeing it.

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
 Add a general meta fixer class. Explains up top why this stuff doesn't
 belong inside MetaTableAccessor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 Break out the filesystem messing so don't have to copy it nor do more
 than is needed doing fixup for Region holes.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java
 Change behavious slightly. If directory exists, don't fail as we did
 but try and keep going and create .regioninfo file if missing (or
 overwrite if in place). This should make it idempotent. Can rerun
 command. Lets see if any repercussions in test suite.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
 Add test.

Signed-off-by: Zheng Hu <openinx@gmail.com>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-08-05 22:33:28 -07:00
stack
2bc9526080 HBASE-22771 [HBCK2] fixMeta method and server-side support
This is a first cut at this patch. Implements hold fixing only
currently.

Add a fixMeta method to Hbck Interface.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Bug fix. If hole is on end of last table, I wasn't seeing it.

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
 Add a general meta fixer class. Explains up top why this stuff doesn't
 belong inside MetaTableAccessor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 Break out the filesystem messing so don't have to copy it nor do more
 than is needed doing fixup for Region holes.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java
 Change behavious slightly. If directory exists, don't fail as we did
 but try and keep going and create .regioninfo file if missing (or
 overwrite if in place). This should make it idempotent. Can rerun
 command. Lets see if any repercussions in test suite.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
 Add test.

Signed-off-by: Zheng Hu <openinx@gmail.com>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-08-05 22:33:20 -07:00
Duo Zhang
10d22b5454 HBASE-22539 WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used (#437)
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-08-05 16:45:00 +08:00
Wellington Chevreuil
5cd0433dc9 HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore 2019-08-01 16:56:57 +01:00
Guanghao Zhang
6a20872e01 HBASE-22737 Add a new admin method and shell cmd to trigger the hbck chore to run (#425)
Signed-off-by: stack <stack@apache.org>
2019-08-01 09:02:26 +08:00
Biju Nair
d335c97081 HBASE-22363 Remove hardcoded number of read cache block buckets
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-07-31 09:19:28 +02:00
stack
2caf6db0c5 HBASE-22741 Show catalogjanitor consistency complaints in new 'HBCK Report' page
Signed-off-by: huzheng <openinx@gmail.com>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-07-30 11:17:15 -07:00
openinx
300f906200 HBASE-22758 Remove the unneccesary info cf deletion in DeleteTableProcedure#deleteFromMeta (#424) 2019-07-30 21:37:52 +08:00
stack
4565e9f43b HBASE-22723 Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta
Refactor of CatalogJanitor so it generates a
Report on the state of hbase:meta when it runs. Also
refactor so CJ runs even if RIT (previous it would
punt on running if RIT) so it can generate a 'Report'
on the interval regardless. If RIT, it just doesn't
go on to do the merge/split GC as it used to.

If report finds an issue, dump as a WARN message
to the master log.

Follow-on is to make the Report actionable/available
for the Master to pull when it goes to draw the hbck
UI page (could also consider shipping the Report as
part of ClusterMetrics?)

Adds new, fatter Visitor to CJ, one that generates
Report on each run keeping around more findings as
it runs.

Moved some methods around so class reads better;
previous methods were randomly ordered in the class.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Make a few handy methods public.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add utility as defaults on the Inteface; i.e. is this the first region
 in table, is it last, does a passed region come next, or does passed
 region overlap this region (added tests for this new stuff).

M hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
 Bugfix... handle case where buffer passed is null.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Lots of change, reorg., but mostly adding consistency checking
 to the visitor used scanning hbase:meta on a period and the
 generation of a Report on what the scan has found traversing
 hbase:meta. Added a main so could try the CatalogJanitor against
 a running cluster.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitorCluster.java
 Fat ugly test for CatalogJanitor consistency checking.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
 Add tests for new functionality in RI.

M hbase-shell/src/main/ruby/hbase/table.rb
 Bug fix for case where meta has a null regioninfo; scan was aborting.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-07-29 16:07:10 -07:00
stack
d23236a452 HBASE-22751 table.jsp fails if ugly regions in table 2019-07-29 15:14:41 -07:00
stack
8b1952a42a HBASE-22751 table.jsp fails if ugly regions in table 2019-07-29 15:06:22 -07:00
Venkatesh Sridharan
4a83de75c7 HBASE-22313 Add method to FsDelegation token to accept token kind (#199)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-07-29 11:18:48 +02:00
meiyi
8bd90f2e55 HBASE-22580 Add a table attribute to make user scan snapshot feature configurable for table (#336) 2019-07-29 10:03:54 +08:00
Guanghao Zhang
4c4f0184e0 HBASE-22709 Add a chore thread in master to do hbck checking (#404)
Signed-off-by: stack <stack@apache.org>
2019-07-27 18:31:33 +08:00
Guanghao Zhang
da4d72c2d4 HBASE-22742 [HBCK2] Add more log for hbck operations at master side
Signed-off-by: stack <stack@apache.org>
2019-07-27 12:39:43 +08:00
stack
f7ac3badbc HBASE-22382 Refactor tests in TestFromClientSide; ADDENDUM to fix tests 2019-07-26 13:41:50 -07:00