Commit Graph

2217 Commits

Author SHA1 Message Date
Wellington Ramos Chevreuil fec4c52499
HBASE-23136 PartionedMobFileCompactor bulkloaded files shouldn't get replicated
Signed-off-by: stack <stack@apache.org>
2019-10-10 10:37:42 +01:00
Istvan Toth 800c35a30e HBASE-23119 ArrayIndexOutOfBoundsException in PrivateCellUtil#qualifierStartsWith (#688)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-10-04 15:13:03 +02:00
stack 2ebdcbc3b0 Revert " HBASE-23055 Alter hbase:meta (#667)"
Revert because new feedback and requested survey of master usage
figuring table state.

This reverts commit 52176184df.
2019-10-01 09:24:10 -07:00
Michael Stack 52176184df
HBASE-23055 Alter hbase:meta (#667)
Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.

 Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2019-09-30 09:29:51 -07:00
Viraj Jasani 944108c38a HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command (#611)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 17:35:22 +08:00
Wellington Ramos Chevreuil 3ae353cbf4
HBASE-22380 break circle replication when doing bulkload (#566)
Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-21 14:02:18 +01:00
Stig Døssing 0ec0a9403f HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… (#361)
* HBase-22027: Split non-MR related parts of TokenUtil off into a ClientTokenUtil, and move ClientTokenUtil to hbase-client

* Replace uses of deprecated TokenUtil methods with ClientTokenUtil methods. Make methods that don't need to be public package-private

* Don't use reflection where not necessary in TestClientTokenUtil

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-09-19 12:42:04 -07:00
Peter Somogyi 1655360951 Revert "HBASE-22700 refactor isMetaClearingException (#501)"
This reverts commit cb62f73406.
2019-09-19 13:21:22 +02:00
Guanghao Zhang 20bfb43db6
HBASE-23044 CatalogJanitor#cleanMergeQualifier may clean wrong parent regions (#637)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 08:08:49 +08:00
johnhomsea cb62f73406 HBASE-22700 refactor isMetaClearingException (#501) 2019-09-17 10:02:37 -07:00
Viraj Jasani 0dbae8f0e1 HBASE-23024 Replace initcause with Constructor arg (#627)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-17 13:39:26 +02:00
Viraj Jasani 1dcc8ee50c HBASE-22760 : Pause/Resume/Query Snapshot Auto Cleanup Activity (#617) 2019-09-13 10:34:01 +05:30
Guanghao Zhang a58149827b HBASE-23014 Should not show split parent regions in hbck report UI (#609) 2019-09-11 10:07:25 -07:00
stack e9e6202f9c HBASE-22796 [HBCK2] Add fix of overlaps to fixMeta hbck Service
Signed-off-by: Sakthi <sakthi@apache.org>
2019-09-10 17:04:06 -07:00
Duo Zhang 4ca760fe9d Revert "HBASE-22664 Move protobuf stuff in hbase-rsgroup to hbase-protocol-shaded (#362)"
This reverts commit d36436408e.
2019-09-10 11:14:37 +08:00
Duo Zhang 936bb82908 Revert "HBASE-22695 Store the rsgroup of a table in table configuration (#426)"
This reverts commit eab7d3d502.
2019-09-10 11:14:17 +08:00
Duo Zhang eab7d3d502 HBASE-22695 Store the rsgroup of a table in table configuration (#426)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-10 10:47:44 +08:00
Duo Zhang d36436408e HBASE-22664 Move protobuf stuff in hbase-rsgroup to hbase-protocol-shaded (#362)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-10 10:47:44 +08:00
Duo Zhang 693f201f3d
HBASE-22963 Netty ByteBuf leak in rpc client implementation (#577)
Signed-off-by: Michael Stack <stack@apache.org>
2019-09-08 21:54:09 +08:00
stack 960a5fdc2a HBASE-22970 split parents show as overlaps in the HBCK Report
Check if overlap is split parent.

Cleaned up the HBCK Report page too with some notes that it is made of
two reports; have the two sections display the same.
2019-09-03 14:48:04 -07:00
meiyi 1d1813de48
HBASE-22878 Show table throttle quotas in table jsp (#510) 2019-09-03 09:37:49 +08:00
ravowlga123 43e732c177 HBASE-22886 Code Coverage Improvement: Create Unit Tests for class ConnectionId
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-09-02 22:35:47 +02:00
chenxu14 d32bf8daed HBASE-22905 Avoid temp ByteBuffer allocation in (#538)
BlockingRpcConnection#writeRequest
2019-09-01 22:01:36 -07:00
meiyi 345c21dbe7 HBASE-22945 Show quota infos in master UI (#560)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-01 22:42:47 +08:00
syedmurtazahassan b642ee0418 HBASE-22766 Code Coverage Improvement: Create Unit Tests for ResultStatsUtil (#520)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-08-31 22:24:05 +08:00
Jan Hentschel 5106f2826e
HBASE-22744 Removed deprecated status and load classes in client module
Signed-off-by: stack <stack@apache.org>
2019-08-27 19:46:08 +02:00
wuguihu 2d4580106a
HBASE-21400 correct spelling error of 'initilize' in comment
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-23 21:31:41 +02:00
Jan Hentschel 808f051b8a
HBASE-22755 Removed deprecated methods from Mutation
Signed-off-by: stack <stack@apache.org>
2019-08-23 20:56:19 +02:00
Jan Hentschel 1b50404f34
HBASE-22844 Fixed Checkstyle violations in client snapshot exceptions
Signed-off-by: stack <stack@apache.org>
2019-08-21 10:30:26 +02:00
Itsuki Toyota 78f5d8ebb2 HBASE-22833 MultiRowRangeFilter should provide a method for creating… (#493)
* HBASE-22833: MultiRowRangeFilter should provide a method for creating a filter which is functionally equivalent to multiple prefix filters

* Delete superfluous comments

* Add description for MultiRowRangeFilter constructor

* Add null check for rowKeyPrefixes

* Fix checkstyle

Signed-off-by: huzheng <openinx@gmail.com>
2019-08-16 09:59:01 +08:00
Sakthi e69af5affe HBASE-22845 Revert MetaTableAccessor#makePutFromTableState access to public (#489)
HBCK2 is dependent on it
2019-08-13 11:26:21 -07:00
Duo Zhang 07ab9ed237 Revert "HBASE-22699 refactor isMetaClearingException (#436)"
This reverts commit 009851d680.
2019-08-13 16:54:06 +08:00
johnhomsea 009851d680 HBASE-22699 refactor isMetaClearingException (#436)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-08-12 15:57:21 +08:00
Jan Hentschel 3867fae04d
HBASE-22829 Removed deprecated methods from SnapshotDescription
Signed-off-by: stack <stack@apache.org>
2019-08-12 08:41:05 +02:00
syedmurtazahassan 15c903b3f8 HBASE-22725 Remove all remaining Javadoc warnings
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-11 21:48:08 +02:00
stack 1b168cd6de HBASE-22777 Add a multi-region merge (for fixing overlaps)
Makes MergeTableRegionsProcedure do more than just two regions at a
time. Compatible as MTRP was done considering one day it'd do more than
two at a time.

Changes hardcoded assumption that merge parent regions are named
mergeA and mergeB in a column on the resultant region. Instead
can have N columns on the merged region, one for each parent
merged. Column qualifiers all being with 'merge'.

Most of code below is undoing the assumption that there are two
parents on a merge only.
2019-08-07 15:22:07 -07:00
stack 30805be078 HBASE-22771 [HBCK2] fixMeta method and server-side support
This is a first cut at this patch. Implements hold fixing only
currently.

Add a fixMeta method to Hbck Interface.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Bug fix. If hole is on end of last table, I wasn't seeing it.

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
 Add a general meta fixer class. Explains up top why this stuff doesn't
 belong inside MetaTableAccessor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 Break out the filesystem messing so don't have to copy it nor do more
 than is needed doing fixup for Region holes.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java
 Change behavious slightly. If directory exists, don't fail as we did
 but try and keep going and create .regioninfo file if missing (or
 overwrite if in place). This should make it idempotent. Can rerun
 command. Lets see if any repercussions in test suite.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
 Add test.

Signed-off-by: Zheng Hu <openinx@gmail.com>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-08-05 22:08:31 -07:00
Jan Hentschel da114ac727
HBASE-22785 Fixed Checkstyle issues in exceptions and enhanced Javadoc
Signed-off-by: stack <stack@apache.org>
2019-08-05 15:05:49 +02:00
Jan Hentschel b021c364fe
HBASE-22786 Fixed Checkstyle issues in tests in hbase-client
Signed-off-by: stack <stack@apache.org>
2019-08-05 11:52:53 +02:00
Wellington Chevreuil 237229cade HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore
Signed-off-by: stack <stack@apache.org>
2019-08-01 16:49:57 +01:00
Guanghao Zhang 35acf3cb78
HBASE-22737 Add a new admin method and shell cmd to trigger the hbck chore to run (#425)
Signed-off-by: stack <stack@apache.org>
2019-08-01 08:54:47 +08:00
Jan Hentschel 24b970ec51
Revert "HBASE-22744 Removed deprecated status and load classes in client module"
This reverts commit 4a61c8b10a.
2019-07-31 12:29:19 +02:00
Jan Hentschel 586e177ded HBASE-22753 Removed deprecated ImmutableHRegionInfo (#420)
Signed-off-by: stack <stack@apache.org>
2019-07-31 08:07:31 +02:00
Thanh NM 5ce31dd120 HBASE-22714 - Fix opertationTimeout typo (#392)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-07-30 21:57:06 +08:00
Jan Hentschel 4a61c8b10a
HBASE-22744 Removed deprecated status and load classes in client module
Signed-off-by: stack <stack@apache.org>
2019-07-30 12:40:42 +02:00
Jan Hentschel f9fd5b65fa HBASE-22752 Removed the deprecated ImmutableHColumnDescriptor and ImmutableHTableDescriptor (#419)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-07-30 12:32:10 +02:00
Jan Hentschel 46a432386e
HBASE-22746 Removed deprecated methods from Delete
Signed-off-by: stack <stack@apache.org>
2019-07-30 08:52:19 +02:00
openinx c286a31590
HBASE-22758 Remove the unneccesary info cf deletion in DeleteTableProcedure#deleteFromMeta (#424) 2019-07-30 14:16:08 +08:00
stack 0c80d5b42b HBASE-22723 Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta
Refactor of CatalogJanitor so it generates a
Report on the state of hbase:meta when it runs. Also
refactor so CJ runs even if RIT (previous it would
punt on running if RIT) so it can generate a 'Report'
on the interval regardless. If RIT, it just doesn't
go on to do the merge/split GC as it used to.

If report finds an issue, dump as a WARN message
to the master log.

Follow-on is to make the Report actionable/available
for the Master to pull when it goes to draw the hbck
UI page (could also consider shipping the Report as
part of ClusterMetrics?)

Adds new, fatter Visitor to CJ, one that generates
Report on each run keeping around more findings as
it runs.

Moved some methods around so class reads better;
previous methods were randomly ordered in the class.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Make a few handy methods public.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add utility as defaults on the Inteface; i.e. is this the first region
 in table, is it last, does a passed region come next, or does passed
 region overlap this region (added tests for this new stuff).

M hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
 Bugfix... handle case where buffer passed is null.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Lots of change, reorg., but mostly adding consistency checking
 to the visitor used scanning hbase:meta on a period and the
 generation of a Report on what the scan has found traversing
 hbase:meta. Added a main so could try the CatalogJanitor against
 a running cluster.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitorCluster.java
 Fat ugly test for CatalogJanitor consistency checking.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
 Add tests for new functionality in RI.

M hbase-shell/src/main/ruby/hbase/table.rb
 Bug fix for case where meta has a null regioninfo; scan was aborting.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-07-29 16:00:57 -07:00
Jan Hentschel 41490e792a HBASE-22754 Removed deprecated methods from Increment (#421)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-07-29 09:10:51 +02:00