Commit Graph

2247 Commits

Author SHA1 Message Date
stack db7fb063d1 HBASE-23596 HBCKServerCrashProcedure can double assign
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

Change its behavior so it will only look in hbase:meta
if the call to the super class turns up zero references.
Only then will it search hbase:meta for references to
'Unknown Servers'. Normal operation where we read Master
context is usual and sufficient. The scan of hbase:meta
is only for case where Master state has been corrupted
and we need to clear out 'Unknown Servers'.
2020-01-02 14:33:54 -08:00
belugabehr 280b94434c HBASE-23628: Remove Apache Commons Digest Base64 (#977)
Signed-off-by: stack <stack@apache.org>
2020-01-02 12:06:19 -08:00
Viraj Jasani e32dbe8ed2
HBASE-23590 : Update maxStoreFileRefCount to maxCompactedStoreFileRefCount for auto region recovery based on old reader references
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-01-01 22:16:54 +05:30
Duo Zhang 1b049a2d34
HBASE-23326 Implement a ProcedureStore which stores procedures in a HRegion (#941)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-12-25 12:02:12 +08:00
XinSun 270eb9886e HBASE-23376 NPE happens while replica region is moving (#906)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-19 15:11:04 +08:00
Ankit Singhal 29d1a97482 HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-12-16 23:44:05 +09:00
Junegunn Choi ec317a6629 HBASE-23582 Unbalanced braces in string representation of table descriptor
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2019-12-16 10:35:14 +01:00
Bharath Vissapragada efa4fe901a HBASE-23575 Remove dead code in AsyncRegistry (#929)
Removes a bunch of dead code and fixes some checkstyle nits.

Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-12-13 13:26:03 -06:00
stack b0233a294f HBASE-23554 Encoded regionname to regionname utility (#923); ADDENDUM 2019-12-11 11:02:59 -08:00
Michael Stack 8742265d85 HBASE-23554 Encoded regionname to regionname utility (#923)
Adds shell command regioninfo:

  hbase(main):001:0>  regioninfo '0e6aa5c19ae2b2627649dc7708ce27d0'
  {ENCODED => 0e6aa5c19ae2b2627649dc7708ce27d0, NAME => 'TestTable,,1575941375972.0e6aa5c19ae2b2627649dc7708ce27d0.', STARTKEY => '', ENDKEY => '00000000000000000000299441'}
  Took 0.4737 seconds

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-11 08:57:59 -08:00
Nick Dimiduk 8548463e74
HBASE-23373 Log `RetriesExhaustedException` context with full time precision (#903)
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-12-06 11:05:58 -08:00
XinSun 27cfe1bb27 HBASE-23345 Table need to replication unless all of cfs are excluded (#881)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-12-03 13:35:23 +08:00
Wellington Ramos Chevreuil 636fa2c6b0
HBASE-23313 [hbck2] setRegionState should update Master in-memory sta… (#864)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-11-27 08:41:23 +00:00
bsglz dbbba7932c HBASE-23334 The table-lock node of zk is not needed since HBASE-16786 (#873)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2019-11-25 19:49:31 +08:00
Wellington Ramos Chevreuil 3b0c276aa3
HBASE-23329 Remove unused methods from RequestConverter (#870)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-11-22 14:33:22 +00:00
Michael Stack 54ad797abb HBASE-23328 info:regioninfo goes wrong when region replicas enabled (#863)
Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
Signed-off-by: Guangxu Cheng <guangxucheng@gmail.com>
2019-11-21 22:58:24 -08:00
belugabehr 33bedf8d4d HBASE-23308: Review of NullPointerExceptions (#836)
Signed-off-by: stack <stack@apache.org>
2019-11-19 13:51:11 -08:00
stack ca6e67a6de HBASE-23315 Miscellaneous HBCK Report page cleanup
* Add a bit of javadoc around SerialReplicationChecker.
 * Miniscule edit to the profiler jsp page and then a bit of doc on how to make it work that might help.
 * Add some detail if NPE getting BitSetNode to help w/ debug.
 * Change HbckChore to log region names instead of encoded names; helps doing diagnostics; can take region name and query in shell to find out all about the region according to hbase:meta.
 * Add some fix-it help inline in the HBCK Report page – how to fix.
 * Add counts in procedures page so can see if making progress; move listing of WALs to end of the page.
2019-11-19 07:34:24 -08:00
stack f8f9a26cb3 HBASE-23282 HBCKServerCrashProcedure for 'Unknown Servers'
Have the existing scheduleRecoveries launch a new HBCKSCP
instead of SCP. It gets regions to recover from Master
in-memory context AND from a scan of hbase:meta. This
new HBCKSCP is For processing 'Unknown Servers', servers that
are 'dead' and purged but still have references in
hbase:meta. Rare occurance but needs tooling to address.
Later have catalogjanitor take care of these deviations
between Master in-memory and hbase:meta content (usually
because of overdriven cluster with failed RPCs to hbase:meta,
etc)

Changed expireServers in ServerManager so could pass in
custom reaction to expired server.... This is how we
run our custom HBCKSCP while keeping all other aspects
of expiring services (rather than try replicate it
externally).
2019-11-18 14:28:55 -08:00
Mingliang Liu 8bfdfe1b85 HBASE-23289 Update links to Hadoop wiki in book and code
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2019-11-18 08:58:07 -08:00
Udai Bhan Kashyap b5e8727617 HBASE-22969 A new binary component comparator(BinaryComponentComparator) to perform comparison of arbitrary length and position (#829)
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2019-11-18 13:28:06 +01:00
Karthik Palanisamy 8f92a14cd1 HBASE-23203 NPE in RSGroup info (#747)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-25 08:43:34 +08:00
Viraj Jasani 14dcf1d0c6 HBASE-22460 : Reopen regions with very high Store Ref Counts (#600)
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2019-10-23 21:16:47 +05:30
Viraj Jasani 2ad62b0162 HBASE-22679 : Revamping CellUtil (#735)
* HBASE-22679 : Revamping CellUtil

* checkstyle fix

* incorporating review

* minor indentation
2019-10-21 23:04:21 -07:00
Wellington Ramos Chevreuil 4d414020bb
HBASE-23136 (#712)
* HBASE-23136 PartionedMobFileCompactor bulkloaded files shouldn't get replicated (addressing buklload replication related issue raised in HBASE-22380)

Signed-off-by: Josh Elser <elserj@apache.org>
2019-10-18 15:07:11 +01:00
meiyi 946f1e9e25
HBASE-23170 Admin#getRegionServers use ClusterMetrics.Option.SERVERS_NAME (#721) 2019-10-18 09:35:35 +08:00
Guanghao Zhang 7924ba39e7
HBASE-23155 May NPE when concurrent AsyncNonMetaRegionLocator#updateCachedLocationOnError (#718) 2019-10-15 11:16:43 +08:00
Karthik Palanisamy c0a09ccbbd HBASE-23152 Compaction_switch does not work by RegionServer name (#713)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 Signed-off-by: stack <stack@apache.org>
2019-10-12 09:08:20 -07:00
Wellington Chevreuil 38b06c7517 Revert "HBASE-23136 PartionedMobFileCompactor bulkloaded files shouldn't get replicated"
This reverts commit fec4c52499.
2019-10-11 14:45:29 +01:00
Viraj Jasani 2f0b3ac91c HBASE-23093 Avoid Optional Anti-Pattern where possible (#673)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-11 15:28:50 +08:00
Wellington Ramos Chevreuil fec4c52499
HBASE-23136 PartionedMobFileCompactor bulkloaded files shouldn't get replicated
Signed-off-by: stack <stack@apache.org>
2019-10-10 10:37:42 +01:00
Istvan Toth 800c35a30e HBASE-23119 ArrayIndexOutOfBoundsException in PrivateCellUtil#qualifierStartsWith (#688)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-10-04 15:13:03 +02:00
stack 2ebdcbc3b0 Revert " HBASE-23055 Alter hbase:meta (#667)"
Revert because new feedback and requested survey of master usage
figuring table state.

This reverts commit 52176184df.
2019-10-01 09:24:10 -07:00
Michael Stack 52176184df
HBASE-23055 Alter hbase:meta (#667)
Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.

 Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2019-09-30 09:29:51 -07:00
Viraj Jasani 944108c38a HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command (#611)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 17:35:22 +08:00
Wellington Ramos Chevreuil 3ae353cbf4
HBASE-22380 break circle replication when doing bulkload (#566)
Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-21 14:02:18 +01:00
Stig Døssing 0ec0a9403f HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… (#361)
* HBase-22027: Split non-MR related parts of TokenUtil off into a ClientTokenUtil, and move ClientTokenUtil to hbase-client

* Replace uses of deprecated TokenUtil methods with ClientTokenUtil methods. Make methods that don't need to be public package-private

* Don't use reflection where not necessary in TestClientTokenUtil

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-09-19 12:42:04 -07:00
Peter Somogyi 1655360951 Revert "HBASE-22700 refactor isMetaClearingException (#501)"
This reverts commit cb62f73406.
2019-09-19 13:21:22 +02:00
Guanghao Zhang 20bfb43db6
HBASE-23044 CatalogJanitor#cleanMergeQualifier may clean wrong parent regions (#637)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 08:08:49 +08:00
johnhomsea cb62f73406 HBASE-22700 refactor isMetaClearingException (#501) 2019-09-17 10:02:37 -07:00
Viraj Jasani 0dbae8f0e1 HBASE-23024 Replace initcause with Constructor arg (#627)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-17 13:39:26 +02:00
Viraj Jasani 1dcc8ee50c HBASE-22760 : Pause/Resume/Query Snapshot Auto Cleanup Activity (#617) 2019-09-13 10:34:01 +05:30
Guanghao Zhang a58149827b HBASE-23014 Should not show split parent regions in hbck report UI (#609) 2019-09-11 10:07:25 -07:00
stack e9e6202f9c HBASE-22796 [HBCK2] Add fix of overlaps to fixMeta hbck Service
Signed-off-by: Sakthi <sakthi@apache.org>
2019-09-10 17:04:06 -07:00
Duo Zhang 4ca760fe9d Revert "HBASE-22664 Move protobuf stuff in hbase-rsgroup to hbase-protocol-shaded (#362)"
This reverts commit d36436408e.
2019-09-10 11:14:37 +08:00
Duo Zhang 936bb82908 Revert "HBASE-22695 Store the rsgroup of a table in table configuration (#426)"
This reverts commit eab7d3d502.
2019-09-10 11:14:17 +08:00
Duo Zhang eab7d3d502 HBASE-22695 Store the rsgroup of a table in table configuration (#426)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-10 10:47:44 +08:00
Duo Zhang d36436408e HBASE-22664 Move protobuf stuff in hbase-rsgroup to hbase-protocol-shaded (#362)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-10 10:47:44 +08:00
Duo Zhang 693f201f3d
HBASE-22963 Netty ByteBuf leak in rpc client implementation (#577)
Signed-off-by: Michael Stack <stack@apache.org>
2019-09-08 21:54:09 +08:00
stack 960a5fdc2a HBASE-22970 split parents show as overlaps in the HBCK Report
Check if overlap is split parent.

Cleaned up the HBCK Report page too with some notes that it is made of
two reports; have the two sections display the same.
2019-09-03 14:48:04 -07:00