7803 Commits

Author SHA1 Message Date
stack
2ebdcbc3b0 Revert " HBASE-23055 Alter hbase:meta (#667)"
Revert because new feedback and requested survey of master usage
figuring table state.

This reverts commit 52176184df595cf591e7c13fe18d959b2ba039a8.
2019-10-01 09:24:10 -07:00
Michael Stack
52176184df
HBASE-23055 Alter hbase:meta (#667)
Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.

 Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2019-09-30 09:29:51 -07:00
Viraj Jasani
944108c38a HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command (#611)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 17:35:22 +08:00
Duo Zhang
ca0d9f3cd2
HBASE-23079 RegionRemoteProcedureBase should override setTimeoutFailure (#672)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-30 11:48:50 +08:00
chenxu14
ce0fbc2af9 HBASE-22965 RS Crash due to DBE reference to an reused ByteBuff (#603)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 09:17:07 +08:00
Guanghao Zhang
1170f28122
HBASE-23078 BaseLoadBalancer should consider region replicas when randomAssignment and roundRobinAssignment (#663)
Signed-off-by: stack <stack@apache.org>
2019-09-29 17:36:07 +08:00
Duo Zhang
c1a476c492
HBASE-23087 Remove the deprecated bulkload method in AsyncClusterConnection (#670)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-09-29 09:26:40 +08:00
Ankit Singhal
836368189a HBASE-23054 Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm 2019-09-28 09:29:17 -07:00
Guanghao Zhang
ef79b40e6e
HBASE-23035 Retain region to the last RegionServer make the failover slower (addendum) (#652) 2019-09-28 20:25:05 +08:00
Sanjeet Nishad
b45c0d0efa HBASE-23038 Provide consistent and clear logging about disabling chores
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-09-27 15:01:15 -05:00
shardul-cr7
3250a8021a HBASE-22012 Prevent DISABLE quota violation policy from disabling violation policy
Space quotas has a feature which intends to avoid enacting a space quota
violation policy when only a subset of the Regions for that Table have
reported their space usage (under the assumption that we cannot make an
informed decision if we do not include all regions in our calculations).

This had the unintended side-effect, when a table is disabled as a part
of a violation policy, of causing the regions for that table to not be
reported which disables the violation policy and enables the table.

Need to make sure that when a table is disabled because of a violation
policy that the code does not automatically move that table out of
violation because region sizes are not being reported (because those
regions are not open).

Closes #572

Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-26 16:15:16 -04:00
Wellington Chevreuil
277c1cc97f Amend HBASE-22380 break circle replication when doing bulkload
Fixing tests broken by this commit: TestBulkLoadHFilesSplitRecovery and TestSecureBulkLoadHFilesSplitRecovery.
2019-09-26 13:35:35 +01:00
BukrosSzabolcs
f0dddd1cc2 HBASE-22982: region server suspend/resume and graceful rolling restart actions (#592)
* Add chaos monkey action for suspend/resume region servers
* Add chaos monkey action for graceful rolling restart
* Add these to relevant chaos monkeys

Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-26 10:07:38 +02:00
zbq.dean
a8e3d23cca HBASE-22975 Add read and write QPS metrics at server level and table level (#615)
Signed-off-by Reid Chan <reidchan@apache.org>
2019-09-23 12:51:25 +08:00
Pankaj
537c5f0660 HBASE-22930 Set unique name to longCompactions/shortCompactions/split threads (#585) 2019-09-22 12:25:02 +02:00
Guanghao Zhang
cb04c6c1a6
HBASE-23035 Retain region to the last RegionServer make the failover slower (#631)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-21 21:11:05 +08:00
Wellington Ramos Chevreuil
3ae353cbf4
HBASE-22380 break circle replication when doing bulkload (#566)
Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-21 14:02:18 +01:00
wuqiong
44cdf5f4ea HBASE-23058 Should be 'Column Family Name' in table.jsp
Signed-off-by: Guanghao Zhang <zhangguanghao1@xiaomi.com>
2019-09-21 19:24:32 +08:00
shardul-cr7
96a94ac3d0 HBASE-22142 Drop table RegionSizes with namespace quota
There was a bug in which we would not drop the RegionSizes
for a table in a namespace, where the namespace had a quota
on it. This allowed a scenario in which recreation of a table
inside of a namespace would unintentionally move into violation
despite the table being empty. Need to make sure the RegionSizes
are dropped on table deletion if there is _any_ quota applying
to that table.

Closes #598

Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-20 13:50:58 -04:00
shardul-cr7
2b3c9b18dd HBASE-22944 Check for hbase:quota table existence in SpaceQuotaRefresherChore
During startup, it's possible that quotas are enabled but the Master has
not yet created the hbase:quotas table.

Closes #559

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-20 13:50:43 -04:00
Duo Zhang
e890776fe0 HBASE-22941 Addendum fix NPE in UT 2019-09-20 14:35:56 +08:00
sreenivasulureddy
49718b8b46
HBASE-22649: Encode StoreFile path URLs in the UI to handle scenarios where CF contains special characters (like # etc.)
Signed-off-by: Sakthi<sakthi@apache.org>
2019-09-19 14:14:15 -07:00
Stig Døssing
0ec0a9403f HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… (#361)
* HBase-22027: Split non-MR related parts of TokenUtil off into a ClientTokenUtil, and move ClientTokenUtil to hbase-client

* Replace uses of deprecated TokenUtil methods with ClientTokenUtil methods. Make methods that don't need to be public package-private

* Don't use reflection where not necessary in TestClientTokenUtil

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-09-19 12:42:04 -07:00
Peter Somogyi
1fec3d1dfd
HBASE-23051 Remove unneeded Mockito.mock invocations (#641)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2019-09-19 19:28:58 +02:00
Wellington Chevreuil
a85c6b4867 HBASE-23047 ChecksumUtil.validateChecksum logs an INFO message inside a "if(LOG.isTraceEnabled())" block.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-19 15:32:03 +01:00
Guanghao Zhang
343a6a7913
HBASE-23049 TableDescriptors#getAll should return the tables ordering by the name which contain namespace (#639)
Signed-off-by: stack <stack@apache.org>
2019-09-19 21:20:42 +08:00
Peter Somogyi
1655360951 Revert "HBASE-22700 refactor isMetaClearingException (#501)"
This reverts commit cb62f734060e1bd66e2f57716620866dbdfe7da1.
2019-09-19 13:21:22 +02:00
Guanghao Zhang
a0e8723b73
HBASE-23041 Should not show split parent regions in HBCK report's unknown server part (#634)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 10:59:01 +08:00
Guanghao Zhang
20bfb43db6
HBASE-23044 CatalogJanitor#cleanMergeQualifier may clean wrong parent regions (#637)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 08:08:49 +08:00
Guanghao Zhang
25bcc91dab
HBASE-23037 Make the split WAL related log more readable (#632)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-18 21:44:05 +08:00
Sean Busbey
ab076b0731 HBASE-23040 give friendly message about unknown server when running unload for RegionMover.
closes #635

Signed-off-by: stack <stack@apache.org>
2019-09-18 06:29:43 -05:00
johnhomsea
cb62f73406 HBASE-22700 refactor isMetaClearingException (#501) 2019-09-17 10:02:37 -07:00
Viraj Jasani
0dbae8f0e1 HBASE-23024 Replace initcause with Constructor arg (#627)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-17 13:39:26 +02:00
Caroline Zhou
7da61ff174
HBASE-22804 Provide an API to get list of successful regions and total expected regions in Canary
Signed-off-by: Xu Cang <xucang@apache.org>
2019-09-16 11:15:27 -07:00
Yiran Wu
f31301dcf8 HBASE-22939 SpaceQuotas - Bulkload from different hdfs failed when space quotas are turned on. (#553)
Signed-off-by: Sakthi <sakthi@apache.org>
2019-09-14 10:46:29 -07:00
ravowlga123
a09c0c88b6 HBASE-22899 logging improvements for snapshot operations w/large manifests (#547)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-09-14 10:16:49 -07:00
Viraj Jasani
1dcc8ee50c HBASE-22760 : Pause/Resume/Query Snapshot Auto Cleanup Activity (#617) 2019-09-13 10:34:01 +05:30
ramkrish86
c0908d4be4
HBASE-22929 - MemStoreLAB ChunkCreator may memory leak(ram) (#614) 2019-09-13 10:01:46 +05:30
stack
3330e5c2e2 HBASE-22796 [HBCK2] Add fix of overlaps to fixMeta hbck Service
Addendum for branch-2+.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-12 13:30:43 -07:00
Guanghao Zhang
c0e5c15887
HBASE-22993 HBCK report UI showed -1 if hbck chore not running (#610)
Signed-off-by: stack <stack@apache.org>
2019-09-12 09:09:57 +08:00
Guanghao Zhang
a58149827b HBASE-23014 Should not show split parent regions in hbck report UI (#609) 2019-09-11 10:07:25 -07:00
Guanghao Zhang
3cfcee9eda HBASE-23005 Table UI showed exception message when table is disabled (#602)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-09-11 09:53:22 +08:00
stack
e9e6202f9c HBASE-22796 [HBCK2] Add fix of overlaps to fixMeta hbck Service
Signed-off-by: Sakthi <sakthi@apache.org>
2019-09-10 17:04:06 -07:00
Duo Zhang
c2c9984c35 Revert "HBASE-22662 Move RSGroupInfoManager to hbase-server (#368)"
This reverts commit 3032c713e11fcde3f8bf0c9dc4de84380d5ae2cd.
2019-09-10 11:14:30 +08:00
Duo Zhang
124ac4d6c4 Revert "HBASE-22676 Move all the code in hbase-rsgroup to hbase-server and remove hbase-rsgroup module (#399)"
This reverts commit f7bbdde9756e67f23a11efd0a6dac9dc58843058.
2019-09-10 11:14:23 +08:00
Duo Zhang
936bb82908 Revert "HBASE-22695 Store the rsgroup of a table in table configuration (#426)"
This reverts commit eab7d3d502be9a2ac58852c6743f3c7536bbf0c3.
2019-09-10 11:14:17 +08:00
Duo Zhang
1c150e0947 Revert "HBASE-22809 Allow creating table in group when rs group contains no live servers (#464)"
This reverts commit 928ecfb443630ee954896d90add14469c1f4ba9e.
2019-09-10 11:14:10 +08:00
Duo Zhang
4a22c74b3c Revert "HBASE-22820 Do not need to persist default rs group now (#482)"
This reverts commit 97ec5d53c015b1ba4bbc96f25b38f979ddc229fa.
2019-09-10 11:14:04 +08:00
Duo Zhang
64581fa33b Revert "HBASE-22819 Automatically migrate the rs group config for table after HBASE-22695 (#498)"
This reverts commit 9196a15de9d1fbbfe9c23e0ed7599a737e01f61a.
2019-09-10 11:13:58 +08:00
Duo Zhang
5a37c98a29 Revert "HBASE-22729 Start RSGroupInfoManager as default (#555)"
This reverts commit 58dee3ea5bdbab714ac38f1bb73a612f06b8d948.
2019-09-10 11:13:50 +08:00