Commit Graph

16912 Commits

Author SHA1 Message Date
Michael Stack 7ee6d59ef8
HBASE-23092 Make the RM tooling in dev-tools/create-release generic (#671)
Make the scripts generic. Adds an option that allows you specify
'project'. Defaults to 'hbase' for core. Pass 'hbase-thirdparty'
or 'hbase-operator-tools' etc.

This commit includes a bunch of bugfixes and miscellaneous
that came of trying to use the scripts making RCs.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-10-02 09:14:17 -07:00
stack 2ebdcbc3b0 Revert " HBASE-23055 Alter hbase:meta (#667)"
Revert because new feedback and requested survey of master usage
figuring table state.

This reverts commit 52176184df.
2019-10-01 09:24:10 -07:00
Michael Stack 52176184df
HBASE-23055 Alter hbase:meta (#667)
Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.

 Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2019-09-30 09:29:51 -07:00
Viraj Jasani 944108c38a HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command (#611)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 17:35:22 +08:00
Duo Zhang ca0d9f3cd2
HBASE-23079 RegionRemoteProcedureBase should override setTimeoutFailure (#672)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-30 11:48:50 +08:00
chenxu14 ce0fbc2af9 HBASE-22965 RS Crash due to DBE reference to an reused ByteBuff (#603)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 09:17:07 +08:00
Guanghao Zhang 1170f28122
HBASE-23078 BaseLoadBalancer should consider region replicas when randomAssignment and roundRobinAssignment (#663)
Signed-off-by: stack <stack@apache.org>
2019-09-29 17:36:07 +08:00
Duo Zhang c1a476c492
HBASE-23087 Remove the deprecated bulkload method in AsyncClusterConnection (#670)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-09-29 09:26:40 +08:00
Ankit Singhal 836368189a HBASE-23054 Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm 2019-09-28 09:29:17 -07:00
Guanghao Zhang ef79b40e6e
HBASE-23035 Retain region to the last RegionServer make the failover slower (addendum) (#652) 2019-09-28 20:25:05 +08:00
Sanjeet Nishad b45c0d0efa HBASE-23038 Provide consistent and clear logging about disabling chores
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-09-27 15:01:15 -05:00
shardul-cr7 3250a8021a HBASE-22012 Prevent DISABLE quota violation policy from disabling violation policy
Space quotas has a feature which intends to avoid enacting a space quota
violation policy when only a subset of the Regions for that Table have
reported their space usage (under the assumption that we cannot make an
informed decision if we do not include all regions in our calculations).

This had the unintended side-effect, when a table is disabled as a part
of a violation policy, of causing the regions for that table to not be
reported which disables the violation policy and enables the table.

Need to make sure that when a table is disabled because of a violation
policy that the code does not automatically move that table out of
violation because region sizes are not being reported (because those
regions are not open).

Closes #572

Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-26 16:15:16 -04:00
Wellington Chevreuil 277c1cc97f Amend HBASE-22380 break circle replication when doing bulkload
Fixing tests broken by this commit: TestBulkLoadHFilesSplitRecovery and TestSecureBulkLoadHFilesSplitRecovery.
2019-09-26 13:35:35 +01:00
BukrosSzabolcs f0dddd1cc2 HBASE-22982: region server suspend/resume and graceful rolling restart actions (#592)
* Add chaos monkey action for suspend/resume region servers
* Add chaos monkey action for graceful rolling restart
* Add these to relevant chaos monkeys

Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-26 10:07:38 +02:00
SteNicholas ea24ea7dd5 HBASE-23075 Upgrade jackson version (#660)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-26 10:09:57 +08:00
Sean Busbey 52f5a85bfc
HBASE-22968 Update website for branch-2.0 EOL (#659)
* Remove from downloads
* Remove nav menu item
* Remove from compat ref guide
* Remove from list of release managers

Signed-off-by: stack <stack@apache.org>
2019-09-24 12:51:15 -05:00
Peter Somogyi 90c8f5e32f
HBASE-22956 Add hbase-operator-tools 1.0.0 to downloads (#658)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-09-24 15:45:13 +02:00
Peter Somogyi 56489819bc
HBASE-23053 Disable concurrent nightly builds (#654)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-09-23 15:00:22 +02:00
Balazs Meszaros d3c1bb42e9
HBASE-23032 Upgrade to Curator 4.2.0
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-23 11:59:06 +02:00
zbq.dean a8e3d23cca HBASE-22975 Add read and write QPS metrics at server level and table level (#615)
Signed-off-by Reid Chan <reidchan@apache.org>
2019-09-23 12:51:25 +08:00
Pankaj 537c5f0660 HBASE-22930 Set unique name to longCompactions/shortCompactions/split threads (#585) 2019-09-22 12:25:02 +02:00
Guanghao Zhang cb04c6c1a6
HBASE-23035 Retain region to the last RegionServer make the failover slower (#631)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-21 21:11:05 +08:00
Wellington Ramos Chevreuil 3ae353cbf4
HBASE-22380 break circle replication when doing bulkload (#566)
Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-21 14:02:18 +01:00
wuqiong 44cdf5f4ea HBASE-23058 Should be 'Column Family Name' in table.jsp
Signed-off-by: Guanghao Zhang <zhangguanghao1@xiaomi.com>
2019-09-21 19:24:32 +08:00
shardul-cr7 96a94ac3d0 HBASE-22142 Drop table RegionSizes with namespace quota
There was a bug in which we would not drop the RegionSizes
for a table in a namespace, where the namespace had a quota
on it. This allowed a scenario in which recreation of a table
inside of a namespace would unintentionally move into violation
despite the table being empty. Need to make sure the RegionSizes
are dropped on table deletion if there is _any_ quota applying
to that table.

Closes #598

Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-20 13:50:58 -04:00
shardul-cr7 2b3c9b18dd HBASE-22944 Check for hbase:quota table existence in SpaceQuotaRefresherChore
During startup, it's possible that quotas are enabled but the Master has
not yet created the hbase:quotas table.

Closes #559

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-20 13:50:43 -04:00
Duo Zhang e890776fe0 HBASE-22941 Addendum fix NPE in UT 2019-09-20 14:35:56 +08:00
sreenivasulureddy 49718b8b46
HBASE-22649: Encode StoreFile path URLs in the UI to handle scenarios where CF contains special characters (like # etc.)
Signed-off-by: Sakthi<sakthi@apache.org>
2019-09-19 14:14:15 -07:00
Stig Døssing 0ec0a9403f HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… (#361)
* HBase-22027: Split non-MR related parts of TokenUtil off into a ClientTokenUtil, and move ClientTokenUtil to hbase-client

* Replace uses of deprecated TokenUtil methods with ClientTokenUtil methods. Make methods that don't need to be public package-private

* Don't use reflection where not necessary in TestClientTokenUtil

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-09-19 12:42:04 -07:00
Peter Somogyi 1fec3d1dfd
HBASE-23051 Remove unneeded Mockito.mock invocations (#641)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2019-09-19 19:28:58 +02:00
Wellington Chevreuil a85c6b4867 HBASE-23047 ChecksumUtil.validateChecksum logs an INFO message inside a "if(LOG.isTraceEnabled())" block.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-19 15:32:03 +01:00
Peter Somogyi db0049558e
HBASE-22927 Upgrade Mockito version for jdk11 - ADDENDUM (#643)
Use correct version for extra-enforcer-rules

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 15:39:50 +02:00
Guanghao Zhang 343a6a7913
HBASE-23049 TableDescriptors#getAll should return the tables ordering by the name which contain namespace (#639)
Signed-off-by: stack <stack@apache.org>
2019-09-19 21:20:42 +08:00
Peter Somogyi 1655360951 Revert "HBASE-22700 refactor isMetaClearingException (#501)"
This reverts commit cb62f73406.
2019-09-19 13:21:22 +02:00
Peter Somogyi 08b82c5c8c
HBASE-23046 Remove compatibility case from truncate command (#638)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-09-19 09:52:39 +02:00
Guanghao Zhang a0e8723b73
HBASE-23041 Should not show split parent regions in HBCK report's unknown server part (#634)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 10:59:01 +08:00
Guanghao Zhang 20bfb43db6
HBASE-23044 CatalogJanitor#cleanMergeQualifier may clean wrong parent regions (#637)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 08:08:49 +08:00
Guanghao Zhang 25bcc91dab
HBASE-23037 Make the split WAL related log more readable (#632)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-18 21:44:05 +08:00
ravowlga123 18ac61091d HBASE-22927 Upgrade Mockito version for jdk11 (#593)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-18 21:16:19 +08:00
Peter Somogyi 759343b7bc HBASE-23043 Use 'advanced' Netty leak detection (#636)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-18 20:37:19 +08:00
Sean Busbey ab076b0731 HBASE-23040 give friendly message about unknown server when running unload for RegionMover.
closes #635

Signed-off-by: stack <stack@apache.org>
2019-09-18 06:29:43 -05:00
johnhomsea cb62f73406 HBASE-22700 refactor isMetaClearingException (#501) 2019-09-17 10:02:37 -07:00
Joey fe74e6e3ac HBASE-23030 Update completebulkload hadoop jar usage in related doc (#630)
Signed-off-by: stack <stack@apache.org>
2019-09-17 09:39:23 -07:00
Viraj Jasani 0dbae8f0e1 HBASE-23024 Replace initcause with Constructor arg (#627)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-17 13:39:26 +02:00
Wellington Ramos Chevreuil 9198525501 HBASE-22846 Internal Error 500 when Using HBASE REST API to Create Na… (#524)
Signed-off-by: stack <stack@apache.org>
(cherry picked from commit f6ff970f39)
2019-09-17 10:50:46 +01:00
Sean Busbey bcad0d9f98 HBASE-22955 Properly set jdk7 as JAVA_HOME for branches-1 precommit and nightly. (#621)
master/branches-2 specific changes: work around yetus overwriting JAVA_HOME
in the container with the host JAVA_HOME.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
(cherry picked from commit 41990ba20a)
2019-09-16 22:51:24 -05:00
Guanghao Zhang 1ae6cccef4
HBASE-23028 Add 2.2.1 to download page (#629)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-17 07:58:50 +08:00
Caroline Zhou 7da61ff174
HBASE-22804 Provide an API to get list of successful regions and total expected regions in Canary
Signed-off-by: Xu Cang <xucang@apache.org>
2019-09-16 11:15:27 -07:00
Sean Busbey 1cb4f687c1
HBASE-23023 upgrade shellcheck used in dockerfile (#624)
Signed-off-by: stack <stack@apache.org>
2019-09-14 14:03:02 -05:00
Yiran Wu f31301dcf8 HBASE-22939 SpaceQuotas - Bulkload from different hdfs failed when space quotas are turned on. (#553)
Signed-off-by: Sakthi <sakthi@apache.org>
2019-09-14 10:46:29 -07:00