Commit Graph

7768 Commits

Author SHA1 Message Date
Andrew Purtell 73d69c6157
HBASE-23153 PrimaryRegionCountSkewCostFunction SLB function should implement CostFunction#isNeeded (#714)
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-12 13:14:13 -07:00
Karthik Palanisamy c0a09ccbbd HBASE-23152 Compaction_switch does not work by RegionServer name (#713)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 Signed-off-by: stack <stack@apache.org>
2019-10-12 09:08:20 -07:00
zbq.dean 5848e149f4 HBASE-23056 Block count is 0 when BucketCache using persistent IOEngine and retrieve from file
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-12 14:49:11 +08:00
chenxu14 1aee5f0552 HBASE-23083 Collect Executor status info periodically and report to (#664)
metrics system
Signed-off-by: stack <stack@apache.org>
2019-10-11 12:05:53 -07:00
OlegPt 473816d922 HBASE-23145 Remove out-of-date comments in StoreFlusher.java (#709)
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-10-11 11:23:56 -07:00
Wellington Chevreuil 38b06c7517 Revert "HBASE-23136 PartionedMobFileCompactor bulkloaded files shouldn't get replicated"
This reverts commit fec4c52499.
2019-10-11 14:45:29 +01:00
Viraj Jasani 2f0b3ac91c HBASE-23093 Avoid Optional Anti-Pattern where possible (#673)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-11 15:28:50 +08:00
Baiqiang Zhao 16da123df4 HBASE-23017 Verify the file integrity in persistent IOEngine
Signed-off-by: Anoop Sam John <anoopsamjohn@apacher.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-11 14:38:00 +08:00
Wellington Ramos Chevreuil fec4c52499
HBASE-23136 PartionedMobFileCompactor bulkloaded files shouldn't get replicated
Signed-off-by: stack <stack@apache.org>
2019-10-10 10:37:42 +01:00
Karthik Palanisamy fdac2ddc81 HBASE-23115 Unit change for StoreFileSize and MemstoreSize (table.jsp) (#682)
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2019-10-09 23:30:56 +09:00
haxiaolin 92bf07a98d HBASE-22767 System table RIT STUCK if their RSGroup has no highest version RSes
Signed-off-by: stack <stack@apache.org>
2019-10-08 16:31:22 -07:00
Karthik Palanisamy ff52994076 HBASE-23095 Reuse FileStatus in StoreFileInfo (#674)
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-08 15:47:12 +08:00
Sanjeet Nishad 10180e232e HBASE-23094 Wrong log message in simpleRegionNormaliser while checking if merge is enabled.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-10-07 16:20:51 +02:00
Bharath Vissapragada 06ff478674 HBASE-23116: Fix a load balancer logging nit. (#687)
This commit adds table name to the logging context when
StochasticLoadBalancer is configured "per table". Added some
test coverage with per-table balancer enabled and manually
verified the logs to make sure the table name is formatted
correctly.

Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.com>
2019-10-04 18:07:49 +01:00
Rushabh 5aa8d3a20b HBASE-22874 Define a public API for Canary checking and a non-public tool implementation
Closes #580

* Canary is now an IA.Public interface
* CanaryTool is now the implementation

Signed-off-by: Sean Busbey <busbey@apache.org>
2019-10-03 13:45:48 -07:00
stack 2ebdcbc3b0 Revert " HBASE-23055 Alter hbase:meta (#667)"
Revert because new feedback and requested survey of master usage
figuring table state.

This reverts commit 52176184df.
2019-10-01 09:24:10 -07:00
Michael Stack 52176184df
HBASE-23055 Alter hbase:meta (#667)
Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.

 Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2019-09-30 09:29:51 -07:00
Viraj Jasani 944108c38a HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command (#611)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 17:35:22 +08:00
Duo Zhang ca0d9f3cd2
HBASE-23079 RegionRemoteProcedureBase should override setTimeoutFailure (#672)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-30 11:48:50 +08:00
chenxu14 ce0fbc2af9 HBASE-22965 RS Crash due to DBE reference to an reused ByteBuff (#603)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 09:17:07 +08:00
Guanghao Zhang 1170f28122
HBASE-23078 BaseLoadBalancer should consider region replicas when randomAssignment and roundRobinAssignment (#663)
Signed-off-by: stack <stack@apache.org>
2019-09-29 17:36:07 +08:00
Duo Zhang c1a476c492
HBASE-23087 Remove the deprecated bulkload method in AsyncClusterConnection (#670)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-09-29 09:26:40 +08:00
Ankit Singhal 836368189a HBASE-23054 Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm 2019-09-28 09:29:17 -07:00
Guanghao Zhang ef79b40e6e
HBASE-23035 Retain region to the last RegionServer make the failover slower (addendum) (#652) 2019-09-28 20:25:05 +08:00
Sanjeet Nishad b45c0d0efa HBASE-23038 Provide consistent and clear logging about disabling chores
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-09-27 15:01:15 -05:00
shardul-cr7 3250a8021a HBASE-22012 Prevent DISABLE quota violation policy from disabling violation policy
Space quotas has a feature which intends to avoid enacting a space quota
violation policy when only a subset of the Regions for that Table have
reported their space usage (under the assumption that we cannot make an
informed decision if we do not include all regions in our calculations).

This had the unintended side-effect, when a table is disabled as a part
of a violation policy, of causing the regions for that table to not be
reported which disables the violation policy and enables the table.

Need to make sure that when a table is disabled because of a violation
policy that the code does not automatically move that table out of
violation because region sizes are not being reported (because those
regions are not open).

Closes #572

Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-26 16:15:16 -04:00
Wellington Chevreuil 277c1cc97f Amend HBASE-22380 break circle replication when doing bulkload
Fixing tests broken by this commit: TestBulkLoadHFilesSplitRecovery and TestSecureBulkLoadHFilesSplitRecovery.
2019-09-26 13:35:35 +01:00
BukrosSzabolcs f0dddd1cc2 HBASE-22982: region server suspend/resume and graceful rolling restart actions (#592)
* Add chaos monkey action for suspend/resume region servers
* Add chaos monkey action for graceful rolling restart
* Add these to relevant chaos monkeys

Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-26 10:07:38 +02:00
zbq.dean a8e3d23cca HBASE-22975 Add read and write QPS metrics at server level and table level (#615)
Signed-off-by Reid Chan <reidchan@apache.org>
2019-09-23 12:51:25 +08:00
Pankaj 537c5f0660 HBASE-22930 Set unique name to longCompactions/shortCompactions/split threads (#585) 2019-09-22 12:25:02 +02:00
Guanghao Zhang cb04c6c1a6
HBASE-23035 Retain region to the last RegionServer make the failover slower (#631)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-21 21:11:05 +08:00
Wellington Ramos Chevreuil 3ae353cbf4
HBASE-22380 break circle replication when doing bulkload (#566)
Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-21 14:02:18 +01:00
wuqiong 44cdf5f4ea HBASE-23058 Should be 'Column Family Name' in table.jsp
Signed-off-by: Guanghao Zhang <zhangguanghao1@xiaomi.com>
2019-09-21 19:24:32 +08:00
shardul-cr7 96a94ac3d0 HBASE-22142 Drop table RegionSizes with namespace quota
There was a bug in which we would not drop the RegionSizes
for a table in a namespace, where the namespace had a quota
on it. This allowed a scenario in which recreation of a table
inside of a namespace would unintentionally move into violation
despite the table being empty. Need to make sure the RegionSizes
are dropped on table deletion if there is _any_ quota applying
to that table.

Closes #598

Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-20 13:50:58 -04:00
shardul-cr7 2b3c9b18dd HBASE-22944 Check for hbase:quota table existence in SpaceQuotaRefresherChore
During startup, it's possible that quotas are enabled but the Master has
not yet created the hbase:quotas table.

Closes #559

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-20 13:50:43 -04:00
Duo Zhang e890776fe0 HBASE-22941 Addendum fix NPE in UT 2019-09-20 14:35:56 +08:00
sreenivasulureddy 49718b8b46
HBASE-22649: Encode StoreFile path URLs in the UI to handle scenarios where CF contains special characters (like # etc.)
Signed-off-by: Sakthi<sakthi@apache.org>
2019-09-19 14:14:15 -07:00
Stig Døssing 0ec0a9403f HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… (#361)
* HBase-22027: Split non-MR related parts of TokenUtil off into a ClientTokenUtil, and move ClientTokenUtil to hbase-client

* Replace uses of deprecated TokenUtil methods with ClientTokenUtil methods. Make methods that don't need to be public package-private

* Don't use reflection where not necessary in TestClientTokenUtil

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-09-19 12:42:04 -07:00
Peter Somogyi 1fec3d1dfd
HBASE-23051 Remove unneeded Mockito.mock invocations (#641)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2019-09-19 19:28:58 +02:00
Wellington Chevreuil a85c6b4867 HBASE-23047 ChecksumUtil.validateChecksum logs an INFO message inside a "if(LOG.isTraceEnabled())" block.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-19 15:32:03 +01:00
Guanghao Zhang 343a6a7913
HBASE-23049 TableDescriptors#getAll should return the tables ordering by the name which contain namespace (#639)
Signed-off-by: stack <stack@apache.org>
2019-09-19 21:20:42 +08:00
Peter Somogyi 1655360951 Revert "HBASE-22700 refactor isMetaClearingException (#501)"
This reverts commit cb62f73406.
2019-09-19 13:21:22 +02:00
Guanghao Zhang a0e8723b73
HBASE-23041 Should not show split parent regions in HBCK report's unknown server part (#634)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 10:59:01 +08:00
Guanghao Zhang 20bfb43db6
HBASE-23044 CatalogJanitor#cleanMergeQualifier may clean wrong parent regions (#637)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 08:08:49 +08:00
Guanghao Zhang 25bcc91dab
HBASE-23037 Make the split WAL related log more readable (#632)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-18 21:44:05 +08:00
Sean Busbey ab076b0731 HBASE-23040 give friendly message about unknown server when running unload for RegionMover.
closes #635

Signed-off-by: stack <stack@apache.org>
2019-09-18 06:29:43 -05:00
johnhomsea cb62f73406 HBASE-22700 refactor isMetaClearingException (#501) 2019-09-17 10:02:37 -07:00
Viraj Jasani 0dbae8f0e1 HBASE-23024 Replace initcause with Constructor arg (#627)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-17 13:39:26 +02:00
Caroline Zhou 7da61ff174
HBASE-22804 Provide an API to get list of successful regions and total expected regions in Canary
Signed-off-by: Xu Cang <xucang@apache.org>
2019-09-16 11:15:27 -07:00
Yiran Wu f31301dcf8 HBASE-22939 SpaceQuotas - Bulkload from different hdfs failed when space quotas are turned on. (#553)
Signed-off-by: Sakthi <sakthi@apache.org>
2019-09-14 10:46:29 -07:00