Commit Graph

16401 Commits

Author SHA1 Message Date
Ankit Singhal ddd61aa825 HBASE-15519 Add per-user metrics with lossy counting
Introducing property hbase.regionserver.user.metrics.enabled(Default:true)
to disable user metrics in case it accounts for any performance issues

Close #661

Signed-off-by: Josh Elser <elserj@apache.org>
2019-10-23 11:54:16 -04:00
Caroline Zhou ee6897b2ad HBASE-23172 HBase Canary region success count metrics reflect column family successes, not region successes 2019-10-21 09:03:47 -07:00
meiyi 11bf11499f HBASE-23042 Parameters are incorrect in procedures jsp (#728) 2019-10-21 10:15:37 +08:00
meiyi d841245115 HBASE-23170 Admin#getRegionServers use ClusterMetrics.Option.SERVERS_NAME (#721) 2019-10-18 10:09:42 +08:00
chenxu14 b0b7e5f5b8 HBASE-23107 Avoid temp byte array creation when doing cacheDataOnWrite (#678)
Signed-off-by: huzheng <openinx@gmail.com>
Signed-off-by: stack <stack@apache.org>
2019-10-17 09:47:14 +08:00
stack cc76318f76 HBASE-23177 If fail to open reference because FNFE, make it plain it is a Reference Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Sean Busbey <busbey@apache.org> Signed-off-by: Viraj Jasani <virajjasani007@gmail.com> 2019-10-16 12:34:06 -07:00
binlijin b909363038 HBASE-22370 ByteBuf LEAK ERROR (#720)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-16 20:46:50 +08:00
Guangxu Cheng f620e1dae4 HBASE-20626 Change the value of "Requests Per Second" on WEBUI 2019-10-16 15:25:19 +08:00
Guanghao Zhang 18ff188d9d HBASE-23155 May NPE when concurrent AsyncNonMetaRegionLocator#updateCachedLocationOnError (#718) 2019-10-15 11:34:12 +08:00
Duo Zhang 44afe8cfde HBASE-23163 Refactor HStore.getStorefilesSize related methods (#719)
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-10-14 20:49:35 +08:00
李小保 699fad39a4 HBASE-21540 when set property "hbase.systemtables.compacting.memstore.type" to "basic" or "eager" will cause an exception
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-14 08:35:08 +08:00
Lijin Bin 6f6d331f9f HBASE-23159 HStore#getStorefilesSize may throw NPE 2019-10-12 19:45:32 -07:00
Andrew Purtell 56e0652aff
HBASE-23153 PrimaryRegionCountSkewCostFunction SLB function should implement CostFunction#isNeeded (#714)
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-12 13:15:22 -07:00
Karthik Palanisamy 7171da09ea HBASE-23154 list_deadservers return incorrect no of rows (#717)
Signed-off-by: stack <stack@apache.org>
2019-10-12 09:15:05 -07:00
Karthik Palanisamy bebf251179 HBASE-23152 Compaction_switch does not work by RegionServer name (#713)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 Signed-off-by: stack <stack@apache.org>
2019-10-12 09:09:44 -07:00
zbq.dean 595bcda9c3 HBASE-23056 Block count is 0 when BucketCache using persistent IOEngine and retrieve from file
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-12 14:50:20 +08:00
Michael Stack d8ad2a0bb5 HBASE-23129 Move core to use hbase-thirdparty-3.1.1 (#695)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-10-11 14:32:19 +02:00
Viraj Jasani 44a7e8b3f1 HBASE-23093 Avoid Optional Anti-Pattern where possible (#673)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-11 17:47:05 +08:00
Baiqiang Zhao 273c0ba69f HBASE-23017 Verify the file integrity in persistent IOEngine
Signed-off-by: Anoop Sam John <anoopsamjohn@apacher.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-11 14:39:06 +08:00
Peter Somogyi 46952d9b17 HBASE-23053 Disable concurrent nightly builds (#654)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-10-10 20:12:34 +02:00
Peter Somogyi 14d58e2e64 HBASE-23114 Use archiveArtifacts in Jenkinsfiles (#681)
Signed-off-by: Michael Stack <stack@apache.org>
2019-10-10 18:58:06 +02:00
Karthik Palanisamy 3025c4250c HBASE-23115 Unit change for StoreFileSize and MemstoreSize (table.jsp) (#682)
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2019-10-09 23:36:30 +09:00
Karthik Palanisamy 928bd78ece HBASE-23138 Drop_all table by regex fail (#704)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit d237106ae6)
2019-10-09 13:57:32 +01:00
Karthik Palanisamy 294db538d0 HBASE-23140 Remove unknown table error (#706)
Signed-off-by: huzheng <openinx@gmail.com>
2019-10-09 16:02:48 +08:00
Andrew Purtell 36c4239f37
HBASE-23139 MapReduce jobs lauched from convenience distribution are nonfunctional (#705)
Add hbase-shaded-gson to base job dependencies.

Conflicts:
	hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
2019-10-08 19:13:48 -07:00
langdamao f40c745499 HBASE-22887 Fix HFileOutputFormat2 writer roll (#554)
Signed-off-by: langdamao <lang--lang--lang@163.com>
2019-10-08 16:47:49 -07:00
haxiaolin fff0f33c5a HBASE-22767 System table RIT STUCK if their RSGroup has no highest version RSes
Signed-off-by: stack <stack@apache.org>
2019-10-08 16:32:07 -07:00
Karthik Palanisamy 3427999ff7 HBASE-23134 Enable_all and Disable_all table by Regex fail from Shell (#698)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-08 18:31:21 +08:00
Karthik Palanisamy 54083a7626 HBASE-23095 Reuse FileStatus in StoreFileInfo (#674)
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-08 15:48:17 +08:00
Viraj Jasani 15ec3c458c HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command (#611)
Signed-off-by: huzheng <openinx@gmail.com>
2019-10-08 09:25:57 +08:00
Sanjeet Nishad 51184980a9 HBASE-23094 Wrong log message in simpleRegionNormaliser while checking if merge is enabled.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-10-07 16:21:39 +02:00
Duo Zhang 45cbe8b038 HBASE-23125 TestRSGroupsAdmin2 is flaky 2019-10-07 18:46:57 +08:00
Bharath Vissapragada 12852d30a3 HBASE-23116: Fix a load balancer logging nit. (#687)
This commit adds table name to the logging context when
StochasticLoadBalancer is configured "per table". Added some
test coverage with per-table balancer enabled and manually
verified the logs to make sure the table name is formatted
correctly.

Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.com>
(cherry picked from commit 06ff478674)
2019-10-04 18:16:07 +01:00
Istvan Toth e3078aa5aa HBASE-23119 ArrayIndexOutOfBoundsException in PrivateCellUtil#qualifierStartsWith (#688)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-10-04 15:14:30 +02:00
Rushabh b5a396749d HBASE-22874 Define a public API for Canary checking and a non-public tool implementation
Closes #580

* Canary is now an IA.Public interface
* CanaryTool is now the implementation

Signed-off-by: Sean Busbey <busbey@apache.org>
2019-10-03 14:34:15 -07:00
stack 61bc57f525 Revert " HBASE-23055 Alter hbase:meta (#655)"
"Too radical for branch-2"

This reverts commit ff217d4269.
2019-10-01 08:34:09 -07:00
Michael Stack ff217d4269
HBASE-23055 Alter hbase:meta (#655)
Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.


   Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2019-09-30 09:27:41 -07:00
Duo Zhang 1a2d47b6c7 HBASE-23079 Addendum fix broken UT 2019-09-30 15:46:46 +08:00
Duo Zhang 1e051a7867 HBASE-23079 RegionRemoteProcedureBase should override setTimeoutFailure (#672)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-30 11:57:19 +08:00
chenxu14 be932487e8 HBASE-22965 RS Crash due to DBE reference to an reused ByteBuff (#603)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 09:20:35 +08:00
Guanghao Zhang 9f703fc3b2 HBASE-23078 BaseLoadBalancer should consider region replicas when randomAssignment and roundRobinAssignment (#663)
Signed-off-by: stack <stack@apache.org>
2019-09-29 18:44:41 +08:00
Ankit Singhal 9f7c73caa2 HBASE-23054 Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm 2019-09-28 09:30:25 -07:00
Guanghao Zhang ebaae125cc HBASE-23035 Retain region to the last RegionServer make the failover slower (addendum) (#652) 2019-09-29 04:32:07 +08:00
Sanjeet Nishad 6def152819 HBASE-23038 Provide consistent and clear logging about disabling chores
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit b45c0d0efa)
2019-09-27 20:13:18 -05:00
shardul-cr7 43b4ad9e8a HBASE-22012 Prevent DISABLE quota violation policy from disabling violation policy
Space quotas has a feature which intends to avoid enacting a space quota
violation policy when only a subset of the Regions for that Table have
reported their space usage (under the assumption that we cannot make an
informed decision if we do not include all regions in our calculations).

This had the unintended side-effect, when a table is disabled as a part
of a violation policy, of causing the regions for that table to not be
reported which disables the violation policy and enables the table.

Need to make sure that when a table is disabled because of a violation
policy that the code does not automatically move that table out of
violation because region sizes are not being reported (because those
regions are not open).

Closes #572

Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-26 16:19:28 -04:00
BukrosSzabolcs cd9367512a HBASE-22982: region server suspend/resume and graceful rolling restart actions (#592)
* Add chaos monkey action for suspend/resume region servers
* Add chaos monkey action for graceful rolling restart
* Add these to relevant chaos monkeys

Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-26 11:56:17 +02:00
SteNicholas f99a9a8b64 HBASE-23075 Upgrade jackson version (#660)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-26 10:24:05 +08:00
Balazs Meszaros c5a5bf7f48 HBASE-23032 Upgrade to Curator 4.2.0
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-23 12:00:33 +02:00
zbq.dean 0c5b6df52e HBASE-22975 Add read and write QPS metrics at server level and table level
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-09-23 15:03:12 +08:00
Pankaj 6fe7063ffc HBASE-22930 Set unique name to longCompactions/shortCompactions/split threads (#585) 2019-09-22 12:26:56 +02:00