16554 Commits

Author SHA1 Message Date
Pierre Zemb
d9c36e0dcf HBASE-23073 Add an optional costFunction to balance regions according to a capacity rule (#677)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 42d535a57a75b58f585b48df9af9c966e6c7e46a)
2019-10-24 08:54:10 +01:00
Duo Zhang
f19b8096f1 HBASE-23193 ConnectionImplementation.isTableAvailable can not deal with meta table on branch-2.x 2019-10-24 10:51:50 +08:00
Andrew Purtell
75dad6cd14
HBASE-23207 Log a region open journal (#751)
Signed-off-by: Abhishek Singh Chouhan <achouhan@apache.org>
2019-10-23 17:18:33 -07:00
Ankit Singhal
ddd61aa825 HBASE-15519 Add per-user metrics with lossy counting
Introducing property hbase.regionserver.user.metrics.enabled(Default:true)
to disable user metrics in case it accounts for any performance issues

Close #661

Signed-off-by: Josh Elser <elserj@apache.org>
2019-10-23 11:54:16 -04:00
Caroline Zhou
ee6897b2ad HBASE-23172 HBase Canary region success count metrics reflect column family successes, not region successes 2019-10-21 09:03:47 -07:00
meiyi
11bf11499f HBASE-23042 Parameters are incorrect in procedures jsp (#728) 2019-10-21 10:15:37 +08:00
meiyi
d841245115 HBASE-23170 Admin#getRegionServers use ClusterMetrics.Option.SERVERS_NAME (#721) 2019-10-18 10:09:42 +08:00
chenxu14
b0b7e5f5b8 HBASE-23107 Avoid temp byte array creation when doing cacheDataOnWrite (#678)
Signed-off-by: huzheng <openinx@gmail.com>
Signed-off-by: stack <stack@apache.org>
2019-10-17 09:47:14 +08:00
stack
cc76318f76 HBASE-23177 If fail to open reference because FNFE, make it plain it is a Reference Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Sean Busbey <busbey@apache.org> Signed-off-by: Viraj Jasani <virajjasani007@gmail.com> 2019-10-16 12:34:06 -07:00
binlijin
b909363038 HBASE-22370 ByteBuf LEAK ERROR (#720)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-16 20:46:50 +08:00
Guangxu Cheng
f620e1dae4 HBASE-20626 Change the value of "Requests Per Second" on WEBUI 2019-10-16 15:25:19 +08:00
Guanghao Zhang
18ff188d9d HBASE-23155 May NPE when concurrent AsyncNonMetaRegionLocator#updateCachedLocationOnError (#718) 2019-10-15 11:34:12 +08:00
Duo Zhang
44afe8cfde HBASE-23163 Refactor HStore.getStorefilesSize related methods (#719)
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-10-14 20:49:35 +08:00
李小保
699fad39a4 HBASE-21540 when set property "hbase.systemtables.compacting.memstore.type" to "basic" or "eager" will cause an exception
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-14 08:35:08 +08:00
Lijin Bin
6f6d331f9f HBASE-23159 HStore#getStorefilesSize may throw NPE 2019-10-12 19:45:32 -07:00
Andrew Purtell
56e0652aff
HBASE-23153 PrimaryRegionCountSkewCostFunction SLB function should implement CostFunction#isNeeded (#714)
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-12 13:15:22 -07:00
Karthik Palanisamy
7171da09ea HBASE-23154 list_deadservers return incorrect no of rows (#717)
Signed-off-by: stack <stack@apache.org>
2019-10-12 09:15:05 -07:00
Karthik Palanisamy
bebf251179 HBASE-23152 Compaction_switch does not work by RegionServer name (#713)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 Signed-off-by: stack <stack@apache.org>
2019-10-12 09:09:44 -07:00
zbq.dean
595bcda9c3 HBASE-23056 Block count is 0 when BucketCache using persistent IOEngine and retrieve from file
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-12 14:50:20 +08:00
Michael Stack
d8ad2a0bb5 HBASE-23129 Move core to use hbase-thirdparty-3.1.1 (#695)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-10-11 14:32:19 +02:00
Viraj Jasani
44a7e8b3f1 HBASE-23093 Avoid Optional Anti-Pattern where possible (#673)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-11 17:47:05 +08:00
Baiqiang Zhao
273c0ba69f HBASE-23017 Verify the file integrity in persistent IOEngine
Signed-off-by: Anoop Sam John <anoopsamjohn@apacher.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-11 14:39:06 +08:00
Peter Somogyi
46952d9b17 HBASE-23053 Disable concurrent nightly builds (#654)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-10-10 20:12:34 +02:00
Peter Somogyi
14d58e2e64 HBASE-23114 Use archiveArtifacts in Jenkinsfiles (#681)
Signed-off-by: Michael Stack <stack@apache.org>
2019-10-10 18:58:06 +02:00
Karthik Palanisamy
3025c4250c HBASE-23115 Unit change for StoreFileSize and MemstoreSize (table.jsp) (#682)
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2019-10-09 23:36:30 +09:00
Karthik Palanisamy
928bd78ece HBASE-23138 Drop_all table by regex fail (#704)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit d237106ae698d3632b9f26f1ea2f1b4fa41811c3)
2019-10-09 13:57:32 +01:00
Karthik Palanisamy
294db538d0 HBASE-23140 Remove unknown table error (#706)
Signed-off-by: huzheng <openinx@gmail.com>
2019-10-09 16:02:48 +08:00
Andrew Purtell
36c4239f37
HBASE-23139 MapReduce jobs lauched from convenience distribution are nonfunctional (#705)
Add hbase-shaded-gson to base job dependencies.

Conflicts:
	hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
2019-10-08 19:13:48 -07:00
langdamao
f40c745499 HBASE-22887 Fix HFileOutputFormat2 writer roll (#554)
Signed-off-by: langdamao <lang--lang--lang@163.com>
2019-10-08 16:47:49 -07:00
haxiaolin
fff0f33c5a HBASE-22767 System table RIT STUCK if their RSGroup has no highest version RSes
Signed-off-by: stack <stack@apache.org>
2019-10-08 16:32:07 -07:00
Karthik Palanisamy
3427999ff7 HBASE-23134 Enable_all and Disable_all table by Regex fail from Shell (#698)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-08 18:31:21 +08:00
Karthik Palanisamy
54083a7626 HBASE-23095 Reuse FileStatus in StoreFileInfo (#674)
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-08 15:48:17 +08:00
Viraj Jasani
15ec3c458c HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command (#611)
Signed-off-by: huzheng <openinx@gmail.com>
2019-10-08 09:25:57 +08:00
Sanjeet Nishad
51184980a9 HBASE-23094 Wrong log message in simpleRegionNormaliser while checking if merge is enabled.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-10-07 16:21:39 +02:00
Duo Zhang
45cbe8b038 HBASE-23125 TestRSGroupsAdmin2 is flaky 2019-10-07 18:46:57 +08:00
Bharath Vissapragada
12852d30a3 HBASE-23116: Fix a load balancer logging nit. (#687)
This commit adds table name to the logging context when
StochasticLoadBalancer is configured "per table". Added some
test coverage with per-table balancer enabled and manually
verified the logs to make sure the table name is formatted
correctly.

Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.com>
(cherry picked from commit 06ff478674867204282899987cc8d3e5bef8bdac)
2019-10-04 18:16:07 +01:00
Istvan Toth
e3078aa5aa HBASE-23119 ArrayIndexOutOfBoundsException in PrivateCellUtil#qualifierStartsWith (#688)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-10-04 15:14:30 +02:00
Rushabh
b5a396749d HBASE-22874 Define a public API for Canary checking and a non-public tool implementation
Closes #580

* Canary is now an IA.Public interface
* CanaryTool is now the implementation

Signed-off-by: Sean Busbey <busbey@apache.org>
2019-10-03 14:34:15 -07:00
stack
61bc57f525 Revert " HBASE-23055 Alter hbase:meta (#655)"
"Too radical for branch-2"

This reverts commit ff217d4269f7b959c24a11c18c97a4f43f5b98e7.
2019-10-01 08:34:09 -07:00
Michael Stack
ff217d4269
HBASE-23055 Alter hbase:meta (#655)
Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.


   Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2019-09-30 09:27:41 -07:00
Duo Zhang
1a2d47b6c7 HBASE-23079 Addendum fix broken UT 2019-09-30 15:46:46 +08:00
Duo Zhang
1e051a7867 HBASE-23079 RegionRemoteProcedureBase should override setTimeoutFailure (#672)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-30 11:57:19 +08:00
chenxu14
be932487e8 HBASE-22965 RS Crash due to DBE reference to an reused ByteBuff (#603)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 09:20:35 +08:00
Guanghao Zhang
9f703fc3b2 HBASE-23078 BaseLoadBalancer should consider region replicas when randomAssignment and roundRobinAssignment (#663)
Signed-off-by: stack <stack@apache.org>
2019-09-29 18:44:41 +08:00
Ankit Singhal
9f7c73caa2 HBASE-23054 Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm 2019-09-28 09:30:25 -07:00
Guanghao Zhang
ebaae125cc HBASE-23035 Retain region to the last RegionServer make the failover slower (addendum) (#652) 2019-09-29 04:32:07 +08:00
Sanjeet Nishad
6def152819 HBASE-23038 Provide consistent and clear logging about disabling chores
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit b45c0d0efa261509ac101748587051ba1c9184bb)
2019-09-27 20:13:18 -05:00
shardul-cr7
43b4ad9e8a HBASE-22012 Prevent DISABLE quota violation policy from disabling violation policy
Space quotas has a feature which intends to avoid enacting a space quota
violation policy when only a subset of the Regions for that Table have
reported their space usage (under the assumption that we cannot make an
informed decision if we do not include all regions in our calculations).

This had the unintended side-effect, when a table is disabled as a part
of a violation policy, of causing the regions for that table to not be
reported which disables the violation policy and enables the table.

Need to make sure that when a table is disabled because of a violation
policy that the code does not automatically move that table out of
violation because region sizes are not being reported (because those
regions are not open).

Closes #572

Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-26 16:19:28 -04:00
BukrosSzabolcs
cd9367512a HBASE-22982: region server suspend/resume and graceful rolling restart actions (#592)
* Add chaos monkey action for suspend/resume region servers
* Add chaos monkey action for graceful rolling restart
* Add these to relevant chaos monkeys

Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-26 11:56:17 +02:00
SteNicholas
f99a9a8b64 HBASE-23075 Upgrade jackson version (#660)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-26 10:24:05 +08:00