7792 Commits

Author SHA1 Message Date
Michael Stack
ac0a605ee9 HBASE-23243 [pv2] Filter out SUCCESS procedures; on decent-sized cluster, plethora overwhelms problems (#790)
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-11-04 09:17:47 -08:00
Michael Stack
4f39b93a34 HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers' (#791)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-11-04 09:01:25 -08:00
binlijin
c8c8eef0d4 HBASE-23241 TestExecutorService sometimes fail (#782)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-11-04 17:19:13 +08:00
stack
df04fbc6db HBASE-23244 NPEs running Canary (#784)
Addendum to fix findbugs complaint.
2019-11-02 09:34:53 -07:00
Michael Stack
74e17be4e2
HBASE-23244 NPEs running Canary (#784)
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
2019-11-01 21:40:23 -07:00
Wellington Chevreuil
18f1aacae0 HBASE-23238 Additional test and checks for null references on ScannerCallableWithReplicas (#780)
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 577db5d7e50c56b4773c9ce92b807aae80bf5706)
2019-11-01 10:32:12 +00:00
stack
97644e20bb Revert "Revert "HBASE-23191 EOFE log spam (#733)""
Reapply. Reverted wrong commit.

This reverts commit 87b4cfabdaeac6e1b04ad724b9db1dcd03c00633.
2019-10-31 10:35:39 -07:00
stack
87b4cfabda Revert "HBASE-23191 EOFE log spam (#733)"
This reverts commit f0c38e8ae67a9be72f635239eac32771215c3dc0.
2019-10-31 08:11:21 -07:00
chenxu14
26663585de HBASE-23184 The HeapAllocation in WebUI is not accurate (#730)
Signed-off-by: stack <stack@apache.org>
2019-10-31 17:57:52 +08:00
Michael Stack
471538ca9e HBASE-23221 Polish the WAL interface after HBASE-23181 (#774)
Removes the closeRegion flag added by HBASE-23181 and instead
relies on reading meta WALEdit content. Modified how qualifier is
written when the meta WALEdit is for a RegionEventDescriptor
so the 'type' is added to the qualifer so can figure type
w/o having to deserialize protobuf value content: e.g.
HBASE::REGION_EVENT::REGION_CLOSE

Added doc on WALEdit and tried to formalize the 'meta' WALEdit
type and how it works. Needs complete redo in part as suggested
by HBASE-8457. Meantime, some doc and cleanup.

Also changed the LogRoller constructor to remove redundant param.
Because of constructor change, need to change also
TestFailedAppendAndSync, TestWALLockup, TestAsyncFSWAL &
WALPerformanceEvaluation.java

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-10-30 22:10:51 -07:00
binlijin
9ab0489eab HBASE-23231 ReplicationSource do not update metrics after refresh (#778)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-31 09:39:49 +08:00
Ankit Singhal
19adfee3d4 HBASE-23175 Yarn unable to acquire delegation token for HBase Spark jobs 2019-10-30 14:50:32 -07:00
binlijin
f02757d38a HBASE-22739 ArrayIndexOutOfBoundsException when balance (#729)
Signed-off-by: stack <stack@apache.org>
2019-10-30 13:33:41 -07:00
Michael Stack
77de8e21e4 HBASE-23192 CatalogJanitor consistencyCheck does not log problematic row on exception (#734)
Adds logging of row and complaint if consistency check fails during CJ
checking. Adds a few more null checks. Does edit on the 'HBCK Report'
top line.

Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-30 13:11:11 -07:00
Karthik Palanisamy
f0c38e8ae6 HBASE-23191 EOFE log spam (#733)
Convert log message added for 2.2.0 from INFO to DEBUG.

Signed-off-by: stack <stack@apache.org>
2019-10-30 12:50:11 -07:00
Ankit Singhal
d5f6329fac HBASE-20827 Use backoff on CallQueueTooBigException when reporting region state transition
Signed-off-by: Josh Elser <elserj@apache.org>
2019-10-30 11:42:59 -04:00
Wellington Chevreuil
d2a027d8be HBASE-23136 PartionedMobFileCompactor bulkloaded files shouldn't get replicated (addressing buklload replication related issue raised in HBASE-22380)
Signed-off-by: Josh Elser <elserj@apache.org>
(cherry picked from commit 4d414020bb3bfd7f214d2a599426be700df772b2, then resolved conflicts)
2019-10-29 11:14:32 +00:00
binlijin
cab9ccc26d HBASE-23187 Update parent region state to SPLIT in meta (#732)
* HBASE-23187 Update parent region state to SPLIT in meta
2019-10-29 12:42:28 +08:00
Karthik Palanisamy
b18681cb3b HBASE-23199 Error populating Table-Attribute fields (#741)
Signed-off-by: GuangxuCheng  <guangxucheng@gmail.com>
2019-10-29 11:44:01 +08:00
Sean Busbey
ff2b290268 HBASE-23222 MOB compaction supportability improvements
* better logging on MOB compaction process
* HFileCleanerDelegate to optionally halt removal of mob hfiles
* use archiving when removing committed mob file after bulkload ref failure

closes #763

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2019-10-28 09:47:55 -05:00
Duo Zhang
3dba799837 HBASE-23181 Blocked WAL archive: "LogRoller: Failed to schedule flush of XXXX, because it is not online on us" (#753)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: stack <stack@apache.org>
2019-10-26 20:47:29 +08:00
Viraj Jasani
33e8156ebc HBASE-22460 : Reopen regions with very high Store Ref Counts (#750)
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
2019-10-24 21:20:24 +05:30
Pierre Zemb
d9c36e0dcf HBASE-23073 Add an optional costFunction to balance regions according to a capacity rule (#677)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit 42d535a57a75b58f585b48df9af9c966e6c7e46a)
2019-10-24 08:54:10 +01:00
Andrew Purtell
75dad6cd14
HBASE-23207 Log a region open journal (#751)
Signed-off-by: Abhishek Singh Chouhan <achouhan@apache.org>
2019-10-23 17:18:33 -07:00
Ankit Singhal
ddd61aa825 HBASE-15519 Add per-user metrics with lossy counting
Introducing property hbase.regionserver.user.metrics.enabled(Default:true)
to disable user metrics in case it accounts for any performance issues

Close #661

Signed-off-by: Josh Elser <elserj@apache.org>
2019-10-23 11:54:16 -04:00
Caroline Zhou
ee6897b2ad HBASE-23172 HBase Canary region success count metrics reflect column family successes, not region successes 2019-10-21 09:03:47 -07:00
meiyi
11bf11499f HBASE-23042 Parameters are incorrect in procedures jsp (#728) 2019-10-21 10:15:37 +08:00
meiyi
d841245115 HBASE-23170 Admin#getRegionServers use ClusterMetrics.Option.SERVERS_NAME (#721) 2019-10-18 10:09:42 +08:00
chenxu14
b0b7e5f5b8 HBASE-23107 Avoid temp byte array creation when doing cacheDataOnWrite (#678)
Signed-off-by: huzheng <openinx@gmail.com>
Signed-off-by: stack <stack@apache.org>
2019-10-17 09:47:14 +08:00
stack
cc76318f76 HBASE-23177 If fail to open reference because FNFE, make it plain it is a Reference Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Sean Busbey <busbey@apache.org> Signed-off-by: Viraj Jasani <virajjasani007@gmail.com> 2019-10-16 12:34:06 -07:00
binlijin
b909363038 HBASE-22370 ByteBuf LEAK ERROR (#720)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-16 20:46:50 +08:00
Guangxu Cheng
f620e1dae4 HBASE-20626 Change the value of "Requests Per Second" on WEBUI 2019-10-16 15:25:19 +08:00
Guanghao Zhang
18ff188d9d HBASE-23155 May NPE when concurrent AsyncNonMetaRegionLocator#updateCachedLocationOnError (#718) 2019-10-15 11:34:12 +08:00
Duo Zhang
44afe8cfde HBASE-23163 Refactor HStore.getStorefilesSize related methods (#719)
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-10-14 20:49:35 +08:00
李小保
699fad39a4 HBASE-21540 when set property "hbase.systemtables.compacting.memstore.type" to "basic" or "eager" will cause an exception
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-14 08:35:08 +08:00
Lijin Bin
6f6d331f9f HBASE-23159 HStore#getStorefilesSize may throw NPE 2019-10-12 19:45:32 -07:00
Andrew Purtell
56e0652aff
HBASE-23153 PrimaryRegionCountSkewCostFunction SLB function should implement CostFunction#isNeeded (#714)
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-12 13:15:22 -07:00
Karthik Palanisamy
bebf251179 HBASE-23152 Compaction_switch does not work by RegionServer name (#713)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 Signed-off-by: stack <stack@apache.org>
2019-10-12 09:09:44 -07:00
zbq.dean
595bcda9c3 HBASE-23056 Block count is 0 when BucketCache using persistent IOEngine and retrieve from file
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-12 14:50:20 +08:00
Viraj Jasani
44a7e8b3f1 HBASE-23093 Avoid Optional Anti-Pattern where possible (#673)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-11 17:47:05 +08:00
Baiqiang Zhao
273c0ba69f HBASE-23017 Verify the file integrity in persistent IOEngine
Signed-off-by: Anoop Sam John <anoopsamjohn@apacher.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-11 14:39:06 +08:00
Karthik Palanisamy
3025c4250c HBASE-23115 Unit change for StoreFileSize and MemstoreSize (table.jsp) (#682)
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2019-10-09 23:36:30 +09:00
haxiaolin
fff0f33c5a HBASE-22767 System table RIT STUCK if their RSGroup has no highest version RSes
Signed-off-by: stack <stack@apache.org>
2019-10-08 16:32:07 -07:00
Karthik Palanisamy
54083a7626 HBASE-23095 Reuse FileStatus in StoreFileInfo (#674)
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-08 15:48:17 +08:00
Viraj Jasani
15ec3c458c HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command (#611)
Signed-off-by: huzheng <openinx@gmail.com>
2019-10-08 09:25:57 +08:00
Sanjeet Nishad
51184980a9 HBASE-23094 Wrong log message in simpleRegionNormaliser while checking if merge is enabled.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-10-07 16:21:39 +02:00
Bharath Vissapragada
12852d30a3 HBASE-23116: Fix a load balancer logging nit. (#687)
This commit adds table name to the logging context when
StochasticLoadBalancer is configured "per table". Added some
test coverage with per-table balancer enabled and manually
verified the logs to make sure the table name is formatted
correctly.

Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.com>
(cherry picked from commit 06ff478674867204282899987cc8d3e5bef8bdac)
2019-10-04 18:16:07 +01:00
Rushabh
b5a396749d HBASE-22874 Define a public API for Canary checking and a non-public tool implementation
Closes #580

* Canary is now an IA.Public interface
* CanaryTool is now the implementation

Signed-off-by: Sean Busbey <busbey@apache.org>
2019-10-03 14:34:15 -07:00
stack
61bc57f525 Revert " HBASE-23055 Alter hbase:meta (#655)"
"Too radical for branch-2"

This reverts commit ff217d4269f7b959c24a11c18c97a4f43f5b98e7.
2019-10-01 08:34:09 -07:00
Michael Stack
ff217d4269
HBASE-23055 Alter hbase:meta (#655)
Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.


   Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2019-09-30 09:27:41 -07:00