7549 Commits

Author SHA1 Message Date
Karthik Palanisamy
54083a7626 HBASE-23095 Reuse FileStatus in StoreFileInfo (#674)
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-08 15:48:17 +08:00
Viraj Jasani
15ec3c458c HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command (#611)
Signed-off-by: huzheng <openinx@gmail.com>
2019-10-08 09:25:57 +08:00
Sanjeet Nishad
51184980a9 HBASE-23094 Wrong log message in simpleRegionNormaliser while checking if merge is enabled.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-10-07 16:21:39 +02:00
Bharath Vissapragada
12852d30a3 HBASE-23116: Fix a load balancer logging nit. (#687)
This commit adds table name to the logging context when
StochasticLoadBalancer is configured "per table". Added some
test coverage with per-table balancer enabled and manually
verified the logs to make sure the table name is formatted
correctly.

Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.com>
(cherry picked from commit 06ff478674867204282899987cc8d3e5bef8bdac)
2019-10-04 18:16:07 +01:00
Rushabh
b5a396749d HBASE-22874 Define a public API for Canary checking and a non-public tool implementation
Closes #580

* Canary is now an IA.Public interface
* CanaryTool is now the implementation

Signed-off-by: Sean Busbey <busbey@apache.org>
2019-10-03 14:34:15 -07:00
stack
61bc57f525 Revert " HBASE-23055 Alter hbase:meta (#655)"
"Too radical for branch-2"

This reverts commit ff217d4269f7b959c24a11c18c97a4f43f5b98e7.
2019-10-01 08:34:09 -07:00
Michael Stack
ff217d4269
HBASE-23055 Alter hbase:meta (#655)
Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.


   Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2019-09-30 09:27:41 -07:00
Duo Zhang
1a2d47b6c7 HBASE-23079 Addendum fix broken UT 2019-09-30 15:46:46 +08:00
Duo Zhang
1e051a7867 HBASE-23079 RegionRemoteProcedureBase should override setTimeoutFailure (#672)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-30 11:57:19 +08:00
chenxu14
be932487e8 HBASE-22965 RS Crash due to DBE reference to an reused ByteBuff (#603)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 09:20:35 +08:00
Guanghao Zhang
9f703fc3b2 HBASE-23078 BaseLoadBalancer should consider region replicas when randomAssignment and roundRobinAssignment (#663)
Signed-off-by: stack <stack@apache.org>
2019-09-29 18:44:41 +08:00
Ankit Singhal
9f7c73caa2 HBASE-23054 Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm 2019-09-28 09:30:25 -07:00
Guanghao Zhang
ebaae125cc HBASE-23035 Retain region to the last RegionServer make the failover slower (addendum) (#652) 2019-09-29 04:32:07 +08:00
Sanjeet Nishad
6def152819 HBASE-23038 Provide consistent and clear logging about disabling chores
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit b45c0d0efa261509ac101748587051ba1c9184bb)
2019-09-27 20:13:18 -05:00
shardul-cr7
43b4ad9e8a HBASE-22012 Prevent DISABLE quota violation policy from disabling violation policy
Space quotas has a feature which intends to avoid enacting a space quota
violation policy when only a subset of the Regions for that Table have
reported their space usage (under the assumption that we cannot make an
informed decision if we do not include all regions in our calculations).

This had the unintended side-effect, when a table is disabled as a part
of a violation policy, of causing the regions for that table to not be
reported which disables the violation policy and enables the table.

Need to make sure that when a table is disabled because of a violation
policy that the code does not automatically move that table out of
violation because region sizes are not being reported (because those
regions are not open).

Closes #572

Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-26 16:19:28 -04:00
BukrosSzabolcs
cd9367512a HBASE-22982: region server suspend/resume and graceful rolling restart actions (#592)
* Add chaos monkey action for suspend/resume region servers
* Add chaos monkey action for graceful rolling restart
* Add these to relevant chaos monkeys

Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-26 11:56:17 +02:00
zbq.dean
0c5b6df52e HBASE-22975 Add read and write QPS metrics at server level and table level
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-09-23 15:03:12 +08:00
Pankaj
6fe7063ffc HBASE-22930 Set unique name to longCompactions/shortCompactions/split threads (#585) 2019-09-22 12:26:56 +02:00
Wellington Ramos Chevreuil
38c8bd3731
HBASE-22380 break circle replication when doing bulkload (#494)
Signed-off-by:  stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Norbert Kalmar <nkalmar@cloudera.com>
2019-09-21 14:39:06 +01:00
Guanghao Zhang
adc0197191 HBASE-23035 Retain region to the last RegionServer make the failover slower (#631)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-21 21:13:10 +08:00
wuqiong
7195ab6a15 HBASE-23058 Should be 'Column Family Name' in table.jsp
Signed-off-by: Guanghao Zhang <zhangguanghao1@xiaomi.com>
2019-09-21 19:27:15 +08:00
Stig Døssing
047bbfd4c2 HBase-22027: Split non-MR related parts of TokenUtil off into a ClientTokenUtil, and move ClientTokenUtil to hbase-client (#649) 2019-09-20 17:38:41 -07:00
shardul-cr7
f1d3b54422 HBASE-22142 Drop table RegionSizes with namespace quota
There was a bug in which we would not drop the RegionSizes
for a table in a namespace, where the namespace had a quota
on it. This allowed a scenario in which recreation of a table
inside of a namespace would unintentionally move into violation
despite the table being empty. Need to make sure the RegionSizes
are dropped on table deletion if there is _any_ quota applying
to that table.

Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-20 12:09:45 -04:00
shardul-cr7
ffbf8503ea HBASE-22944 Check for hbase:quota table existence in SpaceQuotaRefresherChore
During startup, it's possible that quotas are enabled but the Master has
not yet created the hbase:quotas table.

Closes #559

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2019-09-20 12:07:32 -04:00
sreenivasulureddy
63593d6cce HBASE-22649: Encode StoreFile path URLs in the UI to handle scenarios where CF contains special characters (like # etc.)
Signed-off-by: Sakthi<sakthi@apache.org>
(cherry picked from commit 49718b8b46cd9e06aeef3f74647b32c6df99f7ae)
2019-09-20 00:59:08 -07:00
Duo Zhang
5b75bef36e HBASE-22941 Addendum fix NPE in UT 2019-09-20 14:37:01 +08:00
Duo Zhang
0ed11fe452 HBASE-23051 Addendum add missing import 2019-09-20 11:35:34 +08:00
Istvan Toth
908a4d8f2b HBASE-22941 merge operation returns parent regions in random order (#556)
* HBASE-22941 merge operation returns parent regions in random order

store and return the merge parent regions in ascending order

remove left over check for exactly two merged regions

add unit test

* use SortedMap type to emphasise that the Map is sorted.

* use regionCount consistently and checkstyle fixes

* Delete tests that expect multiregion merges to fail.

Signed-off-by: stack <stack@apache.org>
2019-09-20 11:15:21 +08:00
Peter Somogyi
294487ce20 HBASE-23051 Remove unneeded Mockito.mock invocations (#641)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2019-09-19 19:29:27 +02:00
Wellington Chevreuil
fc4fa46d8e HBASE-23047 ChecksumUtil.validateChecksum logs an INFO message inside a "if(LOG.isTraceEnabled())" block.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit a85c6b48672552136ead76a682458c884a558dc2)
2019-09-19 15:35:01 +01:00
Guanghao Zhang
1a85bcff7a HBASE-23049 TableDescriptors#getAll should return the tables ordering by the name which contain namespace (#639)
Signed-off-by: stack <stack@apache.org>
2019-09-20 05:28:23 +08:00
Peter Somogyi
ffec1500f6 Revert "HBASE-22700 refactor isMetaClearingException (#578)"
This reverts commit 5ccab83d6c7f65c7169f74c51c58df48b2cef80e.
2019-09-19 13:22:12 +02:00
Guanghao Zhang
91a6134a41 HBASE-23041 Should not show split parent regions in HBCK report's unknown server part (#634)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 10:59:47 +08:00
Guanghao Zhang
fb82c8ecb0 HBASE-23044 CatalogJanitor#cleanMergeQualifier may clean wrong parent regions (#637)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 08:10:26 +08:00
Guanghao Zhang
e50d1f5a53 HBASE-23037 Make the split WAL related log more readable (#632)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-09-19 08:10:14 +08:00
Sean Busbey
7e04eebc62 HBASE-23040 give friendly message about unknown server when running unload for RegionMover.
closes #635

Signed-off-by: stack <stack@apache.org>
(cherry picked from commit ab076b07313fe7b86a010eca38ac8db7de405969)
2019-09-18 06:31:50 -05:00
Sean Busbey
1110e23a61 HBASE-21056 Findbugs false positive: BucketCache.persistToFile may fail to clean up java.io.OutputStream
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-09-18 09:58:32 +02:00
Viraj Jasani
faed3e457d HBASE-23024 Replace initcause with Constructor arg (#627)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-17 13:40:07 +02:00
Caroline Zhou
aec5638add
HBASE-22804 Provide an API to get list of successful regions and total expected regions in Canary
Signed-off-by: Xu Cang <xucang@apache.org>
2019-09-16 10:57:14 -07:00
johnhomsea
5ccab83d6c HBASE-22700 refactor isMetaClearingException (#578) 2019-09-16 10:31:52 -07:00
ravowlga123
cfce7a7604 HBASE-22899 logging improvements for snapshot operations w/large manifests (#547)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-09-14 10:18:03 -07:00
Viraj Jasani
63568854b6 HBASE-22760 : Pause/Resume/Query Snapshot Auto Cleanup Activity (#618) 2019-09-13 10:37:22 +05:30
ramkrish86
909bbaa667 HBASE-22929 - MemStoreLAB ChunkCreator may memory leak(ram) (#614) 2019-09-13 10:29:30 +05:30
stack
9059a41fe0 HBASE-22796 [HBCK2] Add fix of overlaps to fixMeta hbck Service
Addendum for branch-2+.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-09-12 13:27:29 -07:00
Guanghao Zhang
31a0db01c6 HBASE-22993 HBCK report UI showed -1 if hbck chore not running (#610)
Signed-off-by: stack <stack@apache.org>
2019-09-12 09:11:46 +08:00
Guanghao Zhang
9f31d744f6 HBASE-23014 Should not show split parent regions in hbck report UI (#609) 2019-09-11 10:13:00 -07:00
meiyi
516e921cc0 HBASE-23009 TestSnapshotScannerHDFSAclController is broken on branch-2 2019-09-11 15:34:10 +02:00
Guanghao Zhang
4224977c0f HBASE-23005 Table UI showed exception message when table is disabled (#602)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-09-11 10:01:45 +08:00
stack
5f71c487e7 HBASE-22796 [HBCK2] Add fix of overlaps to fixMeta hbck Service
Signed-off-by: Sakthi <sakthi@apache.org>
2019-09-10 17:05:25 -07:00
stack
ee08bb8d52 HBASE-22859 [HBCK2] Fix the orphan regions on filesystem
Backport of above; only the usage message was changed
in the backport; nothing else. Usage points at refguide.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-09-09 09:31:49 -07:00