Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16,783 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

7809 Commits

Author SHA1 Message Date
Duo Zhang 44afe8cfde HBASE-23163 Refactor HStore.getStorefilesSize related methods (#719) 
Signed-off-by: Lijin Bin <binlijin@apache.org>
 2019-10-14 20:49:35 +08:00
李小保 699fad39a4 HBASE-21540 when set property "hbase.systemtables.compacting.memstore.type" to "basic" or "eager" will cause an exception 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-10-14 08:35:08 +08:00
Lijin Bin 6f6d331f9f HBASE-23159 HStore#getStorefilesSize may throw NPE 2019-10-12 19:45:32 -07:00
Andrew Purtell 56e0652aff
HBASE-23153 PrimaryRegionCountSkewCostFunction SLB function should implement CostFunction#isNeeded (#714) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2019-10-12 13:15:22 -07:00
Karthik Palanisamy bebf251179 HBASE-23152 Compaction_switch does not work by RegionServer name (#713) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 Signed-off-by: stack <stack@apache.org>
 2019-10-12 09:09:44 -07:00
zbq.dean 595bcda9c3 HBASE-23056 Block count is 0 when BucketCache using persistent IOEngine and retrieve from file 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2019-10-12 14:50:20 +08:00
Viraj Jasani 44a7e8b3f1 HBASE-23093 Avoid Optional Anti-Pattern where possible (#673) 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-10-11 17:47:05 +08:00
Baiqiang Zhao 273c0ba69f HBASE-23017 Verify the file integrity in persistent IOEngine 
Signed-off-by: Anoop Sam John <anoopsamjohn@apacher.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
 2019-10-11 14:39:06 +08:00
Karthik Palanisamy 3025c4250c HBASE-23115 Unit change for StoreFileSize and MemstoreSize (table.jsp) (#682) 
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
 2019-10-09 23:36:30 +09:00
haxiaolin fff0f33c5a HBASE-22767 System table RIT STUCK if their RSGroup has no highest version RSes 
Signed-off-by: stack <stack@apache.org>
 2019-10-08 16:32:07 -07:00
Karthik Palanisamy 54083a7626 HBASE-23095 Reuse FileStatus in StoreFileInfo (#674) 
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-10-08 15:48:17 +08:00
Viraj Jasani 15ec3c458c HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command (#611) 
Signed-off-by: huzheng <openinx@gmail.com>
 2019-10-08 09:25:57 +08:00
Sanjeet Nishad 51184980a9 HBASE-23094 Wrong log message in simpleRegionNormaliser while checking if merge is enabled. 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-10-07 16:21:39 +02:00
Bharath Vissapragada 12852d30a3 HBASE-23116: Fix a load balancer logging nit. (#687) 
This commit adds table name to the logging context when
StochasticLoadBalancer is configured "per table". Added some
test coverage with per-table balancer enabled and manually
verified the logs to make sure the table name is formatted
correctly.

Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.com>
(cherry picked from commit 06ff478674)
 2019-10-04 18:16:07 +01:00
Rushabh b5a396749d HBASE-22874 Define a public API for Canary checking and a non-public tool implementation 
Closes #580

* Canary is now an IA.Public interface
* CanaryTool is now the implementation

Signed-off-by: Sean Busbey <busbey@apache.org>
 2019-10-03 14:34:15 -07:00
stack 61bc57f525 Revert " HBASE-23055 Alter hbase:meta (#655)" 
"Too radical for branch-2"

This reverts commit ff217d4269.
 2019-10-01 08:34:09 -07:00
Michael Stack ff217d4269
HBASE-23055 Alter hbase:meta (#655) 
Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.


   Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
 2019-09-30 09:27:41 -07:00
Duo Zhang 1a2d47b6c7 HBASE-23079 Addendum fix broken UT 2019-09-30 15:46:46 +08:00
Duo Zhang 1e051a7867 HBASE-23079 RegionRemoteProcedureBase should override setTimeoutFailure (#672) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-09-30 11:57:19 +08:00
chenxu14 be932487e8 HBASE-22965 RS Crash due to DBE reference to an reused ByteBuff (#603) 
Signed-off-by: huzheng <openinx@gmail.com>
 2019-09-30 09:20:35 +08:00
Guanghao Zhang 9f703fc3b2 HBASE-23078 BaseLoadBalancer should consider region replicas when randomAssignment and roundRobinAssignment (#663) 
Signed-off-by: stack <stack@apache.org>
 2019-09-29 18:44:41 +08:00
Ankit Singhal 9f7c73caa2 HBASE-23054 Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm 2019-09-28 09:30:25 -07:00
Guanghao Zhang ebaae125cc HBASE-23035 Retain region to the last RegionServer make the failover slower (addendum) (#652) 2019-09-29 04:32:07 +08:00
Sanjeet Nishad 6def152819 HBASE-23038 Provide consistent and clear logging about disabling chores 
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit b45c0d0efa)
 2019-09-27 20:13:18 -05:00
shardul-cr7 43b4ad9e8a HBASE-22012 Prevent DISABLE quota violation policy from disabling violation policy 
Space quotas has a feature which intends to avoid enacting a space quota
violation policy when only a subset of the Regions for that Table have
reported their space usage (under the assumption that we cannot make an
informed decision if we do not include all regions in our calculations).

This had the unintended side-effect, when a table is disabled as a part
of a violation policy, of causing the regions for that table to not be
reported which disables the violation policy and enables the table.

Need to make sure that when a table is disabled because of a violation
policy that the code does not automatically move that table out of
violation because region sizes are not being reported (because those
regions are not open).

Closes #572

Signed-off-by: Josh Elser <elserj@apache.org>
 2019-09-26 16:19:28 -04:00
BukrosSzabolcs cd9367512a HBASE-22982: region server suspend/resume and graceful rolling restart actions (#592) 
* Add chaos monkey action for suspend/resume region servers
* Add chaos monkey action for graceful rolling restart
* Add these to relevant chaos monkeys

Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-09-26 11:56:17 +02:00
zbq.dean 0c5b6df52e HBASE-22975 Add read and write QPS metrics at server level and table level 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2019-09-23 15:03:12 +08:00
Pankaj 6fe7063ffc HBASE-22930 Set unique name to longCompactions/shortCompactions/split threads (#585) 2019-09-22 12:26:56 +02:00
Wellington Ramos Chevreuil 38c8bd3731
HBASE-22380 break circle replication when doing bulkload (#494) 
Signed-off-by:  stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Norbert Kalmar <nkalmar@cloudera.com>
 2019-09-21 14:39:06 +01:00
Guanghao Zhang adc0197191 HBASE-23035 Retain region to the last RegionServer make the failover slower (#631) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-09-21 21:13:10 +08:00
wuqiong 7195ab6a15 HBASE-23058 Should be 'Column Family Name' in table.jsp 
Signed-off-by: Guanghao Zhang <zhangguanghao1@xiaomi.com>
 2019-09-21 19:27:15 +08:00
Stig Døssing 047bbfd4c2 HBase-22027: Split non-MR related parts of TokenUtil off into a ClientTokenUtil, and move ClientTokenUtil to hbase-client (#649) 2019-09-20 17:38:41 -07:00
shardul-cr7 f1d3b54422 HBASE-22142 Drop table RegionSizes with namespace quota 
There was a bug in which we would not drop the RegionSizes
for a table in a namespace, where the namespace had a quota
on it. This allowed a scenario in which recreation of a table
inside of a namespace would unintentionally move into violation
despite the table being empty. Need to make sure the RegionSizes
are dropped on table deletion if there is _any_ quota applying
to that table.

Signed-off-by: Josh Elser <elserj@apache.org>
 2019-09-20 12:09:45 -04:00
shardul-cr7 ffbf8503ea HBASE-22944 Check for hbase:quota table existence in SpaceQuotaRefresherChore 
During startup, it's possible that quotas are enabled but the Master has
not yet created the hbase:quotas table.

Closes #559

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2019-09-20 12:07:32 -04:00
sreenivasulureddy 63593d6cce HBASE-22649: Encode StoreFile path URLs in the UI to handle scenarios where CF contains special characters (like # etc.) 
Signed-off-by: Sakthi<sakthi@apache.org>
(cherry picked from commit 49718b8b46)
 2019-09-20 00:59:08 -07:00
Duo Zhang 5b75bef36e HBASE-22941 Addendum fix NPE in UT 2019-09-20 14:37:01 +08:00
Duo Zhang 0ed11fe452 HBASE-23051 Addendum add missing import 2019-09-20 11:35:34 +08:00
Istvan Toth 908a4d8f2b HBASE-22941 merge operation returns parent regions in random order (#556) 
* HBASE-22941 merge operation returns parent regions in random order

store and return the merge parent regions in ascending order

remove left over check for exactly two merged regions

add unit test

* use SortedMap type to emphasise that the Map is sorted.

* use regionCount consistently and checkstyle fixes

* Delete tests that expect multiregion merges to fail.

Signed-off-by: stack <stack@apache.org>
 2019-09-20 11:15:21 +08:00
Peter Somogyi 294487ce20 HBASE-23051 Remove unneeded Mockito.mock invocations (#641) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2019-09-19 19:29:27 +02:00
Wellington Chevreuil fc4fa46d8e HBASE-23047 ChecksumUtil.validateChecksum logs an INFO message inside a "if(LOG.isTraceEnabled())" block. 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit a85c6b4867)
 2019-09-19 15:35:01 +01:00
Guanghao Zhang 1a85bcff7a HBASE-23049 TableDescriptors#getAll should return the tables ordering by the name which contain namespace (#639) 
Signed-off-by: stack <stack@apache.org>
 2019-09-20 05:28:23 +08:00
Peter Somogyi ffec1500f6 Revert "HBASE-22700 refactor isMetaClearingException (#578)" 
This reverts commit 5ccab83d6c.
 2019-09-19 13:22:12 +02:00
Guanghao Zhang 91a6134a41 HBASE-23041 Should not show split parent regions in HBCK report's unknown server part (#634) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-09-19 10:59:47 +08:00
Guanghao Zhang fb82c8ecb0 HBASE-23044 CatalogJanitor#cleanMergeQualifier may clean wrong parent regions (#637) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-09-19 08:10:26 +08:00
Guanghao Zhang e50d1f5a53 HBASE-23037 Make the split WAL related log more readable (#632) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-09-19 08:10:14 +08:00
Sean Busbey 7e04eebc62 HBASE-23040 give friendly message about unknown server when running unload for RegionMover. 
closes #635

Signed-off-by: stack <stack@apache.org>
(cherry picked from commit ab076b0731)
 2019-09-18 06:31:50 -05:00
Sean Busbey 1110e23a61 HBASE-21056 Findbugs false positive: BucketCache.persistToFile may fail to clean up java.io.OutputStream 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-09-18 09:58:32 +02:00
Viraj Jasani faed3e457d HBASE-23024 Replace initcause with Constructor arg (#627) 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-09-17 13:40:07 +02:00
Caroline Zhou aec5638add
HBASE-22804 Provide an API to get list of successful regions and total expected regions in Canary 
Signed-off-by: Xu Cang <xucang@apache.org>
 2019-09-16 10:57:14 -07:00
johnhomsea 5ccab83d6c HBASE-22700 refactor isMetaClearingException (#578) 2019-09-16 10:31:52 -07:00
ravowlga123 cfce7a7604 HBASE-22899 logging improvements for snapshot operations w/large manifests (#547) 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2019-09-14 10:18:03 -07:00
Viraj Jasani 63568854b6 HBASE-22760 : Pause/Resume/Query Snapshot Auto Cleanup Activity (#618) 2019-09-13 10:37:22 +05:30
ramkrish86 909bbaa667 HBASE-22929 - MemStoreLAB ChunkCreator may memory leak(ram) (#614) 2019-09-13 10:29:30 +05:30
stack 9059a41fe0 HBASE-22796 [HBCK2] Add fix of overlaps to fixMeta hbck Service 
Addendum for branch-2+.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-09-12 13:27:29 -07:00
Guanghao Zhang 31a0db01c6 HBASE-22993 HBCK report UI showed -1 if hbck chore not running (#610) 
Signed-off-by: stack <stack@apache.org>
 2019-09-12 09:11:46 +08:00
Guanghao Zhang 9f31d744f6 HBASE-23014 Should not show split parent regions in hbck report UI (#609) 2019-09-11 10:13:00 -07:00
meiyi 516e921cc0 HBASE-23009 TestSnapshotScannerHDFSAclController is broken on branch-2 2019-09-11 15:34:10 +02:00
Guanghao Zhang 4224977c0f HBASE-23005 Table UI showed exception message when table is disabled (#602) 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2019-09-11 10:01:45 +08:00
stack 5f71c487e7 HBASE-22796 [HBCK2] Add fix of overlaps to fixMeta hbck Service 
Signed-off-by: Sakthi <sakthi@apache.org>
 2019-09-10 17:05:25 -07:00
stack ee08bb8d52 HBASE-22859 [HBCK2] Fix the orphan regions on filesystem 
Backport of above; only the usage message was changed
in the backport; nothing else. Usage points at refguide.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-09-09 09:31:49 -07:00
shardul-cr7 e5165428dc HBASE-22013 Prune replicas when computing the reported number of regions for a table 
Closes #570

Signed-off-by: Josh Elser <elserj@apache.org>
 2019-09-09 11:06:40 -04:00
Peter Somogyi e7c943f82f HBASE-22979 Call ChunkCreator.initialize in TestHRegionWithInMemoryFlush (#589) 
Signed-off-by: Zheng Hu <openinx@gmail.com>
 2019-09-09 11:53:53 +02:00
chenxu14 4282d70212 HBASE-22802 Avoid temp ByteBuffer allocation in FileIOEngine#read (#583) 2019-09-09 17:44:49 +08:00
Duo Zhang 1568e05507 HBASE-22963 Netty ByteBuf leak in rpc client implementation (#577) 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-09-08 22:10:18 +08:00
Guanghao Zhang a4a34e0347 HBASE-22964 Fix flaky TestClusterRestartFailover and TestClusterRestartFailoverSplitWithoutZk (#574) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-09-09 02:48:24 +08:00
stack aeacfd3d59 Revert "HBASE-22930 Set unique name to longCompactions/shortCompactions threads (#548)" 
This reverts commit 018181c7a4.
 2019-09-04 09:18:01 -07:00
Pankaj 018181c7a4 HBASE-22930 Set unique name to longCompactions/shortCompactions threads (#548) 2019-09-04 08:47:42 -07:00
stack bfa19f3a5a HBASE-22970 split parents show as overlaps in the HBCK Report 
Check if overlap is split parent.

Cleaned up the HBCK Report page too with some notes that it is made of
two reports; have the two sections display the same.
 2019-09-03 14:50:08 -07:00
stack 6fae2f8d0d HBASE-22961 Deprecate hbck1 in core 
Adds deprecations on HBaseFsck and on supporting classes such as
the reporting Interface. Provides alternatives in FSUtils for
progress reporting and deprecates methods that use hbck1 facility.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-09-03 11:07:13 -07:00
meiyi b0456f3297 HBASE-22878 Show table throttle quotas in table jsp (#510) 2019-09-03 09:53:44 +08:00
meiyi 076bfa1cc6 HBASE-22946 Fix TableNotFound when grant/revoke if AccessController is not loaded (#561) 2019-09-02 15:02:25 +08:00
linkaline 7e096e07ff HBASE-22896 TestHRegion.testFlushMarkersWALFail is flaky (#551) 
* HBASE-22896 TestHRegion.testFlushMarkersWALFail is flaky

* delete blank line
 2019-09-01 21:46:30 -07:00
meiyi 41fddd411e HBASE-22945 Show quota infos in master UI (#560) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-09-01 22:43:29 +08:00
康智冬 576e688cec HBASE-22962 Fix typo in javadoc description (#569) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-09-01 20:16:05 +08:00
Sean Busbey 6ece958268 HBASE-22943 Various procedures should not cache log trace level (#557) 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit 83e77940c5)
 2019-08-29 13:49:12 -05:00
linkaline 84b4a61e88 HBASE-22881 Fix non-daemon threads in hbase server implementation (#512) (#536) 
Signed-off-by: stack <stack@apache.org>
 2019-08-28 09:48:27 -07:00
David Manning 9b9f055ef7
HBASE-22935 Fix false warn of stuck MonitoredRPCHandler MonitoredTask 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-08-27 17:50:09 -07:00
Pierre Zemb a18be1911e HBASE-22618 added the possibility to load custom cost functions 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2019-08-27 14:26:26 +01:00
Istvan Toth caba5cf990 HBASE-22922 Lock all regions to be merged in case of multi region merge (#542) 
* HBASE-22922 Lock all regions to be merged in case of multi region merge

* HBASE-22922 Lock all regions to be merged in case of multi region merge
 (addendum)

fix off-by-one error in patch
Signed-off-by: stack <stack@apache.org>
 2019-08-26 21:10:40 -07:00
Aman Poonia 1eac16e78f HBASE-22872 Don't try to create normalization plan unnecesarily when split and merge both are disabled 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2019-08-27 11:35:07 +08:00
Sakthi fe689bf1d4 HBASE-22895 Fix the flakey TestSpaceQuotas (#526) 
Break down TestSpaceQuotas(TSQ) into TSQBasicFunctioning, TSQDropTable, TSQIncrease, TSQOnBulkLoad, TSQOnNonExistingTables, TSQRemoval & TSQSwitchPolicies

Signed-off-by: Stack <stack@apache.org>
 2019-08-25 00:33:10 -07:00
Bing Xiao df0a40f61f HBASE-22894 Move testOpenRegionFailedMemoryLeak to dedicated class 
Amending-Author: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-08-25 13:42:36 +08:00
Pankaj 5a1603f2ba HBASE-22904 NPE occurs when RS send space quota usage report during HMaster init (#529) 
* NPE occurs when RS send space quota usage report during HMaster init

* Added the debug log

 Signed-off-by: Sakthi <sakthi@apache.org>
 Signed-off-by: stack <stack@apache.org>
 2019-08-24 20:56:52 -07:00
Sakthi 7300108dfb
HBASE-22910 Enable TestMultiVersionConcurrencyControl (#531) 
Signed-off-by: Stack <stack@apache.org>
 2019-08-24 18:24:19 -07:00
Pankaj cb5651b1e6 HBASE-22806 Recreating a deleted column family brings back the deleted cells (#530) 
Signed-off-by: stack <stack@apache.org>
 2019-08-24 18:19:20 -07:00
Tai-Ying Lee da87ebb481
HBASE-20509 putting List in HashSet directly without using addAll function to improve performance 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2019-08-24 00:03:04 +02:00
openinx 4275c5f374 HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table (#513) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
 2019-08-22 20:20:21 +08:00
openinx 53522f36cc HBASE-22810 Initialize an separate ThreadPoolExecutor for taking/restoring snapshot (addendum - use the old config key) (#517) 2019-08-22 11:53:46 +08:00
Viraj Jasani e28a7c29c5 HBASE-22863 Cleanup transitive Jackson1 vulnerable dependencies(forward-port HBASE-22728) (#505) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
 2019-08-21 22:16:52 +08:00
shahrs87 0a1950f43c HBASE-22601 Misconfigured addition of peers leads to cluster shutdown. (#500) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-08-21 21:56:04 +08:00
Andrew Purtell 22814f8138
Revert "HBASE-22823 Mark Canary as Public/Evolving" 
This reverts commit 4b60eaca17.

See HBASE-22874
 2019-08-20 09:46:10 -07:00
stack e9874a6abd HBASE-22882 TestFlushSnapshotFromClient#testConcurrentSnapshottingAttempts is flakey (was written flakey) 
Addendum; just remove the test altogether

 Signed-off-by:  Nick Dimiduk <ndimiduk@apache.org>
 2019-08-19 14:27:06 -07:00
stack b7c7b4a570 HBASE-22882 TestFlushSnapshotFromClient#testConcurrentSnapshottingAttempts is flakey (was written flakey) 2019-08-19 12:58:43 -07:00
satanson 29ed4157b3 HBASE-22870 reflection fails to access a private nested class 
Signed-off-by Reid Chan <reidchan@apache.org>
 2019-08-18 10:30:04 +08:00
Duo Zhang c295fd7c6a HBASE-22871 Move the DirScanPool out and do not use static field (#504) 
Signed-off-by: Zheng Hu <openinx@gmail.com>
Signed-off-by: Reid Chan <reidchan@apache.org>
 2019-08-17 23:46:38 +08:00
Huon Wilson 8cbe20c382 HBASE-22841 Add more factory functions to TimeRange 
These functions make it easier to possible to use
`org.apache.hadoop.hbase.client.Table.CheckAndMutateBuilder#timeRange`
with more interesting ranges, without being forced to use the
deprecated constructors.

Signed-off-by: huzheng <openinx@gmail.com>
 2019-08-17 10:12:48 +08:00
Daisuke Kobayashi f9e20a6189 HBASE-22860 Master's webui returns NPE/HTTP 500 under maintenance mode 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2019-08-16 10:35:20 +01:00
Guanghao Zhang 43603ec3b9 HBASE-22858 Add HBCK Report to master's header.jsp (#497) 
Signed-off-by: Stack <stack@apache.org>
Signed-off-by: Sakthi <sakthi@apache.org>
 2019-08-16 14:37:11 +08:00
meiyi 3ff39386f4 HBASE-22842 Tmp directory should not be deleted when master restart used for user scan snapshot feature (#485) 2019-08-16 10:48:26 +08:00
Itsuki Toyota dfe7e27e25 HBASE-22833 MultiRowRangeFilter should provide a method for creating… (#493) 
* HBASE-22833: MultiRowRangeFilter should provide a method for creating a filter which is functionally equivalent to multiple prefix filters

* Delete superfluous comments

* Add description for MultiRowRangeFilter constructor

* Add null check for rowKeyPrefixes

* Fix checkstyle

Signed-off-by: huzheng <openinx@gmail.com>
 2019-08-16 10:06:44 +08:00
Caroline Zhou 4b60eaca17
HBASE-22823 Mark Canary as Public/Evolving 
Signed-off-by: Xu Cang <xucang@apache.org>
 2019-08-15 14:20:41 -07:00
Guanghao Zhang 76a68bd27c HBASE-22857 Fix the failed ut TestHRegion and TestHRegionWithInMemoryFlush (#495) 
Signed-off-by: Peter Somogyi psomogyi@apache.org
Signed-off-by: stack stack@apache.org
 2019-08-15 13:19:25 -07:00
openinx 6f9db4660a HBASE-22810 Initialize an separate ThreadPoolExecutor for taking/restoring snapshot (#486) 2019-08-15 11:22:08 +08:00
Andrew Purtell 3233f6730f
HBASE-22828 Log a region close journal (#471) 
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
 2019-08-13 15:43:51 -07:00
syedmurtazahassan c5b29265a3
HBASE-22725 Remove all remaining Javadoc warnings 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2019-08-11 22:02:53 +02:00
Geoffrey Jacoby 7f0ab84214
HBASE-22623 - Add RegionObserver coprocessor hook for preWALAppend (#390) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-08-09 14:29:41 -07:00
Daisuke Kobayashi e4157f0fe6 HBASE-22543 Revisit HBASE-21207 to make all values fully sortable 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2019-08-09 14:30:15 +01:00
Guanghao Zhang 9f6c37bcc3 HBASE-22824 Show filesystem path for the orphans regions on filesystem (#469) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-08-09 15:07:48 +08:00
Sakthi 65d6390e50 HBASE-22803 Modify config value range to enable turning off of the hbck chore (#466) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-08-08 17:46:00 -07:00
Guanghao Zhang 796cbf9bc3 HBASE-22808 HBCK Report showed the offline regions which belong to disabled table (#465) 
Signed-off-by: Stack <stack@apache.org>
 2019-08-08 17:00:21 +08:00
meiyi e70a5a687c HBASE-22776 Rename config names in user scan snapshot feature (#440) 2019-08-08 16:18:58 +08:00
Guanghao Zhang ee40d47b3a HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem (#461) 
Signed-off-by: Sakthi <sakthi@apache.org>
 2019-08-08 08:18:40 +08:00
stack 225c6cd5bc HBASE-22777 Add a multi-region merge (for fixing overlaps) 
Makes MergeTableRegionsProcedure do more than just two regions at a
time. Compatible as MTRP was done considering one day it'd do more than
two at a time.

Changes hardcoded assumption that merge parent regions are named
mergeA and mergeB in a column on the resultant region. Instead
can have N columns on the merged region, one for each parent
merged. Column qualifiers all being with 'merge'.

Most of code below is undoing the assumption that there are two
parents on a merge only.
 2019-08-07 16:58:06 -07:00
syedmurtazahassan d39736a274 HBASE-22812 InterfaceAudience annotation in CatalogJanitor uses fully-qualified name (#462) 
Signed-off-by: Stack <stack@apache.org>
Signed-off-by: Sakthi <sakthi@apache.org>
 2019-08-07 10:40:43 -07:00
Wellington Chevreuil e13ed336fe HBASE-22417 DeleteTableProcedure.deleteFromMeta method should remove table from Master's table descriptors cache 
Signed-off-by: stack <stack@apache.org>
 2019-08-07 12:39:01 +01:00
Wellington Chevreuil afb32a3361 HBASE-22731 ReplicationSource and HBaseInterClusterReplicationEndpoint log messages should include a target Peer identifier 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-08-07 10:52:30 +01:00
Andor Molnár cc4b5103de HBASE-22759 Add user info to AUDITLOG events when doing grant/revoke 2019-08-07 11:07:25 +02:00
sreenivasulureddy be84e36bee HBASE-22793 RPC server connection is logging user as NULL principal 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Pankaj Kumar <pankaj.kr@huawei.com>
 2019-08-06 14:43:55 +02:00
stack 5fa8668202 HBASE-22771 [HBCK2] fixMeta method and server-side support 
This is a first cut at this patch. Implements hold fixing only
currently.

Add a fixMeta method to Hbck Interface.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Bug fix. If hole is on end of last table, I wasn't seeing it.

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
 Add a general meta fixer class. Explains up top why this stuff doesn't
 belong inside MetaTableAccessor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 Break out the filesystem messing so don't have to copy it nor do more
 than is needed doing fixup for Region holes.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java
 Change behavious slightly. If directory exists, don't fail as we did
 but try and keep going and create .regioninfo file if missing (or
 overwrite if in place). This should make it idempotent. Can rerun
 command. Lets see if any repercussions in test suite.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
 Add test.

Signed-off-by: Zheng Hu <openinx@gmail.com>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-08-05 22:33:28 -07:00
stack 2bc9526080 HBASE-22771 [HBCK2] fixMeta method and server-side support 
This is a first cut at this patch. Implements hold fixing only
currently.

Add a fixMeta method to Hbck Interface.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Bug fix. If hole is on end of last table, I wasn't seeing it.

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
 Add a general meta fixer class. Explains up top why this stuff doesn't
 belong inside MetaTableAccessor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
 Break out the filesystem messing so don't have to copy it nor do more
 than is needed doing fixup for Region holes.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java
 Change behavious slightly. If directory exists, don't fail as we did
 but try and keep going and create .regioninfo file if missing (or
 overwrite if in place). This should make it idempotent. Can rerun
 command. Lets see if any repercussions in test suite.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMetaFixer.java
 Add test.

Signed-off-by: Zheng Hu <openinx@gmail.com>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-08-05 22:33:20 -07:00
Duo Zhang 10d22b5454 HBASE-22539 WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used (#437) 
Signed-off-by: Zheng Hu <openinx@gmail.com>
 2019-08-05 16:45:00 +08:00
Wellington Chevreuil 5cd0433dc9 HBASE-22707 [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore 2019-08-01 16:56:57 +01:00
Guanghao Zhang 6a20872e01 HBASE-22737 Add a new admin method and shell cmd to trigger the hbck chore to run (#425) 
Signed-off-by: stack <stack@apache.org>
 2019-08-01 09:02:26 +08:00
Biju Nair d335c97081 HBASE-22363 Remove hardcoded number of read cache block buckets 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-07-31 09:19:28 +02:00
stack 2caf6db0c5 HBASE-22741 Show catalogjanitor consistency complaints in new 'HBCK Report' page 
Signed-off-by: huzheng <openinx@gmail.com>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-07-30 11:17:15 -07:00
openinx 300f906200 HBASE-22758 Remove the unneccesary info cf deletion in DeleteTableProcedure#deleteFromMeta (#424) 2019-07-30 21:37:52 +08:00
stack 4565e9f43b HBASE-22723 Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta 
Refactor of CatalogJanitor so it generates a
Report on the state of hbase:meta when it runs. Also
refactor so CJ runs even if RIT (previous it would
punt on running if RIT) so it can generate a 'Report'
on the interval regardless. If RIT, it just doesn't
go on to do the merge/split GC as it used to.

If report finds an issue, dump as a WARN message
to the master log.

Follow-on is to make the Report actionable/available
for the Master to pull when it goes to draw the hbck
UI page (could also consider shipping the Report as
part of ClusterMetrics?)

Adds new, fatter Visitor to CJ, one that generates
Report on each run keeping around more findings as
it runs.

Moved some methods around so class reads better;
previous methods were randomly ordered in the class.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Make a few handy methods public.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add utility as defaults on the Inteface; i.e. is this the first region
 in table, is it last, does a passed region come next, or does passed
 region overlap this region (added tests for this new stuff).

M hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
 Bugfix... handle case where buffer passed is null.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Lots of change, reorg., but mostly adding consistency checking
 to the visitor used scanning hbase:meta on a period and the
 generation of a Report on what the scan has found traversing
 hbase:meta. Added a main so could try the CatalogJanitor against
 a running cluster.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitorCluster.java
 Fat ugly test for CatalogJanitor consistency checking.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
 Add tests for new functionality in RI.

M hbase-shell/src/main/ruby/hbase/table.rb
 Bug fix for case where meta has a null regioninfo; scan was aborting.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2019-07-29 16:07:10 -07:00
stack d23236a452 HBASE-22751 table.jsp fails if ugly regions in table 2019-07-29 15:14:41 -07:00
stack 8b1952a42a HBASE-22751 table.jsp fails if ugly regions in table 2019-07-29 15:06:22 -07:00
Venkatesh Sridharan 4a83de75c7 HBASE-22313 Add method to FsDelegation token to accept token kind (#199) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-07-29 11:18:48 +02:00
meiyi 8bd90f2e55 HBASE-22580 Add a table attribute to make user scan snapshot feature configurable for table (#336) 2019-07-29 10:03:54 +08:00
Guanghao Zhang 4c4f0184e0 HBASE-22709 Add a chore thread in master to do hbck checking (#404) 
Signed-off-by: stack <stack@apache.org>
 2019-07-27 18:31:33 +08:00
Guanghao Zhang da4d72c2d4 HBASE-22742 [HBCK2] Add more log for hbck operations at master side 
Signed-off-by: stack <stack@apache.org>
 2019-07-27 12:39:43 +08:00
stack f7ac3badbc HBASE-22382 Refactor tests in TestFromClientSide; ADDENDUM to fix tests 2019-07-26 13:41:50 -07:00
linkaline 9f961ea1d8 HBASE-22733 TestSplitTransactionOnCluster.testMasterRestartAtRegionSplitPendingCatalogJanitor is flakey (#408) 
Signed-off-by: stack <stack@apache.org>
 2019-07-25 11:11:46 -07:00
Jan Hentschel 62710d5835 HBASE-22594 Clean up for backup examples (#315) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-07-25 15:45:35 +08:00
meiyi ff9589c9ab HBASE-22578 HFileCleaner should not delete empty ns/table directories used for user san snapshot feature (#337) 2019-07-25 10:52:23 +08:00
Andor Molnár ed72cd7c1a HBASE-22382 Refactor tests in TestFromClientSide (#385) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-07-25 10:41:06 +08:00
Guanghao Zhang 73b2bd5185 HBASE-22721 Refactor HBaseFsck: move the inner class out 
Signed-off-by: stack <stack@apache.org>
 2019-07-25 08:13:30 +08:00
Jeongdae Kim f61e987c6a
HBASE-22715 All scan requests should be handled by scan handler threads in RWQueueRpcExecutor (#393) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestSimpleRpcScheduler.java
 2019-07-24 11:56:54 -07:00
syedmurtazahassan 169abb48c4
HBASE-22603 Javadoc Warnings related to @link tag 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2019-07-23 14:33:01 +02:00
syedmurtazahassan fa4466d345 HBASE-22610 [BucketCache] Rename hbase.offheapcache.minblocksize 
Signed-off-by: Reid Chan <reidchan@apache.org>
Co-authored-by: Reid Chan <reidchan@apache.org>
 2019-07-23 11:17:42 +08:00
Reid Chan be5e3de8af Revert "HBASE-22610 [BucketCache] Rename hbase.offheapcache.minblocksize" 
Reason: Deprecated a wrong parameter.

This reverts commit e95bdf415c.
 2019-07-23 11:07:41 +08:00
syedmurtazahassan e95bdf415c HBASE-22610 [BucketCache] Rename hbase.offheapcache.minblocksize 
* Deprecated old attribute and introduced a new one

* Removed unnecessary import

* Added two import configs and removed one

Signed-off-by: Reid Chan <reidchan@apache.org>
 2019-07-23 10:42:41 +08:00
Viraj Jasani 3a72ccefbd
HBASE-22648 Snapshot TTL (#371) 
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotDescriptionUtils.java
 2019-07-22 15:08:20 -07:00
Toshihiro Suzuki 15671e5002 HBASE-22690 Deprecate / Remove OfflineMetaRepair in hbase-2+ 
Signed-off-by: stack <stack@apache.org>
 2019-07-22 23:24:47 +09:00
openinx 6fb03e63b6 HBASE-22663 The HeapAllocationRatio in WebUI is not accurate because all of the heap allocation will happen in another separated allocator named HEAP (#365) 2019-07-20 23:36:21 +08:00
Geoffrey Jacoby 439da96a1c
HBASE-22622 WALKey Extended Attributes (#352) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-07-19 15:31:12 -07:00
linkaline b9b7b6e734 HBASE-21426 TestEncryptionKeyRotation.testCFKeyRotation is flaky (#375) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-07-19 09:43:42 +08:00
Guanghao 4c89354f3b HBASE-22673 Avoid to expose protobuf stuff in Hbck interface 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-07-19 08:10:38 +08:00
sreenivasulureddy 2fa6d062f5 HBASE-22537 Split happened Replica region can not be deleted after deleting table successfully and restarting RegionServer 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2019-07-18 15:26:10 +01:00
Guanghao Zhang 22a991e8b6 HBASE-22527 [hbck2] Add a master web ui to show the problematic regions (addendum) 2019-07-18 17:51:27 +08:00
Guanghao Zhang 9e59b7ff5e HBASE-22527 [hbck2] Add a master web ui to show the problematic regions 2019-07-18 08:26:31 +08:00
Guanghao db4cd6bd45 HBASE-22704 Avoid NPE when access table.jsp and snapshot.jsp but master not finish initialization (#389) 2019-07-18 08:26:20 +08:00
stack d904344421 HBASE-7191 HBCK - Add offline create/fix hbase.version and hbase.id 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-07-17 07:52:32 -07:00
linkaline abf3d183cd HBASE-20368 Fix RIT stuck when a rsgroup has no online servers but AM's pendingAssginQueue is cleared (#354) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-07-17 22:11:24 +08:00
Viraj Jasani 6c834b974c
HBASE-22643 : Delete region without archiving only if regiondir is present 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
 2019-07-16 17:41:24 -07:00
Duo Zhang 5333d8f1b4 HBASE-22684 The log rolling request maybe canceled immediately in LogRoller due to a race (#378) 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-07-15 21:52:55 +08:00
syedmurtazahassan 955e643e1d
HBASE-22586 Javadoc Warnings related to @param tag 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2019-07-13 16:56:53 +02:00
Duo Zhang 3eda33a048 HBASE-22681 The 'assert highestUnsyncedTxid < entry.getTxid();' in AbstractFWAL.append may fail when using AsyncFSWAL (#376) 
Signed-off-by: Zheng Hu <openinx@gmail.com>
 2019-07-13 22:16:34 +08:00
syedmurtazahassan b6e66699a9
HBASE-22571 Javadoc Warnings related to @return tag 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2019-07-13 13:48:45 +02:00
Andrew Purtell 19d8e33798
HBASE-22686 ZkSplitLogWorkerCoordination doesn't allow a regionserver to pick up all of the split work it is capable of (#377) 
Signed-off-by: Xu Cang <xcang@apache.org>
 2019-07-12 18:12:39 -07:00
Reid Chan 8222631ee3 HBASE-22656 [Metrics] Table metrics 'BatchPut' and 'BatchDelete' are never updated 
Signed-off-by: Michael Stack <stack@apache.org>
 2019-07-07 23:26:02 +08:00
openinx f84edde148 HBASE-22582 The Compaction writer may access the lastCell whose memory has been released when appending fileInfo in the final (#341) 2019-07-04 14:30:37 +08:00
haxiaolin 5087fddedf HBASE-22403 Balance in RSGroup should consider throttling and a failure affects the whole 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-07-04 08:51:55 +08:00
Guanghao 78f704796e HBASE-22624 Should sanity check table configuration when clone snapshot to a new table 2019-07-03 18:28:48 +08:00
Andor Molnár 2f8ad5a51f HBASE-13798 TestFromClientSide* don't close the Table 2019-07-02 11:29:01 +02:00
Mate Szalay-Beko ede6a0a97b HBASE-22637 fix flaky test in TestMetaTableMetrics 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Sakthi <sakthivel.azhaku@gmail.com>
 2019-07-02 09:37:01 +02:00
Bing Xiao 12c933ac51 HBASE-21751 WAL creation fails during region open may cause region assign forever fail 
Signed-off-by: Allan Yang <allan163@apache.org>
    Signed-off-by: Michael Stack <stack@apache.org>
 2019-07-01 22:19:42 -07:00
syedmurtazahassan 8fa7800389 HBASE-22572 Javadoc Warnings: @link reference not found (#306) 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: stack <stack@apache.org>
 2019-07-01 21:18:11 -07:00
Bing Xiao 49a4eeab3b HBASE-22169 Open region failed cause memory leak 
Signed-off-by: stack <stack@apache.org>
 2019-07-01 13:51:33 -07:00
Duo Zhang f812352c4e HBASE-22632 SplitTableRegionProcedure and MergeTableRegionsProcedure should skip store files for unknown column families (#344) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-06-27 21:57:11 +08:00
openinx 345f80ea5f HBASE-20060 Add details of off heap memstore into book. (#334) 2019-06-27 16:54:48 +08:00
huzheng 6a3aa789d8 HBASE-22612 Address the final overview reviewing comments of HBASE-21879 2019-06-26 15:40:23 +08:00
openinx af0e23c359 HBASE-22547 Align the config keys and add document for offheap read in HBase Book. (#301) 2019-06-25 22:25:30 +08:00
openinx 4f1b5ab54e HBASE-22531 The HFileReaderImpl#shouldUseHeap return the incorrect true when disabled BlockCache (#304) 2019-06-25 22:22:23 +08:00
openinx 3199175932 HBASE-22491 Separate the heap HFileBlock and offheap HFileBlock because the heap block won't need refCnt and save into prevBlocks list before shipping (#268) 2019-06-25 22:22:16 +08:00
openinx 915d7b2fa4 HBASE-22463 Some paths in HFileScannerImpl did not consider block#release which will exhaust the ByteBuffAllocator (#257) 2019-06-25 22:17:07 +08:00
openinx 1f5cf41345 HBASE-22422 Retain an ByteBuff with refCnt=0 when getBlock from LRUCache (#242) 2019-06-25 22:15:02 +08:00
huzheng 9e5fc2b379 HBASE-22435 Add a UT to address the HFileBlock#heapSize() in TestHeapSize 2019-06-25 22:14:57 +08:00
huzheng b00360731a HBASE-22412 Improve the metrics in ByteBuffAllocator 2019-06-25 22:14:50 +08:00
huzheng 02c528ab5a HBASE-22090 The HFileBlock#CacheableDeserializer should pass ByteBuffAllocator to the newly created HFileBlock 2019-06-25 22:14:42 +08:00
huzheng d7009d258c HBASE-21921 Notify users if the ByteBufAllocator is always allocating ByteBuffers from heap which means the increacing GC pressure 2019-06-25 22:14:36 +08:00
huzheng b3a01b512c HBASE-22122 Change to release mob hfile's block after rpc server shipped response to client 2019-06-25 22:14:23 +08:00
huzheng 39a6a357ff HBASE-21937 Make the Compression#decompress can accept ByteBuff as input 2019-06-25 22:11:51 +08:00
huzheng bb40caf7a8 HBASE-22211 Remove the returnBlock method because we can just call HFileBlock#release directly 2019-06-25 22:11:33 +08:00
huzheng fc460136e0 HBASE-21957 Unify refCount of BucketEntry and refCount of hbase.nio.ByteBuff into one 2019-06-25 22:07:16 +08:00
Mike Drob 92fc520cb0 HBASE-20894 Use proto for BucketCache persistence 2019-06-25 22:00:58 +08:00
huzheng bd8260c07e HBASE-22159 ByteBufferIOEngine should support write off-heap ByteBuff to the bufferArray 2019-06-25 21:28:02 +08:00
huzheng 081dc03e12 HBASE-22127 Ensure that the block cached in the LRUBlockCache offheap is allocated from heap 2019-06-25 21:23:54 +08:00
huzheng 0d294d23a9 HBASE-22005 Use ByteBuff's refcnt to track the life cycle of data block 2019-06-25 21:16:48 +08:00
huzheng f97f6e3e6a HBASE-22016 Rewrite the block reading methods by using hbase.nio.ByteBuff 2019-06-25 21:13:30 +08:00
huzheng 287e014b44 HBASE-21917 Make the HFileBlock#validateChecksum can accept ByteBuff as an input. (addendum) 2019-06-25 21:13:23 +08:00
huzheng 259f8610ca HBASE-21917 Make the HFileBlock#validateChecksum can accept ByteBuff as an input. 2019-06-25 21:13:07 +08:00
huzheng c19592a971 HBASE-21916 Abstract an ByteBuffAllocator to allocate/free ByteBuffer in ByteBufferPool 2019-06-25 21:12:37 +08:00
Duo Zhang 974e1086d7 HBASE-22617 Recovered WAL directories not getting cleaned up (#330) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-06-25 17:32:39 +08:00
Andrew Purtell 581198ef54
HBASE-22616 responseTooXXX logging for Multi should characterize the component ops (#329) 
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2019-06-24 18:04:59 -07:00
meiyi d0cb77ce8a HBASE-21995 Add a coprocessor to set HDFS ACL for hbase granted user 2019-06-24 11:33:25 +08:00
virajjasani 183aae30c5
HBASE-22520 Avoid possible NPE while performing seekBefore in Hal… (#281) 
HBASE-22520 Avoid possible NPE while performing seekBefore in HalfStoreFileReader
 2019-06-20 15:02:29 -07:00
Mingliang Liu bc737d1ed9
HBASE-21284 Default limits for PressureAwareCompactionThroughputController are too low 
This is a forward port HBASE-21000 from branch-1.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-06-20 12:44:14 -07:00
Reid Chan d30a7ca758 HBASE-22596 [Chore] Separate the execution period between CompactionChecker and PeriodicMemStoreFlusher 
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Xu Cang <xucang@apache.org>
 2019-06-20 10:59:07 +08:00
Duo Zhang 10bd49edaf
HBASE-22600 Document that LoadIncrementalHFiles will be removed in 3.0.0 (#321) 
Signed-off-by: huzheng <openinx@gmail.com>
 2019-06-19 22:43:06 +08:00
Artem Ervits 97393a0c90 HBASE-22561 modify HFilePrettyPrinter to accept non-rootdir directories 
Signed-off-by: huzheng <openinx@gmail.com>
 2019-06-19 15:37:11 +08:00
Jan Hentschel 6cff25cc23
HBASE-22344 Documented the deprecation of public and limited private APIs (#208) 
Signed-off-by: stack <stack@apache.org>
 2019-06-17 22:29:51 +02:00
Reid Chan e2d891172a HBASE-22559 [RPC] set guard against CALL_QUEUE_HANDLER_FACTOR_CONF_KEY 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-06-14 14:39:04 +08:00
Andrew Purtell d3fba8c0c5
HBASE-22530 The metrics of store files count of region are returned to clients incorrectly (Eungsop Yoo) 
Signed-off-by: Xu Cang <xucang@apache.org>
 2019-06-13 17:45:28 -07:00
Murtaza Hassan a2b22a6807
HBASE-22565 Javadoc Warnings: @see cannot be used in inline documentation 2019-06-13 23:14:58 +02:00
Josh Elser 08287f5706 HBASE-22562 Remove dead code: skipControl 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2019-06-13 15:46:58 -04:00
Sergey Shelukhin 89ea709cba HBASE-22408 add dead and unknown server open regions metric to AM 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-06-10 15:46:45 -07:00
zhangduo e948402840 HBASE-22552 Rewrite TestEndToEndSplitTransaction.testCanSplitJustAfterASplit 2019-06-10 14:50:18 +08:00
zhangduo b89120682f HBASE-22551 TestMasterOperationsForRegionReplicas is flakey 2019-06-10 07:54:43 +08:00
Jan Hentschel aa210dda47
HBASE-22373 Moved maven-eclipse-plugin to eclipse profile 2019-06-10 00:52:16 +02:00
WenFeiYi a316f0b25b HBASE-22284 optimization StringBuilder.append of AbstractMemStore.toString #182 2019-06-09 14:32:38 -07:00
zhangduo 657f263d74 HBASE-22548 Split TestAdmin1 2019-06-09 20:26:13 +08:00
stack 8328d245b2 HBASE-22546 TestRegionServerHostname#testRegionServerHostname fails reliably for me 2019-06-05 20:54:48 -07:00
Daisuke Kobayashi 3139db5b1f HBASE-22160 Add sorting functionality in regionserver web UI for user 
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
 2019-06-05 11:06:41 +09:00
Sakthi afa263d739 HBASE-22536 TestForeignExceptionSerialization fails when run on JDK11 
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 2280c8f4a88350278447515deb0811c42d0bb476)
 2019-06-04 11:29:51 -05:00
zhangduo 09465f48a8 HBASE-22524 Refactor TestReplicationSyncUpTool 2019-06-04 08:19:00 +08:00
meiyi 17ea99c83d HBASE-22513 Admin#getQuota does not work correctly if exceedThrottleQuota is set 2019-06-03 17:51:28 +08:00
Guanghao 14041b39f2 HBASE-22523 Refactor RegionStates#getAssignmentsByTable to make it easy to understand 2019-06-02 21:37:12 +08:00
Josh Elser a584241e6f HBASE-22511 Add some more rs-status paths to RS UI links 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2019-05-31 14:42:59 -04:00
Duo Zhang 6641f11aed HBASE-22502 Purge the logs when we reach the EOF for the last wal file when replication 2019-05-31 15:37:29 +08:00
Guanghao cc127dd325 HBASE-22503 Failed to upgrade to 2.2+ as the global permission which storaged in zk is not right 2019-05-31 09:11:47 +08:00
Andrew Purtell d06c3af980
HBASE-22459 Expose store reader reference count (#248) 2019-05-30 15:08:32 -07:00
Clay Baenziger 89260f4f3e HBASE-22487 getMostLoadedRegions is unused 
Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-05-30 18:34:50 +02:00
Duo Zhang e4057c5555 HBASE-22472 The newly split TestReplicationStatus* tests are flaky 2019-05-30 21:53:43 +08:00
Jingyun Tian 34514ccf0a HBASE-22454 refactor WALSplitter 2019-05-30 10:50:25 +08:00
Guanghao 78319fab87 HBASE-22488 Cleanup the explicit timeout value for test methods 2019-05-29 14:13:15 +08:00
Guanghao b0354644f8 HBASE-22486 Fix flaky test TestLockManager 2019-05-29 10:59:23 +08:00
Guanghao 8829b6f28f HBASE-22485 Fix failed ut TestClusterRestartFailover 2019-05-29 10:59:11 +08:00
Josh Elser 4f05b22880 HBASE-22467 UI fixes to enable Knox proxying 
Closes #261

Signed-off-by: Sean Busbey <busbey@apache.org>
 2019-05-28 16:32:29 -04:00
Artem Ervits e9e680f3d1 HBASE-22469 replace md5 checksum in saveVersion script with sha512 
Signed-off-by: Josh Elser <elserj@apache.org>
 2019-05-28 15:06:37 -04:00
binlijin ca8c4568b7 HBASE-22441 BucketCache NullPointerException in cacheBlock 2019-05-27 17:15:23 +08:00
zhangduo 3e59a547ae HBASE-22473 Split TestSCP 2019-05-27 08:26:26 +08:00
Duo Zhang 63ca0dc001 HBASE-22456 Polish TestSplitTransitionOnCluster 2019-05-26 14:14:28 +08:00
Duo Zhang 73a62c0906 HBASE-22455 Split TestReplicationStatus 2019-05-25 14:45:00 +08:00
Puleya7 cd385b18c4 HBASE-22440. Override getWalGroupsReplicationStatus to avoid NPE 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-05-23 13:41:56 +08:00
binlijin df6df68900 HBASE-22445 Add file info when throw exceptions in HFileReaderImpl 2019-05-22 20:09:50 +08:00
binlijin e3c27d33a7 HBASE-22447 Check refCount before free block in BucketCache 2019-05-22 19:16:48 +08:00
Andrew Purtell 2c55bd9344
HBASE-22449 https everywhere in Maven metadata (#247) 2019-05-21 12:38:42 -07:00
Duo Zhang 5f79378f79 HBASE-22400 Remove the adapter code in async fs implementation for hadoop-2.7.x 2019-05-18 14:07:18 +08:00
stack 64f5857c2d Revert "HBASE-22169 Open region failed cause memory leak" 
This reverts commit 006170654b.
 2019-05-17 16:53:14 -07:00
Bing Xiao 006170654b HBASE-22169 Open region failed cause memory leak 
Signed-off-by: stack <stack@apache.org>
 2019-05-17 15:09:05 -07:00
Sakthi f5486efdfe HBASE-21991 [Addendum] Mark LossCounting as Private 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-05-17 14:40:56 +08:00
Guanghao af9ada68f1 HBASE-22404 Open/Close region request may be executed twice when master restart 2019-05-16 09:23:39 +08:00
Wellington Chevreuil 9a1ed0b9dc
HBASE-21784 Dump replication queue should show list of wal files ordered chronologically 
Change-Id: I18c372406290e2b1e2b5503e2c87adcb9bf6fe91

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-05-14 14:08:26 -07:00
Andrew Purtell cc0b70db1f
HBASE-22377 Provide API to check the existence of a namespace which does not require ADMIN permissions (#225) 
Signed-off-by: Xu Cang <xucang@apache.org>
 2019-05-14 12:51:51 -07:00
zhangduo 912bbab1bf HBASE-21658 Addendum fix infinite wait when there are no meta locations yet 2019-05-14 08:13:02 +08:00
Caroline Zhou e4c4433787
HBASE-22378. HBase Canary fails with TableNotFoundException when table deleted during Canary run 
Signed-off-by: Xu Cang <xucang@apache.org>
 2019-05-13 12:29:12 -07:00
zhangduo 4b4c3ba6e7 HBASE-22365 Region may be opened on two RegionServers 2019-05-13 15:47:44 +08:00
Puleya7 164924c7df
HBASE-22392. Remove extra/useless + 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2019-05-12 21:31:02 +02:00
zhangduo ee1ebac4c2 HBASE-21658 Should get the meta replica number from zk instead of config at client side 2019-05-11 09:31:46 +08:00
Artem Ervits bbfb8ef80c HBASE-21536 Fix completebulkload usage instructions 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-05-10 10:31:41 +02:00
Xu Cang 911c67e251
HBASE-22274 Cell size limit check on append considers cell's previous size 
change
 2019-05-09 22:32:45 -07:00
chenyang 33d97cd040 HBASE-22324:loss a mass of data when the sequenceId of cells greater than Integer.Max 
Signed-off-by: huzheng <openinx@gmail.com>
 2019-05-09 15:44:00 +08:00
Andrew Purtell 91d8aac2ba
HBASE-22375 Promote AccessChecker to LimitedPrivate(Coprocessor) 2019-05-08 18:14:43 -07:00
Yu Li 07c71d630c HBASE-21777 Only log compaction pressure throughput when changed or traced (addendum) 
Fix the findbugs warning
 2019-05-09 01:00:41 +08:00
ramkrish86 5099ef216c HBASE-22072 High read/write intensive regions may cause long crash (#214) 
* HBASE-22072 High read/write intensive regions may cause long crash
recovery

* Make the 'closing' variable as volatile and move the test case to
standlone class
 2019-05-08 15:53:57 +05:30
Bahram Chehrazy 82fd46e04b HBASE-22360 Abort timer doesn't set when abort is called during graceful shutdown process 
Signed-off-by: Sergey Shelukhin <sershe@apache.org>
 2019-05-07 15:50:09 -07:00
Tak Lon (Stephen) Wu a591c60368 HBASE-21777 Only log compaction pressure throughput when changed or traced 
Signed-off-by: Yu Li <liyu@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-05-08 02:40:07 +08:00
z-york 3cc8b36fd3 HBASE-21070 Add Test for SnapshotFileCache for HBase backed by S3 (#209) 
SnapshotFileCache depends on getting the last modified time of the
snapshot directory, however, S3 FileSystem's do not update the
last modified time of the top 'folder' when objects are added/removed.
This commit adds a test for the previously fixed SnapshotFileCache.
 2019-05-06 16:49:34 -07:00
zhangduo 36aef03613 HBASE-22343 Make procedure retry interval configurable in test 2019-05-04 13:27:52 +08:00
zhangduo d507ae9ff6 HBASE-22190 SnapshotFileCache may fail to load the correct snapshot file list when there is an on-going snapshot operation 2019-05-04 13:09:46 +08:00
Sergey Shelukhin 703da6ffbc HBASE-22354 master never sets abortRequested, and thus abort timeout doesn't work for it 
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2019-05-02 18:09:18 -07:00
zhangduo 401cae4016 HBASE-22350 Rewrite TestClientOperationTimeout so we do not timeout when creating table 2019-05-02 17:26:40 +02:00
Peter Somogyi 32462d5690 HBASE-22340 Corrupt KeyValue is silently ignored (#207) 2019-05-02 11:22:05 +02:00
Andrew Purtell b06ad82b9e
HBASE-22301 Consider rolling the WAL if the HDFS write pipeline is slow 2019-05-01 17:42:53 -07:00
Thiruvel Thirumoolan 38c3178c18
HBASE-21883 Enhancements to Major Compaction tool 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-04-30 17:29:25 -07:00
Sakthi 018dadc3c1 HBASE-22054: Space Quota: Compaction is not working for super user in case of NO_WRITES_COMPACTIONS 
Signed-off-by: Josh Elser <elserj@apache.org>
 2019-04-30 13:41:08 -04:00
zhangduo f1a00b9d57 HBASE-22236 AsyncNonMetaRegionLocator should not cache HRegionLocation with null location 2019-04-30 17:18:56 +08:00
Duo Zhang 37c0ab30c7 HBASE-22322 Use special pause for CallQueueTooBigException 2019-04-30 10:07:47 +08:00
chfeng 04b4c9973a
HBASE-21815 Make isTrackingMetrics and getMetrics of ScannerContext public 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-04-29 13:02:58 -07:00
Sakthi c3e490f75c HBASE-22086: Space Quota issue: Deleting snapshot doesn't update the usage of table 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2019-04-29 14:44:51 +08:00
Duo Zhang 940f374a07 HBASE-22317 Support reading from meta replicas 2019-04-28 15:59:11 +08:00
Zach York aa9f36ae53
HBASE-22291 Fix recovery of recovered.edits files under root dir 
When trying to recover a recovered.edits file under the root directory,
it is possible to get a WrongFS exception.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-04-25 18:11:24 -07:00
Sean Busbey 4862a596ef HBASE-22083 move eclipse settings into a profile. 
Signed-off-by: stack <stack@apache.org>

 Conflicts:
	hbase-backup/pom.xml
	hbase-hadoop-compat/pom.xml
	hbase-protocol/pom.xml
 2019-04-25 14:38:38 -05:00
zhangduo 115ab95cac HBASE-22296 Remove TestFromClientSide.testGetStartEndKeysWithRegionReplicas 2019-04-24 14:45:34 +08:00
李小保 1bd5b5cf7b HBASE-22250 The same constants used in many places should be placed in constant classes 
Signed-off-by: stack <stack@apache.org>
 2019-04-23 21:24:46 -07:00
stack 7c0f90adcc HBASE-22298 branch-2.2 nightly fails "[ForOverride] Method annotated @ForOverride must have protected or package-private visibility" 2019-04-23 14:02:37 -07:00
Jan Hentschel c40e6e2339 HBASE-22231 Removed unused and '*' import 2019-04-23 12:53:52 +02:00
Wellington Chevreuil 02fd482630 HBASE-22200 - WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir 
Signed-off-by: Zach York <zyork@apache.org>
 2019-04-22 15:04:08 -07:00
zhangduo 8973885cf4 HBASE-22279 Add a getRegionLocator method in Table/AsyncTable interface 2019-04-22 18:39:53 +08:00
meiyi c015c6cac0 HBASE-22208 Create access checker and expose it in RS 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-04-22 08:14:50 +08:00
zhangduo 393c5b9502 HBASE-22261 Make use of ClusterStatusListener for async client 2019-04-21 22:26:58 +08:00
zhangduo 65fb40d7ee HBASE-22267 Implement client push back for async client 2019-04-21 16:51:55 +08:00
Bahram Chehrazy 22fca940be HBASE-22150 rssStub in HRegionServer is not thread safe and should not directly be used 
Signed-off-by: Sergey Shelukhin <sershe@apache.org>
 2019-04-17 11:28:39 -07:00
Ben Manes cadf321d53
HBASE-15560 W-TinyLFU based BlockCache 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2019-04-17 11:10:55 -07:00
Andrew Purtell f3532e4c26
HBASE-22235 OperationStatus.{SUCCESS|FAILURE|NOT_RUN} are not visible to 3rd party coprocessors 2019-04-16 13:19:56 -07:00
zhanggangxue 9a0daa8cbd HBASE-21257 misspelled words.[occured -> occurred] 2019-04-14 21:36:24 +08:00
Duo Zhang df255f639d HBASE-22207 Fix flakey TestAssignmentManager.testAssignSocketTimeout 2019-04-14 16:53:34 +08:00
Guanghao Zhang 69e3a4eb32 HBASE-22193 Add backoff when region failed open too many times 2019-04-13 11:14:34 +08:00
Jan Hentschel b5b1f86dc7
HBASE-22187 Removed remaining usages of ClusterConnection.clearRegionCache 2019-04-12 09:59:52 +02:00
Duo Zhang 33b66a3981 HBASE-22202 Fix new findbugs issues after we upgrade hbase-thirdparty dependencies 2019-04-12 14:39:25 +08:00
meiyi 61bc0b4741 HBASE-22084 Rename AccessControlLists to PermissionStorage 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2019-04-12 14:22:57 +08:00
Josh Elser 0f155b9ec9 HBASE-22144 Correct MultiRowRangeFilter to work with reverse scans 
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
 2019-04-11 17:41:24 -04:00
Bahram Chechrazy 86483bf961 HBASE-22194 Snapshot unittests fail on Windows due to invalid file path uri 
Signed-off-by: Sergey Shelukhin <sershe@apache.org>
 2019-04-11 11:34:27 -07:00
Jan Hentschel 45ea084e8b
HBASE-22189 Removed remaining usage of StoreFile.getModificationTimeStamp 2019-04-11 12:13:28 +02:00
Duo Zhang cbea2879c4 HBASE-22198 Fix flakey TestAsyncTableGetMultiThreaded 2019-04-11 16:33:39 +08:00
Duo Zhang ae79db959e HBASE-22196 Split TestRestartCluster 2019-04-11 10:58:09 +08:00
huzheng 2e798f87b7 HBASE-22185 RAMQueueEntry#writeToCache should freeBlock if any exception encountered instead of the IOException catch block 2019-04-10 14:06:10 +08:00