Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,088 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

8463 Commits

Author SHA1 Message Date
Bo Cui d492aea3f8
HBASE-21721 FSHLog : reduce write#syncs() times (#2217) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Anoop <anoopsamjohn@apache.org>
 2020-08-08 09:39:48 +05:30
Josh Elser 124af6392c HBASE-24779 Report on the WAL edit buffer usage/limit for replication 
Closes #2193

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-08-07 12:59:17 -04:00
Viraj Jasani 0b604d921a
HBASE-24750 : All ExecutorService should use guava ThreadFactoryBuilder 
Closes #2196

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
 2020-08-07 20:24:36 +05:30
bsglz 485e0d2fa4
HBASE-24694 Support flush a single column family of table (#2179) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-08-07 11:28:53 +01:00
niuyulin f710d2d654
HBASE-24659 Calculate FIXED_OVERHEAD automatically (#2018) 
Co-authored-by: niuyulin <niuyulin@xiaomi.com>

SIgned-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
 2020-08-06 14:50:32 -07:00
Michael Stack 0ae125ac6f HBASE-24817 Allow configuring WALEntry filters on ReplicationSource (#2198) 
Allow specifying base WALEntry filter on construction of
ReplicationSource. Add means of being able to filter WALs by name.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
 Add constructor that allows passing a predicate for filtering *in* WALs
 and a list of filters for filtering *out* WALEntries. The latter was
 hardcoded to filter out system-table WALEntries. The former did not
 exist but we'll need it if Replication takes in more than just the
 default Provider.

Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-08-06 12:35:42 -07:00
Bharath Vissapragada 8003a15835
HBASE-21905 (addendum): Address compaction races in TestFIFOCompactionPolicy (#2203) 
Looped through the test 100 times and it passes. Without the patch it fails
every ~10 runs or so.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2020-08-06 09:33:06 -07:00
sguggilam f8b887b327
HBASE-24823 Port HBASE-22762 Print the delta between phases in the split/merge/compact/flush transaction journals 
Closes #2204

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-08-06 18:50:12 +05:30
Zheng Wang c39cad220d
HBASE-24821 simplify the logic of getRegionInfo in TestFlushFromClient to reduce redundancy code 
Closes #2200

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-08-05 19:59:50 +05:30
Duo Zhang d2f5a5f27b
HBAE-24507 Remove HTableDescriptor and HColumnDescriptor (#2186) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2020-08-04 10:31:42 +08:00
Michael Stack 148c185486 HBASE-24808 skip empty log cleaner delegate class names (WAS => cleaner.CleanerChore: Can NOT create CleanerDelegate= ClassNotFoundException) (#2181) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2020-08-03 13:16:27 -07:00
Michael Stack 21a0b8ea11 HBASE-24767 Change default to false for HBASE-15519 per-user metrics Set hbase.regionserver.user.metrics.enabled default to false; i.e. off. (#2132) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2020-08-03 11:37:54 -07:00
Viraj Jasani 9ec484a031
HBASE-24795 : TestRegionMover2#testWithSplitRegions fix (ADDENDUM) 2020-08-03 20:26:46 +05:30
Viraj Jasani ad7caf754a
HBASE-24795 : RegionMover to deal with unknown region while (un)loading 
* RegionMover to ignore move failures for split/merged regions with ack mode
* Refactor MoveWithAck and MoveWithoutAck as high level classes
* UT for RegionMover gracefully handling split/merged regions while loading regions and throwing failure while loading offline regions

Closes #2172

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
 2020-08-03 17:25:52 +05:30
gkanade 9a1bad84bf
HBASE-24713 RS startup with FSHLog throws NPE after HBASE-21751 (#2125) 
* null check for writer if not initialized yet during syncrunner run

* Revert "null check for writer if not initialized yet during syncrunner run"

This reverts commit 72932ad0df.

* Writer check while trying to attain safe point
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
 2020-08-03 10:04:55 +05:30
Zheng Wang 492cf108a9
HBASE-24704 Make Table Schema easier to view with multiple families 
Closes #2042

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-08-02 23:26:04 +05:30
Sean Busbey 86ebbdd8a2
HBASE-24805 HBaseTestingUtility.getConnection should be threadsafe 
* refactor how we use connection and async connection to rely on their access methods
* refactor initialization and cleanup of the shared connection
* incompatibly change HCTU's Configuration member variable to be final so it can be safely accessed from multiple threads.

Closes #2180

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-08-01 23:19:07 -05:00
Toshihiro Suzuki e22a2d2700
HBASE-24680 Refactor the checkAndMutate code on the server side (#2094) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2020-08-01 21:02:17 +09:00
Anoop Sam John a3f623eea7
HBASE-24695 FSHLog - close the current WAL file in a background thread. (#2168) 
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-08-01 17:27:20 +05:30
Sean Busbey 840a55761b
HBASE-24794 hbase.rowlock.wait.duration should not be <= 0 (#2174) 
if hbase.rowlock.wait.duration is <=0 then log a message and treat it as a value of 1ms.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-30 12:26:12 -05:00
ddupg 1b9269de4d
HBASE-24797 Move log code out of loop 
Closes #2175

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-30 15:15:52 +05:30
Bharath Vissapragada f07f30ae24
HBASE-20226: Parallelize region manifest deletes (#2159) 
We observed this delete call to be a bottleneck for table with lots of
regions. Patch attempts to parallelize them.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2020-07-29 10:59:23 -07:00
Sandeep Pal d65fb87a2e
HBASE-24788: Fix the connection leaks on getting hbase admin from unclosed connection (#2162) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-29 09:09:07 -07:00
WenFeiYi 5f27a009ab
HBASE-24752 NPE/500 accessing webui on master startup 
Closes #2148

Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-29 20:19:29 +05:30
Michael Stack 652f1e664b HBASE-24766 Document Remote Procedure Execution (#2131) 
Add outline of how the mechanism works generally.

Signed-off-by: Anoop Sam John anoopsamjohn@apache.org
Signed-off-by: Viraj Jasani vjasani@apache.org
Signed-off-by: Jan Hentschel jan.hentschel@ultratendency.com
 2020-07-29 07:44:51 -07:00
Michael Stack 345b77a4ca
HBASE-24632 Enable procedure-based log splitting as default in hbase3 Add deprecation of 'classic' zk-based WAL splitter. (#2156) 
Also fix three bugs:

 * We were trying to delete non-empty directory; weren't doing
 accounting for meta WALs where meta had moved off the server
 (successfully)
 * We were deleting split WALs rather than archiving them.
 * We were not handling corrupt files.

Deprecations and removal of tests of old system.

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
 2020-07-29 07:37:28 -07:00
YeChao Chen 5480f350c2
HBASE-24790 Remove unused counter from SplitLogCounters (#2164) 
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-29 17:22:17 +05:30
Duo Zhang 7e6e7a7051
HBASE-24770 Reimplement the Constraints API and revisit the IA annotations on related classes (#2140) 
Signed-off-by: stack <stack@apache.org>
 2020-07-28 09:24:55 +08:00
Wellington Ramos Chevreuil 8c0d7fa5b8
HBASE-24758 Avoid flooding replication source RSes logs when no sinks… (#2118) 
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-27 10:08:13 +01:00
WenFeiYi 975cdf7b88
HBASE-24665 all wal of RegionGroupingProvider together roll (#2021) 
Co-authored-by: wen_yi <liu.wenwen@immomo.com>
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-24 22:54:25 +05:30
Viraj Jasani 09e7ccd42d
HBASE-24757 : ReplicationSink should limit row count in batch mutation based on hbase.rpc.rows.warning.threshold 
Closes #2127

Signed-off-by: stack <stack@apache.org>
 2020-07-24 13:16:02 +05:30
Duo Zhang 0b85729da4
HBASE-24762 Purge protobuf java 2.5.0 dependency (#2128) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
 2020-07-24 11:48:35 +08:00
Guanghao Zhang 32c7012ad7
HBASE-24743 Reject to add a peer which replicate to itself earlier (#2122) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-07-24 08:08:20 +08:00
surbhi 8191fbdd7d HBASE-22146 Removing a namespace-level space quota does not remove policies against contained tables 
Closes #1935

Signed-off-by: Josh Elser <elserj@apache.org>
 2020-07-21 20:12:46 -04:00
Viraj Jasani f35c5eaadd
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses) (#2109) 
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-07-21 20:24:13 +05:30
Viraj Jasani 32796ad24e
Revert "HBASE-24743 Reject to add a peer which replicate to itself earlier (#2071)" 
This reverts commit 6cf013ddc4.

TestReplicationAdmin and TestReplicationShell are broken on branch-2 and master respectively
 2020-07-21 20:17:27 +05:30
Mingliang Liu 3d270ba6f9
HBASE-24696 Include JVM information on Web UI under "Software Attributes" 
Closes #2087

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2020-07-21 16:53:58 +05:30
Viraj Jasani f2c087caeb
Revert "HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses)" 
This reverts commit 16f306b4a1.

TestAdminShell and TestThriftHBaseServiceHandler are failing consistently
 2020-07-21 13:50:27 +05:30
sguggilam 430602aed8
HBASE-24740 Enable journal logging for HBase snapshot operation (#2104) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-07-20 18:11:41 -07:00
Guanghao Zhang 6cf013ddc4
HBASE-24743 Reject to add a peer which replicate to itself earlier (#2071) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-07-21 08:37:56 +08:00
Sanjeet Nishad a857c6d90c
HBASE-24714 Error message is displayed in UI of table's compaction state if any region of that table is not open 
Closes #2047

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-20 21:40:23 +05:30
Zheng Wang 9b32943697
HBASE-24709 Support MoveCostFunction use a lower multiplier in offpeak hours 
Closes #2099

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-07-19 20:09:12 +05:30
Michael Stack 9b02a26a1d HBASE-24698 Turn OFF Canary WebUI as default (#2037) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2020-07-17 15:52:52 -07:00
Wellington Ramos Chevreuil bd42c75cac
HBASE-24747 Log an ERROR if HBaseSaslRpcServer initialisation fails with an uncaught exception (#2079) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2020-07-17 18:50:03 +01:00
Mohammad Arshad 1781185b7b
HBASE-24675: On Master restart all servers are assigned to default rsgroup 
Closes #2053

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-17 22:38:19 +05:30
Zheng Wang 6ae2fdd6b2
HBASE-24746 The sort icons overlap the col name in master UI 
Closes #2078

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-17 21:47:17 +05:30
Viraj Jasani 16f306b4a1
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses) 
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-07-17 19:16:22 +05:30
Lars a4e5ccc798 HBASE-24742 Improve performance of SKIP vs SEEK logic. 2020-07-16 13:17:05 -07:00
huaxiangsun 2505c7760d
HBASE-24581 Skip compaction request/check for replica regions at the early stage. (#1986) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2020-07-14 15:49:20 -07:00
huaxiangsun 1360bee7f9
HBASE-24705 MetaFixer#fixHoles() does not include the case for read replicas (i.e, replica regions are not created) (#2062) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-14 15:48:19 -07:00
BukrosSzabolcs 3e709c6f53
HBASE-24720: Meta replicas not cleaned when disabled (#2057) 
- make sure to always clean up excess meta replicas not just when their
number get decreased
- make sure NotServingRegionException is handled properly even when
wrapped
- add test

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
 2020-07-14 18:24:55 +02:00
Mohammad Arshad 70cfe2525e
HBASE-24721: rename_rsgroup overwriting the existing rsgroup 
Closes #2059

Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-14 21:30:23 +05:30
bsglz 3bd54217a0
HBASE-24664 Some changing of split region by overall region size rath… (#2011) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-07-13 10:00:48 +01:00
Geoffrey Jacoby af1cc2fc44
HBASE-23744 - FastPathBalancedQueueRpcExecutor should enforce queue length of 0 
Closes #1094

Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-13 12:15:10 +05:30
bsglz 724f0478ed
HBASE-24404 Support flush a single column family of region (#2032) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-07-10 18:13:50 +01:00
Zheng Wang b2321b3583
HBASE-24586 Add table level locality in table.jsp 
Closes #1926

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-10 21:59:59 +05:30
Viraj Jasani f0bc33122e
HBASE-24712 : Remove flakes from TestMasterNoCluster#testStopDuringStart (#2045) 
Forward-port from 91da188d78
 2020-07-10 20:41:03 +05:30
Joseph295 67dfbe0406
HBASE-24663 Add procedure process time statistics UI (#2035) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-07-10 15:03:52 +08:00
niuyulin 5de1dff8f6
HBASE-24653 Show snapshot owner on Master WebUI (#2016) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-07-10 15:03:05 +08:00
XinSun 1d0e285a6f
HBASE-24489 Rewrite TestClusterRestartFailover.test since namespace table is gone on on master (#2025) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-07-10 10:03:57 +08:00
Wellington Chevreuil 7b92510bb4 Revert "HBASE-21596 Delete for a specific cell version can bring back version… (#2009)" 
This reverts commit 1db89773e6.
 2020-07-09 15:55:57 +01:00
Wellington Ramos Chevreuil 1db89773e6
HBASE-21596 Delete for a specific cell version can bring back version… (#2009) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-08 15:17:14 +01:00
huaxiangsun 90f4ff7d7c
HBASE-24688 AssignRegionHandler uses EventType.M_RS_CLOSE_META instead of EventType.M_RS_OPEN_META for meta region (#2029) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-07 09:58:25 -07:00
Toshihiro Suzuki 34e807a8b5
HBASE-24650 Change the return types of the new CheckAndMutate methods introduced in HBASE-8458 (#1991) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-07-07 12:49:20 +09:00
WenFeiYi 287f29818f
HBASE-24546 CloneSnapshotProcedure unlimited retry (#1900) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-07-06 13:45:19 +08:00
WenFeiYi 44691ce23c
HBASE-24578 [WAL] Add a parameter to config RingBufferEventHandler's SyncFuture count (#1923) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2020-07-06 11:24:09 +08:00
chenglei a7a0e1a596
HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length. (#1970) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-07-04 21:00:35 +08:00
Baiqiang Zhao afe2eac7b6
HBASE-24560 Add a new option of designatedfile in RegionMover 
Closes #1901

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-07-01 20:26:51 +05:30
xincunSong 9ad16aa376
HBASE-24624 Optimize table.jsp code (#1963) 
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
 2020-07-01 08:49:51 +08:00
stack 2c43f6cde7 HBASE-24661 TestHeapSize.testSizes failure 
Revert "HBASE-24648 Remove the legacy 'forceSplit' related code at region server side (#1990)"

This reverts commit bfec964aef.
 2020-06-30 15:57:49 -07:00
pengmq1 be57e40f36
HBASE-24289 Heterogeneous Storage for Date Tiered Compaction (#1730) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-30 15:10:04 +08:00
stack bfec964aef HBASE-24648 Remove the legacy 'forceSplit' related code at region server side (#1990) 
Addendum to fix TestHeapSize
 2020-06-29 14:20:39 -07:00
Duo Zhang 37b863bd0b
HBASE-24648 Remove the legacy 'forceSplit' related code at region server side (#1990) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-29 22:50:42 +08:00
Duo Zhang 46bf8944fd
HBASE-24646 Set the log level for ScheduledChore to INFO in HBTU (#1987) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
 2020-06-28 23:26:05 +08:00
Viraj Jasani 47742b2083
HBASE-24588 : createTableBegsSplit should consider tuning mergeEnabled for table (ADDENDUM) (#1988) 2020-06-27 19:09:19 +05:30
Duo Zhang af49b87367 HBASE-24635 Addendum make test more robust 2020-06-27 11:29:43 +08:00
Duo Zhang 16a25b74db
HBASE-24635 Split TestMetaWithReplicas (#1980) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-27 10:36:07 +08:00
Viraj Jasani 51ad588565
HBASE-24588 : normalization log format (ADDENDUM) 2020-06-26 23:09:37 +05:30
Viraj Jasani 71af97cd05
HBASE-24588 : Submit task for NormalizationPlan (#1933) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2020-06-26 22:08:52 +05:30
bsglz c0461207ee
HBASE-24382 Flush partial stores of region filtered by seqId when arc… (#1737) 
* HBASE-24382 Flush partial stores of region filtered by seqId when archive wal due to too many wals

* fix checkstyle and javadoc issue

* fix javadoc issues

* move the geting of stores to HRegion, since it should not be part of FlushPolicy, and comment fix

* fix checkstyle issue

* add some comment

* remove the forceFlushAllStores since we can use families to determine how to select stores to flush

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@duboce.net>
 2020-06-25 22:03:18 -07:00
Bharath Vissapragada 84e246f9b1
HBASE-24603: Make Zookeeper sync() call synchronous (#1945) 
Writing a test for this is tricky. There is enough coverage for
functional tests. Only concern is performance, but there is enough
logging for it to detect timed out/badly performing sync calls.

Additionally, this patch decouples the ZK event processing into it's
own thread rather than doing it in the EventThread's context. That
avoids deadlocks and stalls of the event thread.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-25 13:27:38 -07:00
Michael Stack e54e3afc5c HBASE-24616 Remove BoundedRecoveredHFilesOutputSink dependency on a TableDescriptor (#1955) 
Purge query Master for table descriptors; make do w/ generic options.

Logging cleanup.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
 Undo fetching Table Descriptor. Not reliably available at recovery time.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-25 12:10:17 -07:00
BukrosSzabolcs 8cdb2cca44
HBASE-24562: Stabilize master startup with meta replicas enabled (#1903) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
 2020-06-24 18:38:36 +01:00
huaxiangsun 82fad5d789
HBASE-24552 Replica region needs to check if primary region directory exists at file system in TransitRegionStateProcedure (#1924) 
Signed-off-by:  stack <stack@apache.org>
 2020-06-24 09:43:12 -07:00
Duo Zhang 22103067b6
HBASE-24609 Move MetaTableAccessor out of hbase-client (#1943) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-24 21:20:35 +08:00
XinSun 4b89ee24f3
HBASE-24431 RSGroupInfo add configuration map to store something extra (#1782) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-24 18:46:09 +08:00
xincunSong d46adc7cc8
HBASE-24605 Break long region names in the web UI (#1942) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
 2020-06-23 10:03:18 +08:00
Viraj Jasani 5a79a1c83c
HBASE-24102 : Undo visibility change for RegionMover fields (ADDENDUM) 2020-06-22 22:44:05 +05:30
Duo Zhang 8f1353b447 Revert "HBASE-24403 FsDelegationToken Should Cache Token After Acquired A New One (#1743)" 
This reverts commit 7b396e9b8c.
 2020-06-21 23:43:38 +08:00
Viraj Jasani 73ab216414
HBASE-24594 : Separate resource allocation for TestReplicationStatusSink (#1934) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-21 16:11:32 +05:30
XinSun dfaba965c9
HBASE-24591 get_table_rsgroup ignored the existence of rsgroup config for namespace (#1925) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-21 17:54:39 +08:00
Viraj Jasani 3c31981179
HBASE-24380 : Provide WAL splitting journal logging (#1860) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2020-06-19 16:25:03 -07:00
stack 537a3caccd HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912) 
Addendum 2 fix compile error.
 2020-06-18 08:49:39 -07:00
stack 489f07d945 HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912) 
Addendum; minor log edits
 2020-06-18 08:40:11 -07:00
Michael Stack aadd2bb1a1 HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
 Don't register as a chore on construction if no coordination state
 manager instance (there is no instance when procv2 WAL splitter).

hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Edit logs.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionRemoteProcedureBase.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALRemoteProcedure.java
 Add proc name rather than rely on default behavior. Add detail to the
 toString.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 Factoring

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
 Print the maxLogs... we don't do it any where.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
 Utility method to strip prefix from wal path.

Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: tianjingyun <tianjy@apache.org>
 2020-06-18 08:38:59 -07:00
Michael Stack 6a576696dc HBASE-24577 Doc WALSplitter classes (#1913) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-17 15:03:33 -07:00
Viraj Jasani 192daded61
HBASE-24518 : waitForNamespaceOnline() should return false if any region is offline (#1869) 
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
 2020-06-17 19:44:36 +05:30
ramkrish86 b17ba7b81d
HBASE-24205 - Avoid metric collection for flushes and compactions (#1918) 
* HBASE-24205 - Avoid metric collection for flushes and compactions

* Use the existing matcher.isUserScan() to decide the scan type

* Added a comment saying when we track the metrics
 2020-06-17 16:47:21 +05:30
ramkrish86 510aad3125
HBASE-24205 Create metric to know the number of reads that happens fr… (#1552) 
* HBASE-24205 Create metric to know the number of reads that happens from memstore

* Fix checkstyles and whitespaces

* Checkstyl, whitespace and javadoc

* Fixed review comments

* Fix unused imports

* Rebase with latest commit

* Adding the table vs store metric by consolidating

* Combine get and scan metrics and make all relevant changes

* Track for full row and then increment either memstore or file read
metric

* TestMetricsStore test fix

* Only increment the memstore metric if all cells are from memstore, if
not treat as mixed reads

* Remove metricsstore and aggregate at region level

* Addresses review comments-metric name updated everywhere

* Metric name change

* Review comment changes

Co-authored-by: Ramkrishna <ramkrishna@apache.org>
Signed-off by:Anoop Sam John<anoopsamjohn@gmail.com>
Signed-off by:Viraj Jasani<virajjasani@apache.org>
 2020-06-16 18:30:20 +05:30
xincunSong 3558ee0c3b
HBASE-24478 The regionInfo parameter for MasterProcedureScheduler#wai… (#1819) 
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: clarax <clarax98007@gmail.com>
 2020-06-16 17:34:07 +08:00
Sandeep Pal db4d539190
HBASE-24015: Test for Assign and Unassign of Regions on RegionServer on failure (#1898) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-16 12:47:52 +05:30
Bharath Vissapragada 2567d15218 HBASE-24564: Make RS abort call idempotent. 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-15 22:45:35 -07:00
Duo Zhang d0f190bdf0
HBASE-24517 Addendum add more comments (#1882) 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-15 10:43:57 +08:00
bsglz b83d38fb94
HBASE-24038 Add a metric to show the locality of ssd in table.jsp (#1337) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-14 22:46:18 +05:30
zhaoym6 21fe873eba HBASE-23195 FSDataInputStreamWrapper unbuffer can NOT invoke the classes that NOT implements CanUnbuffer but its parents class implements CanUnbuffer 
Closes #746

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2020-06-12 13:01:02 -04:00
Michael Stack 9fbf1f30c9 HBASE-24545 Add backoff to SCP check on WAL split completion (#1891) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-12 07:58:34 -07:00
sguggilam 1dac9f69c4
HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master (#1885) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-12 16:17:59 +05:30
Toshihiro Suzuki 6b7a93d10c
HBASE-24529 hbase.rs.evictblocksonclose is not honored when removing compacted files and closing the storefiles (#1881) 
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
 2020-06-12 18:59:03 +09:00
Anoop Sam John 4005e42a62
HBASE-24189 WALSplit recreates region dirs for deleted table with recovered edits data. 2020-06-11 21:27:21 +05:30
Viraj Jasani 9b2f961132
HBASE-24532 : Execute region plans with throttle should return succeeded plans (#1884) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-06-11 18:17:59 +05:30
Toshihiro Suzuki efd5a962e5 HBASE-8458 Support for batch version of checkAndMutate() 
Closes #1648

Signed-off-by: Josh Elser <elserj@apache.org>
 2020-06-10 22:34:36 -04:00
sguggilam 1eb159b455
HBASE-24511 Ability to configure timeout between RPC retry to RS from master (#1861) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-10 12:53:08 +05:30
Vico.Wu 7b396e9b8c
HBASE-24403 FsDelegationToken Should Cache Token After Acquired A New One (#1743) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-06-10 08:46:58 +08:00
xincunSong 15ddded26b
HBASE-24441 CacheConfig details logged at Store open is not really u… (#1801) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
 2020-06-09 14:18:21 +08:00
xincunSong eee06c2d7e
HBASE-24468 Add region info when log meessages in HStore. (#1803) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-09 14:12:44 +08:00
Duo Zhang 3d50e73871
HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may cause SCP to accidentally skip assigning a region (#1865) 
Signed-off-by: Michael Stack <stack@apache.org>
 2020-06-09 11:07:16 +08:00
WenFeiYi 2131456e54 HBASE-24444 Should shutdown mini cluster after class in TestMetaAssignmentWithStopMaster (#1802) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-09 08:16:20 +08:00
Guanghao Zhang a4aa5384f1 Revert "HBASE-24444 Should shutdown mini cluster after class in TestMetaAssignmentWithStopMaster (#1802)" 
This reverts commit 3b3bba1b64.
 2020-06-09 08:15:51 +08:00
WenFeiYi 3b3bba1b64
HBASE-24444 Should shutdown mini cluster after class in TestMetaAssignmentWithStopMaster (#1802) 
* shutdown mini cluster after class

* fix

Co-authored-by: wen_yi <liu.wenwen@immomo.com>
 2020-06-09 08:14:18 +08:00
Duo Zhang 2dd34b3f01
HBASE-24517 AssignmentManager.start should add meta region to ServerStateNode (#1866) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
 2020-06-09 08:14:00 +08:00
huaxiangsun 71fae116c3
HBASE-23202 ExportSnapshot (import) will fail if copying files to root directory takes longer than cleaner TTL (#1874) 
Co-authored-by: Guangxu Cheng <guangxucheng@gmail.com>

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
 2020-06-08 14:57:48 -07:00
huaxiangsun 5042b591f8
Revert "HBASE-23202 ExportSnapshot (import) will fail if copying files to root directory takes longer than cleaner TTL (#1791)" (#1873) 
This reverts commit f862f3d9b5.
 2020-06-08 14:51:59 -07:00
huaxiangsun f862f3d9b5
HBASE-23202 ExportSnapshot (import) will fail if copying files to root directory takes longer than cleaner TTL (#1791) 
Co-authored-by: Huaxiang Sun <huaxiangsun@apache.org>

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
 2020-06-08 14:48:21 -07:00
gkanade 920d60f43b
HBASE-24208 Remove RS entry from zk draining servers node after RS been stopped (#1841) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-08 22:47:00 +05:30
Duo Zhang 754ac1d06d
HBASE-24506 async client deadlock (#1858) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-06-08 09:38:58 +08:00
Toshihiro Suzuki 022dd9687f
HBASE-24515 batch Increment/Append fails when retrying the RPC (#1864) 
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
 2020-06-08 09:51:21 +09:00
bsglz 66452afc09
HBASE-24496 The tag of Base Stats not actived by default in table.jsp (#1837) 
Signed-off-by: clarax
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-06 19:15:37 +05:30
Duo Zhang 16116fa35e
HBASE-24510 Remove HBaseTestCase and GenericTestUtils (#1859) 
Signed-off-by: Michael Stack <stack@apache.org>
 2020-06-06 19:06:11 +08:00
Duo Zhang c91829bb41
HBASE-24491 Remove HRegionInfo (#1830) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2020-06-05 22:19:01 +08:00
Duo Zhang 2c2b1f0174
HBASE-24500 The behavior of RegionInfoBuilder.newBuilder(RegionInfo) is strange (#1850) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-05 15:19:04 +08:00
Duo Zhang b2ec4c1ea0
HBASE-24505 Reimplement Hbck.setRegionStateInMeta (#1852) 
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
 2020-06-05 08:48:39 +08:00
Viraj Jasani 61a7468df7
HBASE-24456 : Create ImmutableScan and use it for CustomizedScanInfoBuilder (#1818) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-04 16:25:47 +05:30
XinSun 59031ab661
HBASE-24359 Optionally ignore edits for deleted CFs for replication (#1722) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-04 17:52:52 +08:00
Duo Zhang 08c97d3a87 Revert "HBASE-24500 The behavior of RegionInfoBuilder.newBuilder(RegionInfo) is strange (#1840)" 
This reverts commit ee6938abda.
 2020-06-04 12:17:03 +08:00
Duo Zhang ee6938abda
HBASE-24500 The behavior of RegionInfoBuilder.newBuilder(RegionInfo) is strange (#1840) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2020-06-04 11:06:41 +08:00
Bharath Vissapragada 60c9ae58fb
HBASE-21905: [FIFOCompaction] Don't compact already inflight store files (#1829) 
This one surfaced as a flake test but turns out to be a legit bug
in FIFOCompaction code. FifoCompaction does not check if an empty
store file is already being compacted by an in-flight compaction
request and still enqueues. It can potentially race with a running
compaction (as in this test case, see jira for the exact exception).

Fixes the edge case and cleans up the test code a bit.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2020-06-03 16:37:22 -07:00
Viraj Jasani 8de8c44029
HBASE-24492 : Remove infinite loop from ProtobufLogReader#readNext (#1831) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-03 13:41:15 +05:30
niuyulin f71f1cdfa0
HBASE-24412 Canary support check only one column family per RegionTask (#1817) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-03 09:15:10 +08:00
XinSun 901187ae2f
HBASE-22738 Fallback to default group to choose RS when there are no RS in current group (#1789) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-03 08:26:05 +08:00
Nick Dimiduk cb5c0d2782 HBASE-24418 Consolidate Normalizer implementations 
Simplify our Normalizer story to have just a single, configurable
implementation.

* fold the features of `MergeNormalizer` into
  `SimpleRegionNormalizer`, removing the intermediate abstract class.
* configuration keys for merge-only features now share a common
  structure.
* add configuration to selectively disable normalizer split/merge
  operations.
* `RegionNormalizer` now extends `Configurable` instead of creating a
  new instance of `HBaseConfiguration` or snooping one off of other
  fields.
* avoid the extra RPCs by using `MasterServices` instead of
  `MasterRpcServices`.
* boost test coverage of all the various flags and feature
  combinations.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
 2020-06-02 11:38:39 -07:00
Wellington Ramos Chevreuil e5345b3a7c
HBASE-21406 "status 'replication'" should not show SINK if the cluste… (#1761) 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2020-06-02 09:27:16 +01:00
Duo Zhang bad2d4e409
HBASE-24474 Rename LocalRegion to MasterRegion (#1811) 
Signed-off-by: Michael Stack <stack@apache.org>
 2020-06-02 10:37:43 +08:00
Bharath Vissapragada f5b90fcc44
HBASE-24479: Deflake TestCompaction#testStopStartCompaction (#1820) 
Polling of active compaction count is racy. Tightened the asserts
to be more reliable.

Signed-off-by: Reid Chan <reidchan@apache.org>
 2020-05-31 23:54:17 -07:00
Bharath Vissapragada 4f49a96258
HBASE-24477: Move ConfigurationObserver and related classes to hbase-common (#1815) 
This utility is useful for any module that wants to detect
dynamic config changes. Having it to hbase-common makes it
accessible to all the other modules.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-31 11:18:16 -07:00
bsglz b4a4debdd9
HBASE-24475 Clean up the master thread name getting in SplitLogManager and AssignmentManager (#1810) 2020-05-30 22:23:04 -07:00
jacob-leblanc c120260bce
HBASE-24454 - Read ioErrorStartTime to local temporary variable to avoid issue when it is set to -1 between greater than zero check and calculation of error duration (#1816) 
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
 2020-05-30 21:06:40 -07:00
Duo Zhang 36a6ef9cf9
HBASE-24471 The way we bootstrap meta table is confusing (#1806) 
Signed-off-by: Michael Stack <stack@apache.org>
 2020-05-30 16:03:52 +08:00
Viraj Jasani f0f9a20cd7
HBASE-23941 : FilterBy operator support in get_slowlog_responses API (#1793) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2020-05-30 12:59:59 +05:30
Michael Stack bda2094ae5 HBASE-22287 inifinite retries on failed server in RSProcedureDispatcher (#1800) 
Adds backoff in place of retry every 100ms.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-05-29 10:04:02 -07:00
xincunSong da2e03bb3b
HBASE-24470 : Add store file info when log meessages in HStore. (#1804) 
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-29 18:58:57 +05:30