Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,922 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

8371 Commits

Author SHA1 Message Date
stack 537a3caccd HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912) 
Addendum 2 fix compile error.
 2020-06-18 08:49:39 -07:00
stack 489f07d945 HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912) 
Addendum; minor log edits
 2020-06-18 08:40:11 -07:00
Michael Stack aadd2bb1a1 HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
 Don't register as a chore on construction if no coordination state
 manager instance (there is no instance when procv2 WAL splitter).

hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Edit logs.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionRemoteProcedureBase.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALRemoteProcedure.java
 Add proc name rather than rely on default behavior. Add detail to the
 toString.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 Factoring

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
 Print the maxLogs... we don't do it any where.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
 Utility method to strip prefix from wal path.

Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: tianjingyun <tianjy@apache.org>
 2020-06-18 08:38:59 -07:00
Michael Stack 6a576696dc HBASE-24577 Doc WALSplitter classes (#1913) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-17 15:03:33 -07:00
Viraj Jasani 192daded61
HBASE-24518 : waitForNamespaceOnline() should return false if any region is offline (#1869) 
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
 2020-06-17 19:44:36 +05:30
ramkrish86 b17ba7b81d
HBASE-24205 - Avoid metric collection for flushes and compactions (#1918) 
* HBASE-24205 - Avoid metric collection for flushes and compactions

* Use the existing matcher.isUserScan() to decide the scan type

* Added a comment saying when we track the metrics
 2020-06-17 16:47:21 +05:30
ramkrish86 510aad3125
HBASE-24205 Create metric to know the number of reads that happens fr… (#1552) 
* HBASE-24205 Create metric to know the number of reads that happens from memstore

* Fix checkstyles and whitespaces

* Checkstyl, whitespace and javadoc

* Fixed review comments

* Fix unused imports

* Rebase with latest commit

* Adding the table vs store metric by consolidating

* Combine get and scan metrics and make all relevant changes

* Track for full row and then increment either memstore or file read
metric

* TestMetricsStore test fix

* Only increment the memstore metric if all cells are from memstore, if
not treat as mixed reads

* Remove metricsstore and aggregate at region level

* Addresses review comments-metric name updated everywhere

* Metric name change

* Review comment changes

Co-authored-by: Ramkrishna <ramkrishna@apache.org>
Signed-off by:Anoop Sam John<anoopsamjohn@gmail.com>
Signed-off by:Viraj Jasani<virajjasani@apache.org>
 2020-06-16 18:30:20 +05:30
xincunSong 3558ee0c3b
HBASE-24478 The regionInfo parameter for MasterProcedureScheduler#wai… (#1819) 
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: clarax <clarax98007@gmail.com>
 2020-06-16 17:34:07 +08:00
Sandeep Pal db4d539190
HBASE-24015: Test for Assign and Unassign of Regions on RegionServer on failure (#1898) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-16 12:47:52 +05:30
Bharath Vissapragada 2567d15218 HBASE-24564: Make RS abort call idempotent. 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-15 22:45:35 -07:00
Duo Zhang d0f190bdf0
HBASE-24517 Addendum add more comments (#1882) 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-15 10:43:57 +08:00
bsglz b83d38fb94
HBASE-24038 Add a metric to show the locality of ssd in table.jsp (#1337) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-14 22:46:18 +05:30
zhaoym6 21fe873eba HBASE-23195 FSDataInputStreamWrapper unbuffer can NOT invoke the classes that NOT implements CanUnbuffer but its parents class implements CanUnbuffer 
Closes #746

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2020-06-12 13:01:02 -04:00
Michael Stack 9fbf1f30c9 HBASE-24545 Add backoff to SCP check on WAL split completion (#1891) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-12 07:58:34 -07:00
sguggilam 1dac9f69c4
HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master (#1885) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-12 16:17:59 +05:30
Toshihiro Suzuki 6b7a93d10c
HBASE-24529 hbase.rs.evictblocksonclose is not honored when removing compacted files and closing the storefiles (#1881) 
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
 2020-06-12 18:59:03 +09:00
Anoop Sam John 4005e42a62
HBASE-24189 WALSplit recreates region dirs for deleted table with recovered edits data. 2020-06-11 21:27:21 +05:30
Viraj Jasani 9b2f961132
HBASE-24532 : Execute region plans with throttle should return succeeded plans (#1884) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-06-11 18:17:59 +05:30
Toshihiro Suzuki efd5a962e5 HBASE-8458 Support for batch version of checkAndMutate() 
Closes #1648

Signed-off-by: Josh Elser <elserj@apache.org>
 2020-06-10 22:34:36 -04:00
sguggilam 1eb159b455
HBASE-24511 Ability to configure timeout between RPC retry to RS from master (#1861) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-10 12:53:08 +05:30
Vico.Wu 7b396e9b8c
HBASE-24403 FsDelegationToken Should Cache Token After Acquired A New One (#1743) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-06-10 08:46:58 +08:00
xincunSong 15ddded26b
HBASE-24441 CacheConfig details logged at Store open is not really u… (#1801) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
 2020-06-09 14:18:21 +08:00
xincunSong eee06c2d7e
HBASE-24468 Add region info when log meessages in HStore. (#1803) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-09 14:12:44 +08:00
Duo Zhang 3d50e73871
HBASE-24117 Shutdown AssignmentManager before ProcedureExecutor may cause SCP to accidentally skip assigning a region (#1865) 
Signed-off-by: Michael Stack <stack@apache.org>
 2020-06-09 11:07:16 +08:00
WenFeiYi 2131456e54 HBASE-24444 Should shutdown mini cluster after class in TestMetaAssignmentWithStopMaster (#1802) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-09 08:16:20 +08:00
Guanghao Zhang a4aa5384f1 Revert "HBASE-24444 Should shutdown mini cluster after class in TestMetaAssignmentWithStopMaster (#1802)" 
This reverts commit 3b3bba1b64.
 2020-06-09 08:15:51 +08:00
WenFeiYi 3b3bba1b64
HBASE-24444 Should shutdown mini cluster after class in TestMetaAssignmentWithStopMaster (#1802) 
* shutdown mini cluster after class

* fix

Co-authored-by: wen_yi <liu.wenwen@immomo.com>
 2020-06-09 08:14:18 +08:00
Duo Zhang 2dd34b3f01
HBASE-24517 AssignmentManager.start should add meta region to ServerStateNode (#1866) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
 2020-06-09 08:14:00 +08:00
huaxiangsun 71fae116c3
HBASE-23202 ExportSnapshot (import) will fail if copying files to root directory takes longer than cleaner TTL (#1874) 
Co-authored-by: Guangxu Cheng <guangxucheng@gmail.com>

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
 2020-06-08 14:57:48 -07:00
huaxiangsun 5042b591f8
Revert "HBASE-23202 ExportSnapshot (import) will fail if copying files to root directory takes longer than cleaner TTL (#1791)" (#1873) 
This reverts commit f862f3d9b5.
 2020-06-08 14:51:59 -07:00
huaxiangsun f862f3d9b5
HBASE-23202 ExportSnapshot (import) will fail if copying files to root directory takes longer than cleaner TTL (#1791) 
Co-authored-by: Huaxiang Sun <huaxiangsun@apache.org>

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Zach York <zyork@apache.org>
 2020-06-08 14:48:21 -07:00
gkanade 920d60f43b
HBASE-24208 Remove RS entry from zk draining servers node after RS been stopped (#1841) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-08 22:47:00 +05:30
Duo Zhang 754ac1d06d
HBASE-24506 async client deadlock (#1858) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-06-08 09:38:58 +08:00
Toshihiro Suzuki 022dd9687f
HBASE-24515 batch Increment/Append fails when retrying the RPC (#1864) 
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
 2020-06-08 09:51:21 +09:00
bsglz 66452afc09
HBASE-24496 The tag of Base Stats not actived by default in table.jsp (#1837) 
Signed-off-by: clarax
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-06-06 19:15:37 +05:30
Duo Zhang 16116fa35e
HBASE-24510 Remove HBaseTestCase and GenericTestUtils (#1859) 
Signed-off-by: Michael Stack <stack@apache.org>
 2020-06-06 19:06:11 +08:00
Duo Zhang c91829bb41
HBASE-24491 Remove HRegionInfo (#1830) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2020-06-05 22:19:01 +08:00
Duo Zhang 2c2b1f0174
HBASE-24500 The behavior of RegionInfoBuilder.newBuilder(RegionInfo) is strange (#1850) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-05 15:19:04 +08:00
Duo Zhang b2ec4c1ea0
HBASE-24505 Reimplement Hbck.setRegionStateInMeta (#1852) 
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
 2020-06-05 08:48:39 +08:00
Viraj Jasani 61a7468df7
HBASE-24456 : Create ImmutableScan and use it for CustomizedScanInfoBuilder (#1818) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-04 16:25:47 +05:30
XinSun 59031ab661
HBASE-24359 Optionally ignore edits for deleted CFs for replication (#1722) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-04 17:52:52 +08:00
Duo Zhang 08c97d3a87 Revert "HBASE-24500 The behavior of RegionInfoBuilder.newBuilder(RegionInfo) is strange (#1840)" 
This reverts commit ee6938abda.
 2020-06-04 12:17:03 +08:00
Duo Zhang ee6938abda
HBASE-24500 The behavior of RegionInfoBuilder.newBuilder(RegionInfo) is strange (#1840) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
 2020-06-04 11:06:41 +08:00
Bharath Vissapragada 60c9ae58fb
HBASE-21905: [FIFOCompaction] Don't compact already inflight store files (#1829) 
This one surfaced as a flake test but turns out to be a legit bug
in FIFOCompaction code. FifoCompaction does not check if an empty
store file is already being compacted by an in-flight compaction
request and still enqueues. It can potentially race with a running
compaction (as in this test case, see jira for the exact exception).

Fixes the edge case and cleans up the test code a bit.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2020-06-03 16:37:22 -07:00
Viraj Jasani 8de8c44029
HBASE-24492 : Remove infinite loop from ProtobufLogReader#readNext (#1831) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-06-03 13:41:15 +05:30
niuyulin f71f1cdfa0
HBASE-24412 Canary support check only one column family per RegionTask (#1817) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-03 09:15:10 +08:00
XinSun 901187ae2f
HBASE-22738 Fallback to default group to choose RS when there are no RS in current group (#1789) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-06-03 08:26:05 +08:00
Nick Dimiduk cb5c0d2782 HBASE-24418 Consolidate Normalizer implementations 
Simplify our Normalizer story to have just a single, configurable
implementation.

* fold the features of `MergeNormalizer` into
  `SimpleRegionNormalizer`, removing the intermediate abstract class.
* configuration keys for merge-only features now share a common
  structure.
* add configuration to selectively disable normalizer split/merge
  operations.
* `RegionNormalizer` now extends `Configurable` instead of creating a
  new instance of `HBaseConfiguration` or snooping one off of other
  fields.
* avoid the extra RPCs by using `MasterServices` instead of
  `MasterRpcServices`.
* boost test coverage of all the various flags and feature
  combinations.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
 2020-06-02 11:38:39 -07:00
Wellington Ramos Chevreuil e5345b3a7c
HBASE-21406 "status 'replication'" should not show SINK if the cluste… (#1761) 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
 2020-06-02 09:27:16 +01:00
Duo Zhang bad2d4e409
HBASE-24474 Rename LocalRegion to MasterRegion (#1811) 
Signed-off-by: Michael Stack <stack@apache.org>
 2020-06-02 10:37:43 +08:00
Bharath Vissapragada f5b90fcc44
HBASE-24479: Deflake TestCompaction#testStopStartCompaction (#1820) 
Polling of active compaction count is racy. Tightened the asserts
to be more reliable.

Signed-off-by: Reid Chan <reidchan@apache.org>
 2020-05-31 23:54:17 -07:00
Bharath Vissapragada 4f49a96258
HBASE-24477: Move ConfigurationObserver and related classes to hbase-common (#1815) 
This utility is useful for any module that wants to detect
dynamic config changes. Having it to hbase-common makes it
accessible to all the other modules.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-31 11:18:16 -07:00
bsglz b4a4debdd9
HBASE-24475 Clean up the master thread name getting in SplitLogManager and AssignmentManager (#1810) 2020-05-30 22:23:04 -07:00
jacob-leblanc c120260bce
HBASE-24454 - Read ioErrorStartTime to local temporary variable to avoid issue when it is set to -1 between greater than zero check and calculation of error duration (#1816) 
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
 2020-05-30 21:06:40 -07:00
Duo Zhang 36a6ef9cf9
HBASE-24471 The way we bootstrap meta table is confusing (#1806) 
Signed-off-by: Michael Stack <stack@apache.org>
 2020-05-30 16:03:52 +08:00
Viraj Jasani f0f9a20cd7
HBASE-23941 : FilterBy operator support in get_slowlog_responses API (#1793) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
 2020-05-30 12:59:59 +05:30
Michael Stack bda2094ae5 HBASE-22287 inifinite retries on failed server in RSProcedureDispatcher (#1800) 
Adds backoff in place of retry every 100ms.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-05-29 10:04:02 -07:00
xincunSong da2e03bb3b
HBASE-24470 : Add store file info when log meessages in HStore. (#1804) 
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-29 18:58:57 +05:30
Viraj Jasani bf869def43
HBASE-24428 : Fix for testSplitCompactWithPriority (ADDENDUM) 2020-05-29 18:15:40 +05:30
bsglz 7b7e25a50b
HBASE-24451 Remove the HasThread because the related bug had been fixed since jdk7 (#1797) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-29 17:33:18 +05:30
Duo Zhang f06248ef84
HBASE-24443 Refactor TestCustomSaslAuthenticationProvider (#1790) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-05-28 20:44:31 +08:00
Viraj Jasani 48e9835adb
HBASE-24428 : Update compaction priority for recently split daughter regions (#1784) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2020-05-27 20:56:41 +05:30
binlijin aacb69d098
HBASE-24371 Add more details when print CompactionConfiguration info (#1711) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
 2020-05-27 19:41:43 +08:00
Duo Zhang 476cb16232
HBASE-24437 Addendum just start mini dfs cluster, and make the log roll more robust (#1787) 
Signed-off-by: stack <stack@apache.org>
 2020-05-27 11:25:25 +08:00
Michael Stack e39fd6d11b HBASE-24425 Run hbck_chore_run and catalogjanitor_run on draw of 'HBCK Report' page (#1771) 
Signed-off-by: clarax
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
 2020-05-26 19:51:50 -07:00
stack a9205f8f4d HBASE-24437 Flaky test, TestLocalRegionOnTwoFileSystems#testFlushAndCompact 
Force another WAL roll just in case and make the check more loose (its
about whether the files are present, not their count)
 2020-05-26 11:57:19 -07:00
XinSun e06e1dba10
HBASE-24416 RegionNormalizer spliting region should not be limited by hbase.normalizer.min.region.count (#1770) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-05-26 17:03:14 +08:00
Zheng Wang 29c59c0ba8 HBASE-24423 No need to get lock in canSplit because hasReferences will get lock too 2020-05-25 22:52:38 -07:00
WenFeiYi 285e59af25
HBASE-24427 HStore.add log format error (#1773) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-25 18:28:12 +05:30
Viraj Jasani 301659dcc7
HBASE-24426 : Provide regionName in warning during failure to move region from CLOSING to CLOSED (#1772) 
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2020-05-25 17:33:15 +05:30
wenbang 006e4d5e46
HBASE-24401 Cell size limit check on append should consider 0 or less value to disable the check (#1742) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-05-25 09:58:01 +08:00
huaxiangsun eaa2762a4e
HBASE-24369 Provide more information about merged child regions in Hbck Overlaps section, which cannot be fixed immediately (#1756) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Clara Xiong
Signed-off-by: stack <stack@apache.org>
 2020-05-24 17:21:04 -07:00
Duo Zhang c303f9d329
HBASE-24408 Introduce a general 'local region' to store data on master (#1753) 
Signed-off-by: stack <stack@apache.org>
 2020-05-23 15:59:51 +08:00
huaxiangsun 2b894dc100
HBASE-24379 CatalogJanitor misreports region holes when there are actually over laps. (#1741) 
Signed-off-by: stack <stack@apache.org>
 2020-05-22 16:55:54 -07:00
bsglz 0ce85fe63c
HBASE-24407 Correct the comment of clusterRegionLocationMocks in TestStochasticLoadBalancer (#1750) 2020-05-22 14:14:59 -07:00
huaxiangsun 80b64ef4dc
HBASE-24376 MergeNormalizer is merging non-adjacent regions and causing region overlaps/holes. (#1734) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
 2020-05-21 10:08:43 -07:00
huaxiangsun a8724e8120
HBASE-24370 Avoid aggressive MergeRegion and GCMultipleMergedRegionsProcedure (#1719) 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2020-05-20 14:18:58 -07:00
Viraj Jasani 86a2692dc4
HBASE-23938 : System table hbase:slowlog to store complete slow/large… (#1681) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
 2020-05-20 15:10:29 +05:30
niuyulin 8f4c255b38
HBASE-24386 TableSnapshotScanner support scan limit (#1724) 
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
 2020-05-19 20:52:28 +05:30
ramkrish86 af8398a0ac
HBASE-24365 MetricsTableWrapperAggregateImpl runnable fails due to exception and never runs (#1729) 
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
 2020-05-18 15:11:02 +05:30
binlijin 6ce1e140ee
HBASE-24377 MemStoreFlusher throw NullPointerException (#1721) 
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-05-18 09:58:28 +08:00
Baiqiang Zhao 61efb6d5ad
HBASE-24381 The Size metrics in Master Webui is wrong if the size is 0 (#1723) 2020-05-18 09:14:36 +08:00
Viraj Jasani 15627bb722
HBASE-24327 : Flaky connection in TestMasterShutdown (ADDENDUM) 2020-05-16 12:29:16 +05:30
Guanghao Zhang bab4b1cf8c HBASE-23702 hbase.hstore.flusher.count setting to 0 breaks HMaster (#1703) 
Co-authored-by: Ctest
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-05-15 09:21:53 +08:00
Andrew Purtell 94f36fdb8e
Amend HBASE-24350: Extending and Fixing HBaseTable level replication metrics (#1704) 
- Rename WALEntryBatch#getWaEntriesWithSize -> getWalEntriesWithSize
 2020-05-14 13:42:48 -07:00
stack 32e2682310 HBASE-24368 Let HBCKSCP clear 'Unknown Servers', even if RegionStateNode has RegionLocation == null 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/GCMultipleMergedRegionsProcedure.java
 Edit a log.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/HBCKServerCrashProcedure.java
 Add override of isMatchingRegionLocation. Allow 'null' as a pass in
 HBCKSCP.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
 Add a method for HBCKSCP to override and be less strict filtering
 assigns.

hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
 Some doc on what 'Unknown Servers' are.
 2020-05-14 13:09:03 -07:00
Sandeep Pal 2e5a664233
HBASE-24350: Extending and Fixing HBaseTable level replication metrics (#1704) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2020-05-14 10:34:51 -07:00
bsglz 5e32e08782
HBASE-24164 Retain the ReadRequests and WriteRequests of region on we… (#1500) 
Signed-off-by: stack <stack@apache.org>
 2020-05-13 22:28:45 -07:00
Mingliang Liu a40a0322a7
HBASE-23969 Meta browser should show all `info` columns (#1485) 
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-13 11:50:53 -07:00
Viraj Jasani a782531633
HBASE-24327 : Flaky connection in TestMasterShutdown#testMasterShutdo… (#1690) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-05-13 02:38:47 +05:30
Gkkkk302 9e73244082
HBASE-24355 Fix typos in the HStore#compact annotation (#1699) 
Co-authored-by: guo.kangkang <guo.kangkang@immomo.com>

Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-12 21:07:49 +05:30
Duo Zhang 8601416ee8
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for modules other than hbase-assembly (#1640) 
Signed-off-by: stack <stack@apache.org>
 2020-05-12 12:03:30 +08:00
stack bb3191c563 HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP Addendum2: make same fix for TestRaceBetweenSCPAndTRSP but in a second place (addendum 1 got first place). 2020-05-11 20:48:21 -07:00
stack 2c11d5a0a0 HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP 
Addendum: make same fix for TestRaceBetweenSCPAndTRSP which does same
thing.
 2020-05-11 14:06:56 -07:00
Joseph295 68229c9c99
HBASE-24341 The region should be removed from ConfigurationManager as a ConfigurationObserver when it is closed (#1675) 
Co-authored-by: Joseph295 <johnhomsea@gmail.com>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
 2020-05-11 17:31:54 +05:30
Viraj Jasani 6729cafa1d
Revert "HBASE-24327 : Handle shutdown() if master cannot be contacted (#1684)" (#1691) 
This reverts commit 34a74534fe.
 2020-05-11 16:43:49 +05:30
Geoffrey Jacoby ca81283fe5
HBASE-24321 - Add writable MinVersions and read-only Scan to coproc S… (#1655) 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Lars Hofhansl <larsh@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-11 12:54:22 +05:30
Viraj Jasani 34a74534fe
HBASE-24327 : Handle shutdown() if master cannot be contacted (#1684) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-05-10 17:09:51 +05:30
huaxiangsun 30f07e8187
HBASE-24256 When fixOverlap hits the max region limit, it is possible to include the same region in multiple merge request (#1584) 
Signed-off-by: stack <stack@apache.org>
 2020-05-08 10:16:42 -07:00
Reid Chan 0cd70ed89c
HBASE-24345 [ACL] renameRSGroup should require Admin level permission (#1683) 
Signed-off-by: binlijin <binlijin@gmail.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-08 18:56:05 +08:00
Jan Hentschel 03e5a1402d
HBASE-24305 Removed deprecations in ServerName (#1666) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@apache.org>
 2020-05-08 09:41:55 +02:00
stack fc283f7a68 HBASE-24338 [Flakey Tests] NPE in TestRaceBetweenSCPAndDTP #1669 
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-05-07 13:54:45 -07:00
stack d30268d0ee HBASE-24342 [Flakey Tests] Disable TestClusterPortAssignment.testClusterPortAssignment as it can't pass 100% of the time 2020-05-07 13:22:35 -07:00
niuyulin f4a446c3d2
HBASE-24328 skip duplicate GCMultipleMergedRegionsProcedure while previous finished (#1629) 
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
 2020-05-06 10:06:49 -07:00
Michael Stack 6f059ecff0 HBASE-24331 [Flakey Test] TestJMXListener rmi port clash (#1657) 
Add check that we can make jmx connector in setup. If we can't retry.
 2020-05-06 09:29:57 -07:00
Reid Chan c32d18d4fb
HBASE-24336 [Metrics] FSDataInputStream's localBytesRead is wrong (#1667) 
Signed-off-by: binlijin <binlijin@gmail.com>
 2020-05-06 19:24:05 +08:00
Mohammad Arshad 294c228c67
HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1631) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-05-06 15:12:20 +05:30
Duo Zhang c1cb22f0b3
HBASE-24265 Remove hedged rpc call support, implement the logic in MaterRegistry … (#1593) 
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-05-06 15:55:26 +08:00
Duo Zhang a9a1b9524d
HBASE-24304 Separate a hbase-asyncfs module (#1628) 
Signed-off-by: stack <stack@apache.org>
 2020-05-06 14:40:21 +08:00
huaxiangsun 045c909bdf
HBASE-24316 GCMulitpleMergedRegionsProcedure is not idempotent (#1660) 
It addresses couple issues:
   1. Make sure deleteMergeQualifiers() does not delete the row if there is no columns with "merge" keyword.
   2. GCMulitpleMergedRegionsProcedure now acquire an exclusive lock on the child region.

Signed-off-by: stack <stack@apache.org>
 2020-05-05 23:04:48 -07:00
Nick Dimiduk 07077a3950
HBASE-23976 Use nio for file manipulation in TestVerifyBucketCacheFile (#1342) 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-05-05 22:04:26 -07:00
Michael Stack 33eadb5bb1 HBASE-24307 [Flakey Tests] krb server for secure thrift tests throws BindException (#1656) 
Move the random free port generate back into hbasecommontestingutility
  from hbasetestingutility.

  Add a create simple kdc server utility that will start a kdc server and
  if a bindexception, create a new one on a new random port in hbase-common.

  Add new BoundSocketMaker helpful when trying to manufacture
  BindExceptions because of port clash.

  Change thrift and http kdc tests to use this new utility (removes
  code duplication around kdc server setup).
 2020-05-05 21:36:59 -07:00
stack 24c995da97 HBASE-24118 [Flakey Tests] TestCloseRegionWhileRSCrash 
Addendum; add back bit missing off end of test.
 2020-05-05 21:16:33 -07:00
Yu Wang 3d96007c37
HBASE-24137:The max merge count of metafixer may be remind in hbase-site.xml (#1478) 2020-05-05 16:14:14 -07:00
Michael Stack 5488124be0 HBASE-24284 [h3/jdk11] REST server won't start Exclude transitive includes of jax-rs 1.x and then explicitly include jax-rs 2.x glassfish impl for REST context when hadoop3. (#1625) 2020-05-05 15:36:01 -07:00
huaxiangsun a240ba8214
HBASE-24273 HBCK's "Orphan Regions on FileSystem" reports regions with referenced HFiles (#1613) 
Signed-off-by: stack <stack@apache.org>
 2020-05-05 11:10:02 -07:00
Duo Zhang 3340c0024e
HBASE-24314 Some classes still use log4j logger directly (#1637) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
 2020-05-05 20:03:55 +08:00
Minwoo Kang 1b11ea2e1f
HBASE-23968 Periodically check whether a system stop is requested in compaction by time. (#1274) 
Signed-off-by: stack <stack@apache.org>
 2020-05-04 14:45:20 -07:00
Nick Dimiduk e37aafcfc2 HBASE-24260 Add a ClusterManager that issues commands via coprocessor 
Implements `ClusterManager` that relies on the new
`ShellExecEndpointCoprocessor` for remote shell command execution.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-05-04 10:53:02 -07:00
Anoop Sam John 7313b4f5aa
HBASE-24311 Add more details in MultiVersionConcurrencyControl STUCK log message (#1636) 2020-05-04 21:26:29 +05:30
stack 255470a139 HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration (#1572) 
Addendum to address minor feedback on text
 2020-05-04 08:27:21 -07:00
Michael Stack ef4cc669b9 HBASE-24246 Miscellaneous hbck2 fixMeta bulk merge fixes: better logging around merges/overlap-fixing, 'HBCK Report' overlap listing, and configuration (#1572) 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Fix weird brackets around each region name when logging.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
  Log when we hit the max merge limit. Also up limit to 64.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
 Make logs make more sense to operator.

hbase-server/src/main/resources/hbase-webapps/master/hbck.jsp
 Make RegionName show when you mouseover so long names don't mess up
 display of holes and overlaps.

Address Mingliang Liu liuml07 feedback

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-05-04 08:27:10 -07:00
niuyulin c44bb2e99a HBASE-24221 Support bulkLoadHFile by family (ADDENDUM) 
Co-authored-by: niuyulin <niuyulin@xiaomi.com>

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2020-05-04 15:21:38 +01:00
Wellington Chevreuil b6e81c0e3f Revert "HBASE-24221 Support bulkLoadHFile by family, fix UT (#1633)" 
This reverts commit 5d41588df6.
 2020-05-04 15:21:01 +01:00
niuyulin 5d41588df6
HBASE-24221 Support bulkLoadHFile by family, fix UT (#1633) 
Co-authored-by: niuyulin <niuyulin@xiaomi.com>
 2020-05-04 15:18:55 +01:00
xincunSong d67f065594
[HBASE-24288]Allow admin user to create table and do bulkLoad (#1612) 
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: binlijin <binlijin@gmail.com>
 2020-05-01 11:57:33 -07:00
gkanade 9785f5d1c9
HBASE-24199 Procedure related metrics is not consumed in the JMX metric (#1616) 
Signed-off-by: stack <stack@apache.org>
 2020-05-01 11:45:16 -07:00
Duo Zhang e8504ba817
HBASE-24277 TestZooKeeper is flaky (#1617) 2020-05-01 20:34:42 +08:00
niuyulin 512d00e75d
HBASE-24221 Support bulkLoadHFile by family (#1569) 
Signed-off-by: Guanghao Zhang <zghao@apache.org>
 2020-04-30 18:33:37 +08:00
bsglz b810c9bb91
HBASE-22710 Wrong result in one case of scan that use raw and version… (#767) 
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-04-30 13:51:37 +05:30
xincunSong 856e21d199
HBASE-24282 'scanDetail' log message is missing when responseTooSlow happens on the first scan rpc call (#1604) 
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: stack <stack@apache.org>
 2020-04-29 16:03:24 -07:00
stack e3c5527ce5 HBASE-24247 Failed multi-merge because two regions not adjacent (legitimately). 
Addendum: Restore old semantic where merge of non-adjacent required
'force'
 2020-04-29 14:35:48 -07:00
stack 9cff7a4ff4 HBASE-24215 [Flakey Tests] [ERROR] TestSecureRESTServer java.lang.NoClassDefFoundError: com/sun/jersey/core/spi/factory/AbstractRuntimeDelegate 
Addendum: Add jersey-servlet to hadoop3 profile.

Made the hadoop3 profile in top-level pom same as it is for hadoop2
when it comes to exclusions. Then backed out previous attempt mostly.
Made the failing test medium-sized so it ran in its own jvm.
 2020-04-29 07:39:59 -07:00
Joseph295 f46dba865f
HBASE-24278 The Java doc for minFilesToCompact is wrong (#1599) 
Co-authored-by: Joseph295 <johnhomsea@gmail.com>
SIgned-off-by: stack <stack@apache.org>
 2020-04-28 22:09:56 -07:00
Duo Zhang 9f52e6b725
HBASE-24249 Move code in FSHDFSUtils to FSUtils and mark related clas… (#1586) 
Signed-off-by: stack <stack@apache.org>
 2020-04-29 10:44:34 +08:00
Duo Zhang 6928674eb8
HBASE-24228 Merge the code in hbase-hadoop2-compat module to hbase-hadoop-compat (#1563) 
Signed-off-by: stack <stack@apache.org>
 2020-04-29 10:34:53 +08:00
Michael Stack 8fceec8cb1 HBASE-24247 Failed multi-merge because two regions not adjacent (legitimately) (#1570) 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add new isOverlap method that takes list of RegionInfos checking that
 current RegionInfo is overlapped by the passed in Regions.

    Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
    Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
 2020-04-28 13:24:04 -07:00
Duo Zhang 39a1bc53f8 Revert "HBASE-24212: Create table is slow in large cluster when AccessController is enabled. (#1546)" 
Wrong JIRA issue number and also breaks TestAccessController.testAccessControllerUserPermsRegexHandling

This reverts commit e6cc5eb2f0.
 2020-04-27 22:45:52 +08:00
Duo Zhang 84f2e95dd5
HBASE-24264 Disable TestNettyIPC.testHedgedAsyncEcho (#1589) 2020-04-26 23:36:07 +08:00
bsglz ec0d9d767e
HBASE-23264 Resolve a TODO of BucketAllocator: "Why we add the extra … (#801) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-04-26 15:15:31 +05:30
Josh Elser 8774614a8c HBASE-23881 Ensure Netty client receives at least one response before considering SASL negotiation complete 
The PLAIN mechanism test added in the Shade authentication example has
different semantics than GSSAPI mechanism -- the client reports that the
handshake is done after the original challenge is computed. The javadoc
on SaslClient, however, tells us that we need to wait for a response
from the server before proceeding.

The client, best as I can see, does not receive any data from HBase;
however the application semantics (e.g. throw an exception on auth'n
error) do not work as we intend as a result of this bug.

Extra trace logging was also added to debug this, should a similar error
ever happen again with some other mechanism.

Closes #1260

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
 2020-04-26 10:24:07 +08:00
Duo Zhang c1f0634462 Revert "HBASE-23381 Ensure Netty client receives at least one response before considering SASL negotiation complete" 
Incorrect jira issue number

This reverts commit 78eecd3a7d.
 2020-04-26 10:23:40 +08:00
Mohammad Arshad e6cc5eb2f0
HBASE-24211: Create table is slow in large cluster when AccessController is enabled. (#1546) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Pankaj <pankajkumar@apache.org>
 2020-04-25 17:41:43 +05:30
Duo Zhang 24961c5c34
HBASE-23848 Addendum fix TestSeekOptimizations (#1574) 2020-04-24 19:50:30 +08:00
niuyulin 6cea5c6462
HBASE-24222 remove FSUtils.checkAccess and replace with FileSystem.access in HBCK (#1557) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-04-23 23:20:38 +08:00
Viraj Jasani 4ed15e51d9
HBASE-24024 : Reject multi() requests with rows higher than threshold (#1560) 
Signed-off-by: Reid Chan <reidchan@apache.org>
 2020-04-22 12:46:40 +05:30
Jan Hentschel 75c717d4c2
HBASE-23848 Removed deprecated setStopRow from Scan (#1184) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2020-04-22 15:15:17 +08:00
niuyulin bc9184ee00
HBASE-23933 Separate a hbase-balancer module (#1436) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2020-04-22 14:55:38 +08:00
Beata Sudi 5f45c8293d
HBASE-24139 : Balancer should avoid leaving idle region servers (#1511) 
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
 2020-04-22 09:51:53 +05:30
stack e7b5c34d28 HBASE-24226 Address other hard references to '/tmp' found in Configuration 2020-04-21 17:40:35 -07:00