Commit Graph

8309 Commits

Author SHA1 Message Date
bsglz 724f0478ed
HBASE-24404 Support flush a single column family of region (#2032)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-10 18:13:50 +01:00
Zheng Wang b2321b3583
HBASE-24586 Add table level locality in table.jsp
Closes #1926

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-10 21:59:59 +05:30
Viraj Jasani f0bc33122e
HBASE-24712 : Remove flakes from TestMasterNoCluster#testStopDuringStart (#2045)
Forward-port from 91da188d78
2020-07-10 20:41:03 +05:30
Joseph295 67dfbe0406
HBASE-24663 Add procedure process time statistics UI (#2035)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-10 15:03:52 +08:00
niuyulin 5de1dff8f6
HBASE-24653 Show snapshot owner on Master WebUI (#2016)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-10 15:03:05 +08:00
XinSun 1d0e285a6f
HBASE-24489 Rewrite TestClusterRestartFailover.test since namespace table is gone on on master (#2025)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-10 10:03:57 +08:00
Wellington Chevreuil 7b92510bb4 Revert "HBASE-21596 Delete for a specific cell version can bring back version… (#2009)"
This reverts commit 1db89773e6.
2020-07-09 15:55:57 +01:00
Wellington Ramos Chevreuil 1db89773e6
HBASE-21596 Delete for a specific cell version can bring back version… (#2009)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-08 15:17:14 +01:00
huaxiangsun 90f4ff7d7c
HBASE-24688 AssignRegionHandler uses EventType.M_RS_CLOSE_META instead of EventType.M_RS_OPEN_META for meta region (#2029)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-07 09:58:25 -07:00
Toshihiro Suzuki 34e807a8b5
HBASE-24650 Change the return types of the new CheckAndMutate methods introduced in HBASE-8458 (#1991)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-07-07 12:49:20 +09:00
WenFeiYi 287f29818f
HBASE-24546 CloneSnapshotProcedure unlimited retry (#1900)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-06 13:45:19 +08:00
WenFeiYi 44691ce23c
HBASE-24578 [WAL] Add a parameter to config RingBufferEventHandler's SyncFuture count (#1923)
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-07-06 11:24:09 +08:00
chenglei a7a0e1a596
HBASE-24625 AsyncFSWAL.getLogFileSizeIfBeingWritten does not return the expected synced file length. (#1970)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-07-04 21:00:35 +08:00
Baiqiang Zhao afe2eac7b6
HBASE-24560 Add a new option of designatedfile in RegionMover
Closes #1901

Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-01 20:26:51 +05:30
xincunSong 9ad16aa376
HBASE-24624 Optimize table.jsp code (#1963)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2020-07-01 08:49:51 +08:00
stack 2c43f6cde7 HBASE-24661 TestHeapSize.testSizes failure
Revert "HBASE-24648 Remove the legacy 'forceSplit' related code at region server side (#1990)"

This reverts commit bfec964aef.
2020-06-30 15:57:49 -07:00
pengmq1 be57e40f36
HBASE-24289 Heterogeneous Storage for Date Tiered Compaction (#1730)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-30 15:10:04 +08:00
stack bfec964aef HBASE-24648 Remove the legacy 'forceSplit' related code at region server side (#1990)
Addendum to fix TestHeapSize
2020-06-29 14:20:39 -07:00
Duo Zhang 37b863bd0b
HBASE-24648 Remove the legacy 'forceSplit' related code at region server side (#1990)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-29 22:50:42 +08:00
Duo Zhang 46bf8944fd
HBASE-24646 Set the log level for ScheduledChore to INFO in HBTU (#1987)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-06-28 23:26:05 +08:00
Viraj Jasani 47742b2083
HBASE-24588 : createTableBegsSplit should consider tuning mergeEnabled for table (ADDENDUM) (#1988) 2020-06-27 19:09:19 +05:30
Duo Zhang af49b87367 HBASE-24635 Addendum make test more robust 2020-06-27 11:29:43 +08:00
Duo Zhang 16a25b74db
HBASE-24635 Split TestMetaWithReplicas (#1980)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-27 10:36:07 +08:00
Viraj Jasani 51ad588565
HBASE-24588 : normalization log format (ADDENDUM) 2020-06-26 23:09:37 +05:30
Viraj Jasani 71af97cd05
HBASE-24588 : Submit task for NormalizationPlan (#1933)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-06-26 22:08:52 +05:30
bsglz c0461207ee
HBASE-24382 Flush partial stores of region filtered by seqId when arc… (#1737)
* HBASE-24382 Flush partial stores of region filtered by seqId when archive wal due to too many wals

* fix checkstyle and javadoc issue

* fix javadoc issues

* move the geting of stores to HRegion, since it should not be part of FlushPolicy, and comment fix

* fix checkstyle issue

* add some comment

* remove the forceFlushAllStores since we can use families to determine how to select stores to flush

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: stack <stack@duboce.net>
2020-06-25 22:03:18 -07:00
Bharath Vissapragada 84e246f9b1
HBASE-24603: Make Zookeeper sync() call synchronous (#1945)
Writing a test for this is tricky. There is enough coverage for
functional tests. Only concern is performance, but there is enough
logging for it to detect timed out/badly performing sync calls.

Additionally, this patch decouples the ZK event processing into it's
own thread rather than doing it in the EventThread's context. That
avoids deadlocks and stalls of the event thread.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-25 13:27:38 -07:00
Michael Stack e54e3afc5c HBASE-24616 Remove BoundedRecoveredHFilesOutputSink dependency on a TableDescriptor (#1955)
Purge query Master for table descriptors; make do w/ generic options.

Logging cleanup.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
 Undo fetching Table Descriptor. Not reliably available at recovery time.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-25 12:10:17 -07:00
BukrosSzabolcs 8cdb2cca44
HBASE-24562: Stabilize master startup with meta replicas enabled (#1903)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-06-24 18:38:36 +01:00
huaxiangsun 82fad5d789
HBASE-24552 Replica region needs to check if primary region directory exists at file system in TransitRegionStateProcedure (#1924)
Signed-off-by:  stack <stack@apache.org>
2020-06-24 09:43:12 -07:00
Duo Zhang 22103067b6
HBASE-24609 Move MetaTableAccessor out of hbase-client (#1943)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-24 21:20:35 +08:00
XinSun 4b89ee24f3
HBASE-24431 RSGroupInfo add configuration map to store something extra (#1782)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-24 18:46:09 +08:00
xincunSong d46adc7cc8
HBASE-24605 Break long region names in the web UI (#1942)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2020-06-23 10:03:18 +08:00
Viraj Jasani 5a79a1c83c
HBASE-24102 : Undo visibility change for RegionMover fields (ADDENDUM) 2020-06-22 22:44:05 +05:30
Duo Zhang 8f1353b447 Revert "HBASE-24403 FsDelegationToken Should Cache Token After Acquired A New One (#1743)"
This reverts commit 7b396e9b8c.
2020-06-21 23:43:38 +08:00
Viraj Jasani 73ab216414
HBASE-24594 : Separate resource allocation for TestReplicationStatusSink (#1934)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-21 16:11:32 +05:30
XinSun dfaba965c9
HBASE-24591 get_table_rsgroup ignored the existence of rsgroup config for namespace (#1925)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-21 17:54:39 +08:00
Viraj Jasani 3c31981179
HBASE-24380 : Provide WAL splitting journal logging (#1860)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-06-19 16:25:03 -07:00
stack 537a3caccd HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912)
Addendum 2 fix compile error.
2020-06-18 08:49:39 -07:00
stack 489f07d945 HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912)
Addendum; minor log edits
2020-06-18 08:40:11 -07:00
Michael Stack aadd2bb1a1 HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
 Don't register as a chore on construction if no coordination state
 manager instance (there is no instance when procv2 WAL splitter).

hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Edit logs.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionRemoteProcedureBase.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALRemoteProcedure.java
 Add proc name rather than rely on default behavior. Add detail to the
 toString.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 Factoring

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
 Print the maxLogs... we don't do it any where.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
 Utility method to strip prefix from wal path.

Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: tianjingyun <tianjy@apache.org>
2020-06-18 08:38:59 -07:00
Michael Stack 6a576696dc HBASE-24577 Doc WALSplitter classes (#1913)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-17 15:03:33 -07:00
Viraj Jasani 192daded61
HBASE-24518 : waitForNamespaceOnline() should return false if any region is offline (#1869)
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
2020-06-17 19:44:36 +05:30
ramkrish86 b17ba7b81d
HBASE-24205 - Avoid metric collection for flushes and compactions (#1918)
* HBASE-24205 - Avoid metric collection for flushes and compactions

* Use the existing matcher.isUserScan() to decide the scan type

* Added a comment saying when we track the metrics
2020-06-17 16:47:21 +05:30
ramkrish86 510aad3125
HBASE-24205 Create metric to know the number of reads that happens fr… (#1552)
* HBASE-24205 Create metric to know the number of reads that happens from memstore

* Fix checkstyles and whitespaces

* Checkstyl, whitespace and javadoc

* Fixed review comments

* Fix unused imports

* Rebase with latest commit

* Adding the table vs store metric by consolidating

* Combine get and scan metrics and make all relevant changes

* Track for full row and then increment either memstore or file read
metric

* TestMetricsStore test fix

* Only increment the memstore metric if all cells are from memstore, if
not treat as mixed reads

* Remove metricsstore and aggregate at region level

* Addresses review comments-metric name updated everywhere

* Metric name change

* Review comment changes

Co-authored-by: Ramkrishna <ramkrishna@apache.org>
Signed-off by:Anoop Sam John<anoopsamjohn@gmail.com>
Signed-off by:Viraj Jasani<virajjasani@apache.org>
2020-06-16 18:30:20 +05:30
xincunSong 3558ee0c3b
HBASE-24478 The regionInfo parameter for MasterProcedureScheduler#wai… (#1819)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: clarax <clarax98007@gmail.com>
2020-06-16 17:34:07 +08:00
Sandeep Pal db4d539190
HBASE-24015: Test for Assign and Unassign of Regions on RegionServer on failure (#1898)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-16 12:47:52 +05:30
Bharath Vissapragada 2567d15218 HBASE-24564: Make RS abort call idempotent.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-15 22:45:35 -07:00
Duo Zhang d0f190bdf0
HBASE-24517 Addendum add more comments (#1882)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-15 10:43:57 +08:00
bsglz b83d38fb94
HBASE-24038 Add a metric to show the locality of ssd in table.jsp (#1337)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-14 22:46:18 +05:30