Commit Graph

17891 Commits

Author SHA1 Message Date
Sean Busbey 497862bb47
HBASE-23339 release scripts should use host gpg-agent when running in container.
* put together a docker container that can use host gpg-agent forwarded over ssh.
* use gpg-agent forwarding container on OS X and directly forward the agent on Linux
* clean up the release container on exit
* use docker mounts instead of the deprecated volume syntax
* use image names within our project namespace
* update README to walk through running on GCE with gpg-agent forwarding

closes #1620

Signed-off-by: Matt Foley <mattf@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-06-25 00:39:43 -05:00
Toshihiro Suzuki 7fee4b5fb6
HBASE-24600 Empty RegionAction added to MultiRequest in case of RowMutations/CheckAndMutate batch (#1938)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-25 09:52:44 +09:00
Nick Dimiduk f67846ed35 HBASE-24630 Purge dev javadoc from client bin tarball
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-06-24 17:11:23 -07:00
BukrosSzabolcs 8cdb2cca44
HBASE-24562: Stabilize master startup with meta replicas enabled (#1903)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-06-24 18:38:36 +01:00
huaxiangsun 82fad5d789
HBASE-24552 Replica region needs to check if primary region directory exists at file system in TransitRegionStateProcedure (#1924)
Signed-off-by:  stack <stack@apache.org>
2020-06-24 09:43:12 -07:00
Duo Zhang 22103067b6
HBASE-24609 Move MetaTableAccessor out of hbase-client (#1943)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-24 21:20:35 +08:00
XinSun 4b89ee24f3
HBASE-24431 RSGroupInfo add configuration map to store something extra (#1782)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-24 18:46:09 +08:00
Wellington Ramos Chevreuil b556343292 HBASE-21773 - Addendum - Bring back "public static Job createSubmitta… (#1953)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

(cherry picked from commit edf75f2535)
2020-06-23 16:14:59 +01:00
Nick Dimiduk 667f52b4bc
HBASE-24231 Add hadoop 3.2.x in our support matrix (#1954)
Add a line for hadoop-3.2.x. Values are based on the if-statement in
our personality file,

```
  if [[ "${PATCH_BRANCH}" = branch-1* ]]; then
    yetus_info "Setting Hadoop 3 versions to test based on branch-1.x rules."
    hbase_hadoop3_versions=""
  elif [[ "${PATCH_BRANCH}" = branch-2.0 ]] || [[ "${PATCH_BRANCH}" = branch-2.1 ]]; then
    yetus_info "Setting Hadoop 3 versions to test based on branch-2.0/branch-2.1 rules"
    if [[ "${QUICK_HADOOPCHECK}" == "true" ]]; then
      hbase_hadoop3_versions="3.0.3 3.1.2"
    else
      hbase_hadoop3_versions="3.0.3 3.1.1 3.1.2"
    fi
  else
    yetus_info "Setting Hadoop 3 versions to test based on branch-2.2+/master/feature branch rules"
    if [[ "${QUICK_HADOOPCHECK}" == "true" ]]; then
      hbase_hadoop3_versions="3.1.2 3.2.1"
    else
      hbase_hadoop3_versions="3.1.1 3.1.2 3.2.0 3.2.1"
    fi
  fi
```

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-06-23 07:32:47 -07:00
Michael Stack 500a1df379 HBASE-23055 Alter hbase:meta (#1956)
Addendum to fix illegal removal of unused constant w/o
a deprecation cycle.
2020-06-22 20:25:15 -07:00
xincunSong d46adc7cc8
HBASE-24605 Break long region names in the web UI (#1942)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2020-06-23 10:03:18 +08:00
Nick Dimiduk 90b339a4f2
HBASE-24567 Create release should url-encode all characters when building git uri
By default, `urllib.quote` will skipp over `/` characters, which are
valid for use in passwords.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Matt Foley <mattf@apache.org>
2020-06-22 15:22:21 -07:00
Viraj Jasani 5a79a1c83c
HBASE-24102 : Undo visibility change for RegionMover fields (ADDENDUM) 2020-06-22 22:44:05 +05:30
niuyulin 2eeea123ef
HBASE-24604 Remove the stable-1 notice on our download page
Closes #1948

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-22 20:20:17 +05:30
Viraj Jasani 2b5ea44978
HBASE-24611 : Bring back old constructor of SnapshotDescription as deprecated (#1944)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-22 12:04:20 +05:30
Duo Zhang 8f1353b447 Revert "HBASE-24403 FsDelegationToken Should Cache Token After Acquired A New One (#1743)"
This reverts commit 7b396e9b8c.
2020-06-21 23:43:38 +08:00
Viraj Jasani 73ab216414
HBASE-24594 : Separate resource allocation for TestReplicationStatusSink (#1934)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-21 16:11:32 +05:30
XinSun dfaba965c9
HBASE-24591 get_table_rsgroup ignored the existence of rsgroup config for namespace (#1925)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-21 17:54:39 +08:00
Sandeep Pal f22c7a6583
HBASE-23126: Removing the un-used integration test class - IntegrationTestRSGroup
Closes #1936

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-20 22:44:03 +05:30
Viraj Jasani 3c31981179
HBASE-24380 : Provide WAL splitting journal logging (#1860)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-06-19 16:25:03 -07:00
WenFeiYi 6b62c0b6d2
HBASE-24550 Passing '-h' or '--help' to bin/hbase doesn't do as expected (ADDENDUM)
Closes #1932

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-19 22:52:18 +05:30
stack 537a3caccd HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912)
Addendum 2 fix compile error.
2020-06-18 08:49:39 -07:00
stack 489f07d945 HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912)
Addendum; minor log edits
2020-06-18 08:40:11 -07:00
Michael Stack aadd2bb1a1 HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
 Don't register as a chore on construction if no coordination state
 manager instance (there is no instance when procv2 WAL splitter).

hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Edit logs.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionRemoteProcedureBase.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALRemoteProcedure.java
 Add proc name rather than rely on default behavior. Add detail to the
 toString.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 Factoring

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
 Print the maxLogs... we don't do it any where.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
 Utility method to strip prefix from wal path.

Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: tianjingyun <tianjy@apache.org>
2020-06-18 08:38:59 -07:00
BukrosSzabolcs bd79c4065c
HBASE-24579: Failed SASL authentication does not result in an exception on client side (#1921)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-18 12:25:43 +01:00
WenFeiYi f73480febf
HBASE-24550 Passing '-h' or '--help' to bin/hbase doesn't do as expected
Closes #1914

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-18 15:42:57 +05:30
Michael Stack 6a576696dc HBASE-24577 Doc WALSplitter classes (#1913)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-17 15:03:33 -07:00
Viraj Jasani 192daded61
HBASE-24518 : waitForNamespaceOnline() should return false if any region is offline (#1869)
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
2020-06-17 19:44:36 +05:30
ramkrish86 b17ba7b81d
HBASE-24205 - Avoid metric collection for flushes and compactions (#1918)
* HBASE-24205 - Avoid metric collection for flushes and compactions

* Use the existing matcher.isUserScan() to decide the scan type

* Added a comment saying when we track the metrics
2020-06-17 16:47:21 +05:30
ramkrish86 510aad3125
HBASE-24205 Create metric to know the number of reads that happens fr… (#1552)
* HBASE-24205 Create metric to know the number of reads that happens from memstore

* Fix checkstyles and whitespaces

* Checkstyl, whitespace and javadoc

* Fixed review comments

* Fix unused imports

* Rebase with latest commit

* Adding the table vs store metric by consolidating

* Combine get and scan metrics and make all relevant changes

* Track for full row and then increment either memstore or file read
metric

* TestMetricsStore test fix

* Only increment the memstore metric if all cells are from memstore, if
not treat as mixed reads

* Remove metricsstore and aggregate at region level

* Addresses review comments-metric name updated everywhere

* Metric name change

* Review comment changes

Co-authored-by: Ramkrishna <ramkrishna@apache.org>
Signed-off by:Anoop Sam John<anoopsamjohn@gmail.com>
Signed-off by:Viraj Jasani<virajjasani@apache.org>
2020-06-16 18:30:20 +05:30
xincunSong 3558ee0c3b
HBASE-24478 The regionInfo parameter for MasterProcedureScheduler#wai… (#1819)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: clarax <clarax98007@gmail.com>
2020-06-16 17:34:07 +08:00
Wellington Ramos Chevreuil 3ac99ad192
HBASE-21405 [DOC] Add Details about Output of "status 'replication'" (#1894)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
    Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-16 10:02:29 +01:00
Sandeep Pal db4d539190
HBASE-24015: Test for Assign and Unassign of Regions on RegionServer on failure (#1898)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-16 12:47:52 +05:30
Bharath Vissapragada 2567d15218 HBASE-24564: Make RS abort call idempotent.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-15 22:45:35 -07:00
Viraj Jasani 2ecbaf130e
HBASE-24547 : Thrift support for HBASE-23941 (Operator support for get_slowlog_responses API) (#1892)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-06-15 14:14:28 +05:30
Duo Zhang d0f190bdf0
HBASE-24517 Addendum add more comments (#1882)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-15 10:43:57 +08:00
bsglz b83d38fb94
HBASE-24038 Add a metric to show the locality of ssd in table.jsp (#1337)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-14 22:46:18 +05:30
Toshihiro Suzuki d6e9c3164d
HBASE-8458 Support for batch version of checkAndMutate() (addendum) (#1899) 2020-06-14 19:27:00 +09:00
Bharath Vissapragada fd5002d0da
HBASE-24535: Tweak the master registry docs for branch-2 (#1890)
Updated to include changes in HBASE-24265 and some rewording
to make it version agnostic.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-06-12 14:59:04 -07:00
zhaoym6 21fe873eba HBASE-23195 FSDataInputStreamWrapper unbuffer can NOT invoke the classes that NOT implements CanUnbuffer but its parents class implements CanUnbuffer
Closes #746

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-06-12 13:01:02 -04:00
Michael Stack 9fbf1f30c9 HBASE-24545 Add backoff to SCP check on WAL split completion (#1891)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-12 07:58:34 -07:00
sguggilam 1dac9f69c4
HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master (#1885)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-12 16:17:59 +05:30
Toshihiro Suzuki 6b7a93d10c
HBASE-24529 hbase.rs.evictblocksonclose is not honored when removing compacted files and closing the storefiles (#1881)
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
2020-06-12 18:59:03 +09:00
Nick Dimiduk 42a82df3af
HBASE-24534 Delete reference off to Hadoop wiki's HBase FAQ
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-06-11 14:20:16 -07:00
Anoop Sam John 4005e42a62
HBASE-24189 WALSplit recreates region dirs for deleted table with recovered edits data. 2020-06-11 21:27:21 +05:30
Viraj Jasani 9b2f961132
HBASE-24532 : Execute region plans with throttle should return succeeded plans (#1884)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-06-11 18:17:59 +05:30
Toshihiro Suzuki efd5a962e5 HBASE-8458 Support for batch version of checkAndMutate()
Closes #1648

Signed-off-by: Josh Elser <elserj@apache.org>
2020-06-10 22:34:36 -04:00
Wellington Ramos Chevreuil d8247ebae3
HBASE-24524 SyncTable logging improvements (#1878)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-10 09:36:30 +01:00
sguggilam 1eb159b455
HBASE-24511 Ability to configure timeout between RPC retry to RS from master (#1861)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-10 12:53:08 +05:30
Vico.Wu 7b396e9b8c
HBASE-24403 FsDelegationToken Should Cache Token After Acquired A New One (#1743)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-06-10 08:46:58 +08:00