17792 Commits

Author SHA1 Message Date
Nick Dimiduk
1378776a91
HBASE-24631 Loosen Dockerfile pinned package versions of the "debian-revision" (#1965)
After a bit of research into [0] and [1], and a bit of
experimentation, it seems we can use a partial wild-card expression
for these version strings. Let's try this for now. If it works out, we
should expand this usage to all the version package numbers, pinning
them to their epic:upstream-version components.

[0]: http://manpages.ubuntu.com/manpages/xenial/en/man5/deb-version.5.html
[1]: http://manpages.ubuntu.com/manpages/xenial/man8/apt-get.8.html

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-25 21:24:10 +08:00
Sean Busbey
497862bb47
HBASE-23339 release scripts should use host gpg-agent when running in container.
* put together a docker container that can use host gpg-agent forwarded over ssh.
* use gpg-agent forwarding container on OS X and directly forward the agent on Linux
* clean up the release container on exit
* use docker mounts instead of the deprecated volume syntax
* use image names within our project namespace
* update README to walk through running on GCE with gpg-agent forwarding

closes #1620

Signed-off-by: Matt Foley <mattf@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-06-25 00:39:43 -05:00
Toshihiro Suzuki
7fee4b5fb6
HBASE-24600 Empty RegionAction added to MultiRequest in case of RowMutations/CheckAndMutate batch (#1938)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-25 09:52:44 +09:00
Nick Dimiduk
f67846ed35 HBASE-24630 Purge dev javadoc from client bin tarball
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-06-24 17:11:23 -07:00
BukrosSzabolcs
8cdb2cca44
HBASE-24562: Stabilize master startup with meta replicas enabled (#1903)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
2020-06-24 18:38:36 +01:00
huaxiangsun
82fad5d789
HBASE-24552 Replica region needs to check if primary region directory exists at file system in TransitRegionStateProcedure (#1924)
Signed-off-by:  stack <stack@apache.org>
2020-06-24 09:43:12 -07:00
Duo Zhang
22103067b6
HBASE-24609 Move MetaTableAccessor out of hbase-client (#1943)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-24 21:20:35 +08:00
XinSun
4b89ee24f3
HBASE-24431 RSGroupInfo add configuration map to store something extra (#1782)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-24 18:46:09 +08:00
Wellington Ramos Chevreuil
b556343292 HBASE-21773 - Addendum - Bring back "public static Job createSubmitta… (#1953)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>

(cherry picked from commit edf75f2535b6f6853a997525efe3f6f4ef4984b1)
2020-06-23 16:14:59 +01:00
Nick Dimiduk
667f52b4bc
HBASE-24231 Add hadoop 3.2.x in our support matrix (#1954)
Add a line for hadoop-3.2.x. Values are based on the if-statement in
our personality file,

```
  if [[ "${PATCH_BRANCH}" = branch-1* ]]; then
    yetus_info "Setting Hadoop 3 versions to test based on branch-1.x rules."
    hbase_hadoop3_versions=""
  elif [[ "${PATCH_BRANCH}" = branch-2.0 ]] || [[ "${PATCH_BRANCH}" = branch-2.1 ]]; then
    yetus_info "Setting Hadoop 3 versions to test based on branch-2.0/branch-2.1 rules"
    if [[ "${QUICK_HADOOPCHECK}" == "true" ]]; then
      hbase_hadoop3_versions="3.0.3 3.1.2"
    else
      hbase_hadoop3_versions="3.0.3 3.1.1 3.1.2"
    fi
  else
    yetus_info "Setting Hadoop 3 versions to test based on branch-2.2+/master/feature branch rules"
    if [[ "${QUICK_HADOOPCHECK}" == "true" ]]; then
      hbase_hadoop3_versions="3.1.2 3.2.1"
    else
      hbase_hadoop3_versions="3.1.1 3.1.2 3.2.0 3.2.1"
    fi
  fi
```

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-06-23 07:32:47 -07:00
Michael Stack
500a1df379 HBASE-23055 Alter hbase:meta (#1956)
Addendum to fix illegal removal of unused constant w/o
a deprecation cycle.
2020-06-22 20:25:15 -07:00
xincunSong
d46adc7cc8
HBASE-24605 Break long region names in the web UI (#1942)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2020-06-23 10:03:18 +08:00
Nick Dimiduk
90b339a4f2
HBASE-24567 Create release should url-encode all characters when building git uri
By default, `urllib.quote` will skipp over `/` characters, which are
valid for use in passwords.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Matt Foley <mattf@apache.org>
2020-06-22 15:22:21 -07:00
Viraj Jasani
5a79a1c83c
HBASE-24102 : Undo visibility change for RegionMover fields (ADDENDUM) 2020-06-22 22:44:05 +05:30
niuyulin
2eeea123ef
HBASE-24604 Remove the stable-1 notice on our download page
Closes #1948

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-22 20:20:17 +05:30
Viraj Jasani
2b5ea44978
HBASE-24611 : Bring back old constructor of SnapshotDescription as deprecated (#1944)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-22 12:04:20 +05:30
Duo Zhang
8f1353b447 Revert "HBASE-24403 FsDelegationToken Should Cache Token After Acquired A New One (#1743)"
This reverts commit 7b396e9b8ca93361de6a6c4bc8a40442db77c4da.
2020-06-21 23:43:38 +08:00
Viraj Jasani
73ab216414
HBASE-24594 : Separate resource allocation for TestReplicationStatusSink (#1934)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-21 16:11:32 +05:30
XinSun
dfaba965c9
HBASE-24591 get_table_rsgroup ignored the existence of rsgroup config for namespace (#1925)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-21 17:54:39 +08:00
Sandeep Pal
f22c7a6583
HBASE-23126: Removing the un-used integration test class - IntegrationTestRSGroup
Closes #1936

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-20 22:44:03 +05:30
Viraj Jasani
3c31981179
HBASE-24380 : Provide WAL splitting journal logging (#1860)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-06-19 16:25:03 -07:00
WenFeiYi
6b62c0b6d2
HBASE-24550 Passing '-h' or '--help' to bin/hbase doesn't do as expected (ADDENDUM)
Closes #1932

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-19 22:52:18 +05:30
stack
537a3caccd HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912)
Addendum 2 fix compile error.
2020-06-18 08:49:39 -07:00
stack
489f07d945 HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912)
Addendum; minor log edits
2020-06-18 08:40:11 -07:00
Michael Stack
aadd2bb1a1 HBASE-24574 Procedure V2 - Distributed WAL Splitting => LOGGING (#1912)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
 Don't register as a chore on construction if no coordination state
 manager instance (there is no instance when procv2 WAL splitter).

hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitWALManager.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Edit logs.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionRemoteProcedureBase.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/SplitWALRemoteProcedure.java
 Add proc name rather than rely on default behavior. Add detail to the
 toString.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
 Factoring

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AbstractFSWAL.java
 Print the maxLogs... we don't do it any where.

hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
 Utility method to strip prefix from wal path.

Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: tianjingyun <tianjy@apache.org>
2020-06-18 08:38:59 -07:00
BukrosSzabolcs
bd79c4065c
HBASE-24579: Failed SASL authentication does not result in an exception on client side (#1921)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-18 12:25:43 +01:00
WenFeiYi
f73480febf
HBASE-24550 Passing '-h' or '--help' to bin/hbase doesn't do as expected
Closes #1914

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-18 15:42:57 +05:30
Michael Stack
6a576696dc HBASE-24577 Doc WALSplitter classes (#1913)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-17 15:03:33 -07:00
Viraj Jasani
192daded61
HBASE-24518 : waitForNamespaceOnline() should return false if any region is offline (#1869)
Signed-off-by: ramkrish86 <ramkrishna@apache.org>
2020-06-17 19:44:36 +05:30
ramkrish86
b17ba7b81d
HBASE-24205 - Avoid metric collection for flushes and compactions (#1918)
* HBASE-24205 - Avoid metric collection for flushes and compactions

* Use the existing matcher.isUserScan() to decide the scan type

* Added a comment saying when we track the metrics
2020-06-17 16:47:21 +05:30
ramkrish86
510aad3125
HBASE-24205 Create metric to know the number of reads that happens fr… (#1552)
* HBASE-24205 Create metric to know the number of reads that happens from memstore

* Fix checkstyles and whitespaces

* Checkstyl, whitespace and javadoc

* Fixed review comments

* Fix unused imports

* Rebase with latest commit

* Adding the table vs store metric by consolidating

* Combine get and scan metrics and make all relevant changes

* Track for full row and then increment either memstore or file read
metric

* TestMetricsStore test fix

* Only increment the memstore metric if all cells are from memstore, if
not treat as mixed reads

* Remove metricsstore and aggregate at region level

* Addresses review comments-metric name updated everywhere

* Metric name change

* Review comment changes

Co-authored-by: Ramkrishna <ramkrishna@apache.org>
Signed-off by:Anoop Sam John<anoopsamjohn@gmail.com>
Signed-off by:Viraj Jasani<virajjasani@apache.org>
2020-06-16 18:30:20 +05:30
xincunSong
3558ee0c3b
HBASE-24478 The regionInfo parameter for MasterProcedureScheduler#wai… (#1819)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: clarax <clarax98007@gmail.com>
2020-06-16 17:34:07 +08:00
Wellington Ramos Chevreuil
3ac99ad192
HBASE-21405 [DOC] Add Details about Output of "status 'replication'" (#1894)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
    Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-16 10:02:29 +01:00
Sandeep Pal
db4d539190
HBASE-24015: Test for Assign and Unassign of Regions on RegionServer on failure (#1898)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-16 12:47:52 +05:30
Bharath Vissapragada
2567d15218 HBASE-24564: Make RS abort call idempotent.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-15 22:45:35 -07:00
Viraj Jasani
2ecbaf130e
HBASE-24547 : Thrift support for HBASE-23941 (Operator support for get_slowlog_responses API) (#1892)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-06-15 14:14:28 +05:30
Duo Zhang
d0f190bdf0
HBASE-24517 Addendum add more comments (#1882)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-15 10:43:57 +08:00
bsglz
b83d38fb94
HBASE-24038 Add a metric to show the locality of ssd in table.jsp (#1337)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-14 22:46:18 +05:30
Toshihiro Suzuki
d6e9c3164d
HBASE-8458 Support for batch version of checkAndMutate() (addendum) (#1899) 2020-06-14 19:27:00 +09:00
Bharath Vissapragada
fd5002d0da
HBASE-24535: Tweak the master registry docs for branch-2 (#1890)
Updated to include changes in HBASE-24265 and some rewording
to make it version agnostic.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-06-12 14:59:04 -07:00
zhaoym6
21fe873eba HBASE-23195 FSDataInputStreamWrapper unbuffer can NOT invoke the classes that NOT implements CanUnbuffer but its parents class implements CanUnbuffer
Closes #746

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-06-12 13:01:02 -04:00
Michael Stack
9fbf1f30c9 HBASE-24545 Add backoff to SCP check on WAL split completion (#1891)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-12 07:58:34 -07:00
sguggilam
1dac9f69c4
HBASE-24446 Use EnvironmentEdgeManager to compute clock skew in Master (#1885)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-12 16:17:59 +05:30
Toshihiro Suzuki
6b7a93d10c
HBASE-24529 hbase.rs.evictblocksonclose is not honored when removing compacted files and closing the storefiles (#1881)
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
2020-06-12 18:59:03 +09:00
Nick Dimiduk
42a82df3af
HBASE-24534 Delete reference off to Hadoop wiki's HBase FAQ
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-06-11 14:20:16 -07:00
Anoop Sam John
4005e42a62
HBASE-24189 WALSplit recreates region dirs for deleted table with recovered edits data. 2020-06-11 21:27:21 +05:30
Viraj Jasani
9b2f961132
HBASE-24532 : Execute region plans with throttle should return succeeded plans (#1884)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-06-11 18:17:59 +05:30
Toshihiro Suzuki
efd5a962e5 HBASE-8458 Support for batch version of checkAndMutate()
Closes #1648

Signed-off-by: Josh Elser <elserj@apache.org>
2020-06-10 22:34:36 -04:00
Wellington Ramos Chevreuil
d8247ebae3
HBASE-24524 SyncTable logging improvements (#1878)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-10 09:36:30 +01:00
sguggilam
1eb159b455
HBASE-24511 Ability to configure timeout between RPC retry to RS from master (#1861)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-06-10 12:53:08 +05:30