Commit Graph

18497 Commits

Author SHA1 Message Date
Bryan Beaudreault ad395b49ff HBASE-27391 Downgrade ERROR log to DEBUG in ConnectionUtils.updateStats (#4804)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-28 11:34:17 -04:00
Wellington Ramos Chevreuil 3aad87664d HBASE-27370 Avoid decompressing blocks when reading from bucket cache… (#4781)
Co-authored-by: Josh Elser <elserj@apache.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
2022-09-20 10:58:58 +01:00
Duo Zhang f661976066 HBASE-27372 Update java versions in our Dockerfiles (#4784)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
Reviewed-by: SiCheng-Zheng <643463623@qq.com>
(cherry picked from commit 3f5c0a505a)
2022-09-19 15:47:13 +08:00
Duo Zhang ca86eea482
HBASE-27373 Fix new spotbugs warnings after upgrading spotbugs to 4.7.2 (#4787) (#4790)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-09-19 07:56:47 +08:00
Duo Zhang df5c1ca854 HBASE-27368 Do not need to throw IllegalStateException when peer is not active in ReplicationSource.initialize (#4779)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 460bb47c04)
2022-09-15 15:35:44 +08:00
Duo Zhang b6badfa57e HBASE-27371 Bump spotbugs version (#4783)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit 00aba9ca74)
2022-09-14 23:32:28 +08:00
Ujjawal 606882e9ef HBASE-27352 - Quoted string argument with spaces passed from command line are propagated wrongly to the underlying java class (#4754)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-09-13 14:43:25 +02:00
binlijin 18c23f75f5 HBASE-27314 Make index block be customized and configured (#4763)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-13 10:10:10 +08:00
Duo Zhang c1cad7f653 HBASE-27360 The trace related assertions are flaky for async client tests (#4767)
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit f3f88ff6c1)
2022-09-09 23:03:16 +08:00
Michael Stack 5454c804fc HBASE-27361 Add .flattened-pom.xml to .gitignore (#4769)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 0dfc45bfa3)
2022-09-08 17:00:38 +08:00
chenglei fa11c8024e HBASE-27362 CompactSplit.requestCompactionInternal may bypass compactionsEnabled check (#4768)
Co-authored-by: comnetwork <comnetwork@163.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit a26cbf1057)
2022-09-08 16:44:02 +08:00
Ujjawal 8b411e1451 HBASE-27353 - opentelemetry-context jar missing at runtime causes MR jobs to fail (#4759)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-09-07 10:51:38 -07:00
Bryan Beaudreault 6f6857b83f HBASE-27224 HFile tool statistic sampling produces misleading results (#4638)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by:  Clay Baenziger <cwb@clayb.net>
2022-09-06 15:37:52 -04:00
Xiaolin Ha 890d6dedfe HBASE-27332 Remove RejectedExecutionHandler for long/short compaction thread pools (#4731)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-05 21:16:23 +08:00
Michael Stack 71cf98f63f
HBASE-27340 Artifacts with resolved profiles (#4740)
Make it so our published poms carry the minimum needed to run
    an hbase; the published pom has no profiles -- the profiles
    specified at build time are resolved, their dependencies inlined,
    and then they are stripped -- and no build-time, or plugins
    dependencies or properties, etc. Resultant poms have explicit
    hadoop lib versions baked in -- no more being able to choose
    hbase with hadoop2 or haddop3 at downstream build time by setting
    a '-Dhadoop.profile=X.0'.

    Pattern is to add profiles when none in sub-modules when
    the flatten plugin complains it can't resolve an hadoop
    dependency's 'version' (e.g. hadoop-common, hadoop-hdfs).
    Adding the hadoop-2.0 and hadoop-3.0 profiles in the sub-module
    make it so the flatten plugin can figure 'hadoop.version'
    definitively.

    Another spin on the above happens when profiles already exist
    in submodule but the flatten plugin is complaining it can't
    figure figure version on an hadoop dependency NOT under
    profiles. Below, we move the delinquent hadoop dependency under
    existing profiles (minikdc was the usual dependency outside
    profiles in sub-modules that flatten complained about).

    Sometimes, moving an hadoop dependency under a profile, there
    would be excludes on the local dependency. If the parent pom
    excludes section was missing the local excludes, we added them
    up to the parent module so all excluding is done up there in
    the parent profile dependencyManagement section.

    * hbase-asyncfs/pom.xml
    * hbase-endpoint/pom.xml
    * hbase-examples/pom.xml
    * hbase-http/pom.xml
    * hbase-rest/pom.xml
    * hbase-server/pom.xml
     Move the minikdc under profiles so it picks up appropriate hadoop version
     when the flatten plugin runs.

    * hbase-hadoop2-compat/pom.xml
     Add hadoop2 and hadoop3 profiles and move hadoop-common, etc.
     under them so we pick up appropriate hadoop version when flatten
     plugin runs.

    * hbase-mapreduce/pom.xml
     Move hadoop dependencies under profiles so right version is
     available when the flatten plugin runs.

    * hbase-shaded/hbase-shaded-testing-util/pom.xml
     Add profiles for hadoop-2.0 and hadoop-3.0 and move the
     hadoop dependencies under them.

    pom.xml
     Add the flatten plugin with the flatten profiles enabled.
     Add a few excludes on hadoop profiles picked up from sub-modules.
     E.g. exclude bouncycastle bcprov-jdk15 when we include minikdc.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-02 16:43:29 -07:00
Istvan Toth 81ba1231dc
HBASE-22939 SpaceQuotas - Bulkload from different hdfs failed when space quotas are turned on. (#4749)
Signed-off-by: Sakthi <sakthi@apache.org>
2022-08-31 15:50:10 +02:00
LiangJun He 5f6998b955 HBASE-27335 HBase shell hang for a minute when quiting (#4737)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 6a7a91a256)
2022-08-31 11:06:45 +08:00
Xiaolin Ha b0edb77b72
HBASE-27152 Under compaction mark may leak (#4725)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-08-30 01:20:28 +08:00
Michael Stack 9e080d02e6 HBASE-27338 brotli compression lib tests fail on arm64 (#4735)
Up the brotli lib from 1.7.1 to 1.8.0

Signed-off-by: Bryan Beaudreault <bbeaudreault>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-08-29 09:14:05 -07:00
LiangJun He a40d645312 HBASE-27336 The region visualizer shows 'undefined' region server (#4736)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit fa02350a27)
2022-08-28 22:53:00 +08:00
Ujjawal 13841f52f0 HBASE-25922 - Disabled sanity checks ignored on snapshot restore (#4533)
Co-authored-by: Ujjawal Kumar <u.kumar@ukumar-ltmit1s.internal.salesforce.com>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
2022-08-26 08:58:21 -04:00
Yutong Xiao 51bedf6a9e HBASE-27246 RSGroupMappingScript#getRSGroup has thread safety problem (#4657)
Co-authored-by: Sean Xiao Yutong <sean.xiaoyt@shopee.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b44bfc52cc)

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java
2022-08-24 15:40:59 +08:00
Ruanhui 98391a3c55 HBASE-27320 hide some sensitive configuration information in the UI (#4723)
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b4e5875dd9)
2022-08-24 14:28:03 +08:00
Nick Dimiduk f1f79b9efe Preparing development version 2.5.1-SNAPSHOT
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-23 15:44:51 +00:00
Nick Dimiduk 2ecd8bd6d6 Preparing hbase release 2.5.0RC1; tagging and updates to CHANGES.md and RELEASENOTES.md
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-23 15:44:34 +00:00
Nick Dimiduk 29475d6edb Preparing development version 2.5.1-SNAPSHOT
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-23 14:04:18 +00:00
Nick Dimiduk 6cdbc46c02 Preparing hbase release 2.5.0RC1; tagging and updates to CHANGES.md and RELEASENOTES.md
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-23 14:03:59 +00:00
Tak Lon (Stephen) Wu a97a5c7cc4
HBASE-26882 "HBASE-26810 Add dynamic configuration support for system coprocessors" to branch-2 2022-08-23 14:06:38 +02:00
Ruanhui 51576ae45a HBASE-27305 add an option to skip file splitting when bulkload hfiles (#4709)
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 00a719e76f)

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java
2022-08-22 22:45:55 +08:00
LiangJun He 78f587f559 HBASE-27104 Add a tool command list_unknownservers (#4714)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 1bd0b581cb)
2022-08-22 21:26:19 +08:00
tianhang 3984891114 HBASE-27129 Add a config that allows us to configure region-level storage policies (#4547)
Co-authored-by: Tang Tianhang <tianhang.tang@shopee.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f238a92b6d)
2022-08-22 10:07:41 +08:00
wenwj0 eff5053f99 HBASE-27089 Add “commons.crypto.stream.buffer.size” configuration (#4491)
Co-authored-by: wenwj0 <wenweijian2@huawei.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 7ff927dbd8)
2022-08-21 22:32:53 +08:00
Andrew Purtell f2345af6d8 HBASE-27206 Clean up error-prone findings in hbase-common (#4645)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit fbe3b90e0c)

Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/ServerName.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfo.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
	hbase-common/src/test/java/org/apache/hadoop/hbase/io/crypto/TestKeyStoreKeyProvider.java
	hbase-common/src/test/java/org/apache/hadoop/hbase/types/TestOrderedFloat32.java
2022-08-21 21:36:25 +08:00
Andrew Purtell 8c92a5d4d3 HBASE-27234 Clean up error-prone warnings in hbase-examples
Close #4647

Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 35fb37cf86)

Conflicts:
	hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/BulkDeleteEndpoint.java
	hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/RowCountEndpoint.java
	hbase-examples/src/test/java/org/apache/hadoop/hbase/security/provider/example/TestShadeSaslAuthenticationProvider.java
2022-08-21 00:27:41 +08:00
Andrew Purtell 7c3fc45b45 HBASE-27252 Clean up error-prone findings in hbase-it
Close #4662

Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 1004876bad)

Conflicts:
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestBackupRestore.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestIngest.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/trace/IntegrationTestSendTraceRequests.java
2022-08-20 23:39:26 +08:00
Fantasy-Jay 46ffdabd13 HBASE-27268 In trace log mode, the client does not print callId/startTime and the server does not print receiveTime (#4710)
Co-authored-by: jay.zhu <jay.zhu@huolala.cn>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 32c135d8a5)
2022-08-20 21:09:53 +08:00
BukrosSzabolcs 9f3c52265d HBASE-26969:Eliminate MOB renames when SFT is enabled
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>

closes #4712
2022-08-19 12:24:27 +02:00
Peter Somogyi e239f70f2d HBASE-25970 MOB data loss - incorrect concatenation of MOB_FILE_REFS (#3355)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2022-08-19 12:17:30 +02:00
Sean Busbey a9f4ec3759 HBASE-24163 MOB compactor implementations should use format specifiers when calling String.format
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2022-08-17 13:58:31 +02:00
Duo Zhang b269884728 HBASE-27294 Add new hadoop releases in our hadoop checks (#4692)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
(cherry picked from commit 4a57cbd2cb)
2022-08-16 21:43:37 +08:00
huaxiangsun e595f154d1
HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#4617) (#4704)
* HBASE-23723 Ensure MOB compaction works in optimized mode after snapshot clone (#1446)

* Reorganize MOB compaction tests for more reuse.
* Add tests for mob compaction after snapshot clone operations
* note the original table used to write a given mob hfile and use that to find it later.

Signed-off-by: Esteban Gutierrez <esteban@apache.org>

* spotless:apply to fix HBaseTestingUtility

* Fix error-prone errors

Signed-off-by: Esteban Gutierrez <esteban@apache.org>
Co-authored-by: Sean Busbey <busbey@apache.org>

Signed-off-by: Esteban Gutierrez <esteban@apache.org>
Co-authored-by: Andrew Purtell <apurtell@apache.org>
Co-authored-by: Sean Busbey <busbey@apache.org>
2022-08-15 13:16:00 -07:00
liangxs 3980e6d087 HBASE-26982 Add index and bloom filter statistics of LruBlockCache on… (#4376)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: stack <stack@apache.org>
2022-08-15 11:12:34 -07:00
huaxiangsun 7f6f184f06
HBASE-27296 Some Cell's implementation of toString() such as IndividualBytesFieldCell prints out value and tags which is too verbose (#4695) (#4702)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-08-15 09:16:51 -07:00
Duo Zhang 7c226727c0 HBASE-27299 Bump minimum hadoop 2 version to 2.10.2 (#4699)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 60394e95a7)
2022-08-15 18:57:00 +08:00
Duo Zhang 6cedefc192 HBASE-27221 Bump spotless version to 2.24.1 (#4693)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 9215066e2b)
2022-08-15 15:23:40 +08:00
SiCheng-Zheng 0ceb3577fc HBASE-27301 Add Delete addFamilyVersion timestamp verify (#4700)
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b531e71455)
2022-08-15 15:23:34 +08:00
dependabot[bot] da472aac06 HBASE-27298 Bump hadoop version to 3.2.4 (#4696)
Bumps hadoop-common from 3.2.3 to 3.2.4.

---
updated-dependencies:
- dependency-name: org.apache.hadoop:hadoop-common
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Duo Zhang <zhangduo@apache.org>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 97fb5cc6d2)
2022-08-12 14:50:49 +08:00
Peter Somogyi ad6770652f HBASE-27281 Add default implementation for Connection$getClusterId (#4683)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-08-11 17:21:33 +02:00
Duo Zhang 1040ab4fd6 HBASE-27293 Remove jenkins and personality scripts support for 1.x (#4690)
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit bffae99f40)
2022-08-11 17:10:32 +08:00
Andrew Purtell 18eef2bc69 HBASE-27240 Clean up error-prone findings in hbase-http (#4653)
Signed-off-by: Duo Zhang <zhangduo@apache.org>

(cherry picked from commit 5eaeff5fcd)
2022-08-11 14:47:16 +08:00