Commit Graph

18645 Commits

Author SHA1 Message Date
Andrew Purtell f93bd14bd2 HBASE-27424 Upgrade Jettison for CVE-2022-40149/40150 (#4822)
Jettison versions <= 1.5.0 are subject to CVE-2022-40149 and CVE-2022-40150.

Move jettison.version to 1.5.1.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-10-11 10:12:04 -07:00
Duo Zhang 1ad9574a09 HBASE-27419 Update to hbase-thirdparty 4.1.2 (#4818)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 79f853ee7c)
2022-10-11 10:11:45 +08:00
Andrew Purtell 517e6172eb HBASE-27411 Update and clean up bcprov-jdk15on dependency (#4817)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-10-10 22:57:13 +00:00
Luca Kovács f5df7693c5 HBASE-27395 Adding description to Prometheus metrics (#4807)
Signed-off-by: Tamas Payer <payert@apache.org>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2022-10-07 11:32:26 +02:00
Duo Zhang 397ab6900d HBASE-27401 Addendum fix more javadoc issues
(cherry picked from commit 4b45256910)
2022-10-07 00:55:13 +08:00
Duo Zhang 5cab4be075 HBASE-27401 Clean up current broken 'n's in our javadoc (#4812)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 63cdd026f0)

Conflicts:
	hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupManager.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/HRegionLocation.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Append.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueExcludeFilter.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueFilter.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
	hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestReplication.java
	hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/client/TestRemoteTable.java
	hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManager.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/HBaseServerBase.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/flush/RegionServerFlushTableProcedureManager.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Region.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/HBaseReplicationEndpoint.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsSource.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtil.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/tool/TestBulkLoadHFilesSplitRecovery.java
	hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftUtilities.java
2022-10-06 22:13:14 +08:00
Rajeshbabu Chintaguntla d8749658db HBASE-27365 Minimise block addition failures due to no space in bucket cache writers queue by introducing wait time - addendum added mistakenly removed in-memory flag 2022-10-06 16:40:10 +05:30
Duo Zhang 54089b4d9c HBASE-27409 Fix the javadoc for WARCRecord (#4814)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit ced1d642ae)
2022-10-06 18:19:01 +08:00
Luca Kovács 94ed02d7ed HBASE-27407 Fixing check for "description" request param in JMXJsonServlet.java (#4816)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-10-06 10:51:16 +01:00
Rajeshbabu Chintaguntla 5395f92178 HBASE-27365 Minimise block addition failures due to no space in bucket cache writers queue by introducing wait time - addendum fixing the test case failure 2022-10-06 06:51:33 +05:30
Rajeshbabu Chintaguntla 0566f0cca5 HBASE-27365 Minimise block addition failures due to no space in bucket cache writers queue by introducing wait time-addendum 2022-10-05 04:29:31 +05:30
Rajeshbabu Chintaguntla 108479e95e HBASE-27365 Minimise block addition failures due to no space in bucket cache writers queue by introducing wait time 2022-10-04 23:35:56 +05:30
Bryan Beaudreault af28101ccc HBASE-27381 Still seeing 'Stuck' in static initialization creating RegionInfo instance (#4813)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-10-04 08:29:20 -04:00
Duo Zhang 1dc4fd53d0 HBASE-27403 Remove 'Remove unhelpful javadoc stubs' spotless rule for now (#4809)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
(cherry picked from commit 6abc1aefd1)
2022-10-03 23:17:32 +08:00
Bryan Beaudreault 913ca3ed3c HBASE-27391 Downgrade ERROR log to DEBUG in ConnectionUtils.updateStats (#4804)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-28 11:32:15 -04:00
LiangJun He f69b47275e
HBASE-27304 Support using IP to expose master/rs servers for some special scenarios (#4798)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-25 22:50:06 +08:00
Bryan Beaudreault 2a92819f0b HBASE-27280 Add mutual authentication support to TLS (#4796)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by: Andor Molnár <andor@cloudera.com>
2022-09-23 09:11:30 -04:00
Wellington Ramos Chevreuil 34f644095f HBASE-27386 Use encoded size for calculating compression ratio in block size predicator (#4795)
Signed-off-by: Ankit Singhal <ankit@apache.org>
2022-09-23 10:59:12 +01:00
Wellington Ramos Chevreuil f8c0c35fce HBASE-27370 Avoid decompressing blocks when reading from bucket cache… (#4781)
Co-authored-by: Josh Elser <elserj@apache.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
2022-09-20 10:50:00 +01:00
SiCheng-Zheng 205582d962 HBASE-27310 Add disabled table is displayed red on master web tables list (#4755)
Co-authored-by: zhengsicheng <zhengsicheng@jd.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 8bc6a5954a)
2022-09-20 17:19:09 +08:00
Duo Zhang b20bdd2cf2 HBASE-27372 Update java versions in our Dockerfiles (#4784)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
Reviewed-by: SiCheng-Zheng <643463623@qq.com>
(cherry picked from commit 3f5c0a505a)
2022-09-19 15:46:55 +08:00
Duo Zhang 9fb3aee22b
HBASE-27373 Fix new spotbugs warnings after upgrading spotbugs to 4.7.2 (#4787) (#4789)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-09-18 18:31:05 +08:00
Duo Zhang 11b327fddc HBASE-27368 Do not need to throw IllegalStateException when peer is not active in ReplicationSource.initialize (#4779)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 460bb47c04)
2022-09-15 15:35:33 +08:00
Duo Zhang 7e140e55fb HBASE-27371 Bump spotbugs version (#4783)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
(cherry picked from commit 00aba9ca74)
2022-09-14 23:32:12 +08:00
Abhradeep Kundu a303a9fd71 HBASE-27317 Rectifying the option for columnfamily as mandatory (#4773)
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2022-09-14 13:43:49 +02:00
Ujjawal 74126ea647 HBASE-27352 - Quoted string argument with spaces passed from command line are propagated wrongly to the underlying java class (#4754)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2022-09-13 14:43:20 +02:00
Duo Zhang 7ae66a1d2e HBASE-27360 The trace related assertions are flaky for async client tests (#4767)
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
(cherry picked from commit f3f88ff6c1)
2022-09-09 23:03:05 +08:00
Kota-SH a1fc483ba8
HBASE-27313: Persist list of Hfiles names for which prefetch is done (#4771)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2022-09-08 20:52:50 +01:00
Michael Stack e298782ae7 HBASE-27361 Add .flattened-pom.xml to .gitignore (#4769)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 0dfc45bfa3)
2022-09-08 17:00:31 +08:00
chenglei 88d1bd0942 HBASE-27362 CompactSplit.requestCompactionInternal may bypass compactionsEnabled check (#4768)
Co-authored-by: comnetwork <comnetwork@163.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit a26cbf1057)
2022-09-08 16:43:52 +08:00
binlijin bfd8501e23
HBASE-27314 Make index block be customized and configured (#4763)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-08 09:27:53 +08:00
Ujjawal 175f5af06b HBASE-27353 - opentelemetry-context jar missing at runtime causes MR jobs to fail (#4759)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2022-09-07 10:51:33 -07:00
Andor Molnár c3a30232a8 HBASE-27346 Autodetect key/truststore file type from file extension (#4757)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
2022-09-06 16:50:54 -04:00
Bryan Beaudreault b103685a1b HBASE-27224 HFile tool statistic sampling produces misleading results (#4638)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewed-by:  Clay Baenziger <cwb@clayb.net>
2022-09-06 15:23:18 -04:00
Xiaolin Ha 18d03058aa HBASE-27332 Remove RejectedExecutionHandler for long/short compaction thread pools (#4731)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-05 21:14:08 +08:00
Michael Stack 0f29423cc5 HBASE-27340 Artifacts with resolved profiles (#4740)
Make it so our published poms carry the minimum needed to run
    an hbase; the published pom has no profiles -- the profiles
    specified at build time are resolved, their dependencies inlined,
    and then they are stripped -- and no build-time, or plugins
    dependencies or properties, etc. Resultant poms have explicit
    hadoop lib versions baked in -- no more being able to choose
    hbase with hadoop2 or haddop3 at downstream build time by setting
    a '-Dhadoop.profile=X.0'.

    Pattern is to add profiles when none in sub-modules when
    the flatten plugin complains it can't resolve an hadoop
    dependency's 'version' (e.g. hadoop-common, hadoop-hdfs).
    Adding the hadoop-2.0 and hadoop-3.0 profiles in the sub-module
    make it so the flatten plugin can figure 'hadoop.version'
    definitively.

    Another spin on the above happens when profiles already exist
    in submodule but the flatten plugin is complaining it can't
    figure figure version on an hadoop dependency NOT under
    profiles. Below, we move the delinquent hadoop dependency under
    existing profiles (minikdc was the usual dependency outside
    profiles in sub-modules that flatten complained about).

    Sometimes, moving an hadoop dependency under a profile, there
    would be excludes on the local dependency. If the parent pom
    excludes section was missing the local excludes, we added them
    up to the parent module so all excluding is done up there in
    the parent profile dependencyManagement section.

    * hbase-asyncfs/pom.xml
    * hbase-endpoint/pom.xml
    * hbase-examples/pom.xml
    * hbase-http/pom.xml
    * hbase-rest/pom.xml
    * hbase-server/pom.xml
     Move the minikdc under profiles so it picks up appropriate hadoop version
     when the flatten plugin runs.

    * hbase-hadoop2-compat/pom.xml
     Add hadoop2 and hadoop3 profiles and move hadoop-common, etc.
     under them so we pick up appropriate hadoop version when flatten
     plugin runs.

    * hbase-mapreduce/pom.xml
     Move hadoop dependencies under profiles so right version is
     available when the flatten plugin runs.

    * hbase-shaded/hbase-shaded-testing-util/pom.xml
     Add profiles for hadoop-2.0 and hadoop-3.0 and move the
     hadoop dependencies under them.

    pom.xml
     Add the flatten plugin with the flatten profiles enabled.
     Add a few excludes on hadoop profiles picked up from sub-modules.
     E.g. exclude bouncycastle bcprov-jdk15 when we include minikdc.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-02 16:44:27 -07:00
Andor Molnár 9838c070aa
HBASE-27342 Use Hadoop Credentials API to retrieve passwords of TLS key/trust stores (#4751)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
2022-09-01 10:06:15 +02:00
Istvan Toth a7dab37b7f
HBASE-22939 SpaceQuotas - Bulkload from different hdfs failed when space quotas are turned on. (#4748)
Signed-off-by: Sakthi <sakthi@apache.org>
2022-08-31 15:47:52 +02:00
LiangJun He 6a7a91a256
HBASE-27335 HBase shell hang for a minute when quiting (#4737)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-08-31 11:01:10 +08:00
Xiaolin Ha bd032460b8
HBASE-27152 Under compaction mark may leak (#4725) (#4744)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-08-30 01:19:09 +08:00
Michael Stack 0640efac7e HBASE-27338 brotli compression lib tests fail on arm64 (#4735)
Up the brotli lib from 1.7.1 to 1.8.0

Signed-off-by: Bryan Beaudreault <bbeaudreault>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2022-08-29 09:13:34 -07:00
LiangJun He 2e53a517a2 HBASE-27336 The region visualizer shows 'undefined' region server (#4736)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit fa02350a27)
2022-08-28 22:52:49 +08:00
Duo Zhang 935b0d2277 HBASE-27321 The ReplicationLogCleaner is not thread safe but can be called from different threads at the same time (#4730)
Signed-off-by: Xin Sun <ddupgs@gmail.com>
(cherry picked from commit 37651ee1b0)
2022-08-28 00:30:34 +08:00
Ujjawal c71e984a1f HBASE-25922 - Disabled sanity checks ignored on snapshot restore (#4533)
Co-authored-by: Ujjawal Kumar <u.kumar@ukumar-ltmit1s.internal.salesforce.com>

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
2022-08-26 08:57:12 -04:00
Ruanhui d92d533924 HBASE-27325 the bulkload max call queue size can be update to a wrong value (#4728)
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 92cf962fee)
2022-08-25 11:00:28 +08:00
Yutong Xiao 713ed46d03 HBASE-27246 RSGroupMappingScript#getRSGroup has thread safety problem (#4657)
Co-authored-by: Sean Xiao Yutong <sean.xiaoyt@shopee.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b44bfc52cc)

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java
2022-08-24 15:38:26 +08:00
Ruanhui f27a12a2ae HBASE-27320 hide some sensitive configuration information in the UI (#4723)
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit b4e5875dd9)
2022-08-24 14:27:50 +08:00
Luca Kovács c04edf7835 HBASE-20904 Prometheus /metrics http endpoint for monitoring (#4691)
Co-authored-by: Luca Kovacs <kovacs.luca.agota@gmail.com>
Co-authored-by: Madhusoodan P <akshayapataki123@gmail.com>
Co-authored-by: Luca Kovacs <lkovacs@cloudera.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f9ea7ee0d6)

Conflicts:
	src/main/asciidoc/_chapters/ops_mgt.adoc
2022-08-24 11:31:16 +08:00
Ruanhui f66d67bbdf HBASE-27305 add an option to skip file splitting when bulkload hfiles (#4709)
Co-authored-by: huiruan <huiruan@tencent.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 00a719e76f)

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/tool/BulkLoadHFilesTool.java
2022-08-22 22:45:30 +08:00
LiangJun He 1bd0b581cb
HBASE-27104 Add a tool command list_unknownservers (#4714)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-08-22 21:15:47 +08:00