Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,739 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

19739 Commits

Author SHA1 Message Date
Duo Zhang abc517f75a HBASE-27793 Addendum fix spotless issue 2023-05-27 22:01:45 +08:00
Andrew Purtell f347867440
HBASE-27872 xerial's snappy-java requires GLIBC >= 2.32 (#5245) 
We need to add a native library load check with a helpful error message
if xerial snappy fails to initialize due to a too old glibc or similar
reason, and disable the unit test if the native library fails to load.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2023-05-25 17:05:57 -07:00
Andrew Purtell 6a5a710e31
HBASE-27867 Close the L1 victim handler race (#5239) 
When we evict a block from L1 and move it to L2 there is a brief window of
time where we have removed the block from the L1 map and yet the victim
handler has not completed execution. Some read-your-write use cases can be
significantly impacted even though the window is small.

Victim handling can be made atomic with respect to the unmapping operation.
The upside is there will be no L1+L2 misses during the transition. The
downside is if the victim handler takes a long time to execute – currently
they are all very fast, so only a theoretical risk – then other removals or
insertions in L1 can block until it completes.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
 2023-05-25 16:46:15 -07:00
Andrew Purtell 7510d061c3
HBASE-27843 If moveAndClose fails HFileArchiver should delete any incomplete archive side changes (#5240) 
When HFiles are placed on a filesystem other than HDFS a rename operation can be
a non-atomic file copy operation. It can take a long time to copy a large hfile
and if interrupted there may be a partially copied file present at the destination.

If we fail to “rename” the files into the archive we will continue to fail
indefinitely. Before larger changes are considered, perhaps to StoreFileTracker, we
should mitigate this problem.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2023-05-25 15:42:46 -07:00
Yash Dodeja ea3a44ea2d
HBASE-27811 Enable cache control for logs endpoint and set max age as 0 (#5204) 2023-05-25 12:58:16 +05:30
Rajeshbabu Chintaguntla d7b2fb463e
HBASE-27793 Make HBCK be able to report unknown servers (#5229) 
Co-authored-by: Rajeshbabu Chintaguntla <rajeshbabu@apache.org>
 2023-05-25 08:58:26 +05:30
Shuhei Yamasaki 256e5e9cd8
HBASE-27876 Only generate SBOM when releasing (#5251) 
Move CycloneDX plugin to profile section

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-24 23:20:39 +08:00
dependabot[bot] 22526a6339
HBASE-27880 Bump requests from 2.28.1 to 2.31.0 in /dev-support/flaky-tests (#5250) 
Bumps [requests](https://github.com/psf/requests) from 2.28.1 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.28.1...v2.31.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-23 22:56:30 +08:00
dependabot[bot] d98b962b9a
HBASE-27879 Bump requests in /dev-support/git-jira-release-audit (#5249) 
Bumps [requests](https://github.com/psf/requests) from 2.22.0 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.22.0...v2.31.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-23 22:56:06 +08:00
Duo Zhang dc30ca552b
HBASE-27277 TestRaceBetweenSCPAndTRSP fails in pre commit (#5248) 
Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>
 2023-05-23 22:45:18 +08:00
Wellington Ramos Chevreuil e4e79179d4 HBASE-27874 Problem in flakey generated report causes pre-commit run to fail (#5244) 
Forward cherry-pick from branch-2.

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Reviewew-by: Kota-SH <shanmukhaharipriya@gmail.com>
(cherry picked from commit 0ba562ab4d)
 2023-05-22 11:47:38 +08:00
Shuhei Yamasaki 49bdf6140e
HBASE-27634 Builds emit errors related to SBOM parsing (#5246) 
Update CycloneDX version

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-21 19:52:07 +08:00
guluo ce29f97a80
HBASE-27848:Should fast-fail if unmatched column family exists when using ImportTsv (#5225) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-21 19:47:27 +08:00
Yash Dodeja cf9684de44
HBASE-27812 Provide option in HBase UI to disable stack trace for security (#5205) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-21 19:46:10 +08:00
Duo Zhang 71d7996047
HBASE-27870 Eliminate the 'WARNING: package jdk.internal.util.random not in java.base' when running UTs with jdk11 (#5242) 
Signed-off-by: Tianhang Tang <tianhang@apache.org>
 2023-05-18 23:17:01 +08:00
Liangjun He 93ddd7060d HBASE-27516 Document the table based replication queue storage in ref guide (#5203) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-15 22:53:48 +08:00
Liangjun He b34356202a HBASE-27809 Attach move replication queue storage from zookeeper to a separated HBase table design doc to git repo (#5197) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-15 22:53:48 +08:00
Liangjun He 679dfcfd4f HBASE-27274 Re-enable the disabled tests when implementing HBASE-27212 (#5178) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-15 22:53:48 +08:00
Duo Zhang 8fe691c705 HBASE-27775 Use a separate WAL provider for hbase:replication table (#5157) 
Signed-off-by: Liangjun He <heliangjun@apache.org>
 2023-05-15 22:53:48 +08:00
Liangjun He 650350a749 HBASE-27623 Start a new ReplicationSyncUp after the previous failed (#5150) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-15 22:53:48 +08:00
Duo Zhang 000e64abda HBASE-27216 Revisit the ReplicationSyncUp tool (#4966) 
Signed-off-by: Liangjun He <heliangjun@apache.org>
 2023-05-15 22:53:48 +08:00
Duo Zhang 7c74f9e8c5 HBASE-27430 Should disable replication log cleaner when migrating replication queue data (#4901) 
Signed-off-by: Liangjun He <heliangjun@apache.org>
 2023-05-15 22:53:48 +08:00
Duo Zhang dfb125f3b2 HBASE-27429 Add exponential retry backoff support for MigrateReplicationQueueFromZkToTableProcedure 
Signed-off-by: Liangjun He <heliangjun@apache.org>
 2023-05-15 22:53:48 +08:00
LiangJun He ffad1ff727 HBASE-27217 Revisit the DumpReplicationQueues tool (#4810) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-15 22:53:48 +08:00
Duo Zhang 0d57ee147e HBASE-27218 Support rolling upgrading (#4808) 
Signed-off-by: Yu Li <liyu@apache.org>
 2023-05-15 22:53:48 +08:00
LiangJun He 5f95a914b6 HBASE-27405 Fix the replication hfile/log cleaner report that the replication table does not exist (#4811) 
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-15 22:53:48 +08:00
Duo Zhang c01c8e45b4 HBASE-27392 Add a new procedure type for implementing some global operations such as migration (#4803) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2023-05-15 22:53:48 +08:00
Duo Zhang 7044150545 HBASE-27215 Add support for sync replication (#4762) 
Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>
 2023-05-15 22:53:48 +08:00
Duo Zhang 6d0311c1d9 HBASE-27214 Implement the new replication hfile/log cleaner (#4722) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2023-05-15 22:53:48 +08:00
Duo Zhang f81bdebedb HBASE-27213 Add support for claim queue operation (#4708) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2023-05-15 22:53:48 +08:00
Duo Zhang e2d8db6dfc HBASE-27212 Implement a new table based replication queue storage and make the minimum replication system work (#4672) 
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2023-05-15 22:53:48 +08:00
Duo Zhang 8b8c684bcc
HBASE-27864 Reduce the Cardinality for TestFuzzyRowFilterEndToEndLarge (#5237) 
From 50^3 to 30^3, 125000 -> 27000

Signed-off-by: Liangjun He <heliangjun@apache.org>
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2023-05-15 10:35:40 +08:00
Duo Zhang fd4f984d87
HBASE-27865 TestThriftServerCmdLine fails with org.apache.hadoop.hbase.SystemExitRule (#5238) 
Signed-off-by: Liangjun He <heliangjun@apache.org>
Signed-off-by: Xin Sun <ddupgs@gmail.com>
 2023-05-15 10:35:24 +08:00
Duo Zhang b28feeb989
HBASE-27863 Add hadoop 3.3.5 check in our personality script (#5236) 
Signed-off-by: Liangjun He <heliangjun@apache.org>
 2023-05-15 10:34:46 +08:00
Jonathan Albrecht 67612cc264
HBASE-27858 Update surefire version to 3.1.0 and use SurefireForkNodeFactory (#5234) 
Surefire version updated from 3.0.0-M6 -> 3.1.0.

SurefireForkNodeFactory is a new strategy to control how the forked
nodes communicate with the main maven process. It uses a tcp channel
instead of pipes and fixes some corrupted messages seen in the s390x
build.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-13 18:52:30 +08:00
Shuhei Yamasaki bee09bbd61
HBASE-27860 Fix build error against Hadoop 3.3.5 (#5233) 
Add packages for hadoop-3.3.5

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-13 08:56:37 +08:00
Jonathan Albrecht 220eacf7c5
HBASE-27857 Fix timeout exception handling in HBaseClassTestRule. (#5231) 
HBaseClassTestRule applies a timeout and a system exit rule to tests.
The timeout rule throws an exception if it hits the timeout threshold.
Since the timeout rule is applied after the system exit rule, the
system exit rule does not see the exception and does not re-enable
the system exit behavior which can cause maven to hang on some
tests.

This change applies the timeout rule before the system exit rule so
that normal system exit can be restored before the surefire forked
node is shutdown.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-13 08:54:50 +08:00
Kota-SH e343584b50
HBASE-27852: Interrupt BucketCachePersister thread when BucketCache is shutdown (#5230) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2023-05-12 10:43:27 +01:00
Jonathan Albrecht 55aff4ceef
HBASE-27851 Fix TestListTablesByState which is silently failing due to a surefire bug (#5227) 
surefire version 3.0.0-M6 has a bug where tests end up being removed from
the test results if they fail with a long exception message. See:

https://issues.apache.org/jira/browse/SUREFIRE-2079

TestListTablesByState is currently failing due to an error. However,
it is being missed because of the surefire bug. I found this while testing
the final surfire 3.0.0 version which fixes the bug and the test then
shows up as failing.

Co-authored-by: Jonathan Albrecht <jonathan.albrecht@ibm.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-11 23:08:12 +08:00
alan.zhao 5cea8112fd
HBASE-27733 hfile split occurs during bulkload, the new HFile file does not specify favored nodes (#5121) 
Co-authored-by: alanzhao <alanzhao@126.com>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2023-05-10 21:47:20 +08:00
bsglz 5d82d4f746
HBASE-27788 Skip family comparing when compare cells inner the store (#5171) 
* HBASE-27788 Skip family comparing when compare cells inner the store
 2023-05-10 10:15:07 +08:00
Ruanhui d49df8b7a9
HBASE-27844 changed type names to avoid conflicts with built-in types (#5223) 
Co-authored-by: huiruan <876107431@qq.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-08 21:32:21 +08:00
Kota-SH dca0622b62 HBASE-27752: Update the list of prefetched files upon region movement (#5194) 
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
 2023-05-05 09:55:17 +01:00
Wellington Ramos Chevreuil ab8880d3be Revert "HBASE-27752: Update the list of prefetched files upon region movement (#5194)" 
This reverts commit ece8d014af.
 2023-05-05 09:54:28 +01:00
Jonathan Albrecht 5d2b6956e5
HBASE-27838 Update zstd-jni from version 1.5.4-2 -> 1.5.5-2. (#5220) 
Note: This version includes a fix that prevented zstd-jni from
running on s390x.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
 2023-05-05 11:52:30 +08:00
Ray Mattingly bfdf1c049c
HBASE-27799: RpcThrottlingException wait interval message is misleading between 0-1s (#5192) 
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
 2023-05-04 14:12:11 -04:00
Kota-SH ece8d014af
HBASE-27752: Update the list of prefetched files upon region movement (#5194) 
Co-authored-by: Shanmukha Kota <skota@cloudera.com>
 2023-05-04 16:00:32 +01:00
Duo Zhang 78610decaa
HBASE-27824 NPE in MetricsMasterWrapperImpl.isRunning (#5218) 
Signed-off-by: Liangjun He <heliangjun@apache.org>
 2023-05-04 20:59:40 +08:00
Duo Zhang 89e80da57f
HBASE-27823 NPE in ClaimReplicationQueuesProcedure when running TestAssignmentManager.testAssignSocketTimeout (#5216) 
Also done some cleanup around the MockMasterServices related classes and tests

Signed-off-by: Liangjun He <heliangjun@apache.org>
 2023-05-04 20:59:07 +08:00
Duo Zhang b59eb96407
HBASE-27822 TestFromClientSide5.testAppendWithoutWAL is flaky (#5211) 
Signed-off-by: Liangjun He <heliangjun@apache.org>
 2023-05-04 20:58:38 +08:00