Commit Graph

838 Commits

Author SHA1 Message Date
Andrew Purtell 084745de18
HBASE-22235 OperationStatus.{SUCCESS|FAILURE|NOT_RUN} are not visible to 3rd party coprocessors 2019-04-16 13:18:00 -07:00
Andrew Purtell 953e0f5f77
HBASE-22034 Backport HBASE-21401 and HBASE-22032 to branch-1
HBASE-21401 Sanity check when constructing the KeyValue

HBASE-22032 KeyValue validation should check for null byte array
2019-03-21 17:27:54 -07:00
Sean Busbey 35308a5d82 HBASE-22044 Addendum for checkstyle. 2019-03-13 13:42:53 -05:00
Sean Busbey 8cab60cd2b HBASE-22044 Restore compat for ByteBufferUtils. Also deprecated because it will be IA.Private in HBase 3.0
(cherry picked from commit f13a897bf7484f0bc512048e7a16f0d8f04cc238)

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-03-13 13:41:17 -05:00
Mingliang Liu f4af3881c1
HBASE-21164 reportForDuty should do backoff rather than retry
Remove unused methods from Sleeper (its ok, its @Private).
Remove notion of startTime from Sleeper handling (it is is unused).
Allow passing in how long to sleep so can maintain externally.
In HRS, use a RetryCounter to calculate backoff sleep time for when
reportForDuty is failing against a struggling Master.

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java
2019-02-20 18:15:30 -08:00
Andrew Purtell 47e42dab75
Revert "HBASE-21164 reportForDuty should do backoff rather than retry"
This reverts commit f3f3798575.

This change causes TestMasterShutdown to reproducibly fail in some environments.
2019-02-20 09:25:46 -08:00
Peter Somogyi 48e8675728 HBASE-21727 Simplify documentation around client timeout - ADDENDUM
Add back HBaseConfiguration#getInt with deprecation

Signed-off-by: zhangduo <zhangduo@apache.org>
2019-02-05 18:28:03 +01:00
Andrew Purtell ce6a6014da
Update POMs and CHANGES.txt for 1.5.0 RC0 2019-02-01 12:36:10 -08:00
Andrew Purtell ff0e587b8f
HBASE-21735 Port HBASE-18784 (Use of filesystem that requires hflush / hsync / append / etc should query outputstream capabilities) to branch-1
HBASE-18784 if available, query underlying outputstream capabilities where we need hflush/hsync.

* pull things that don't rely on HDFS in hbase-server/FSUtils into hbase-common/CommonFSUtils
* refactor setStoragePolicy so that it can move into hbase-common/CommonFSUtils
* refactor WALProcedureStore so that it handles its own FS interactions
* add a reflection-based lookup of stream capabilities
* call said lookup in places where we make WALs to make sure hflush/hsync is available.

HBASE-19289 Add flag to disable stream capability enforcement

HBASE-19841 Tests against hadoop3 fail with StreamLacksCapabilityException

HBASE-19947 ITU should overwrite HTU local FS

HBASE-20354 better docs for impact of proactively checking hsync support.

* Add to the quickstart guide disabling the hsync check, with a
  big warning about how we'll lose data if the check is disabled.
* Add to troubleshooting section so folks searching can get a pointer.
2019-01-24 09:23:14 -08:00
Peter Somogyi cad3630c70
HBASE-21727 Simplify documentation around client timeout
Signed-off-by: Michael Stack <stack@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
	src/main/asciidoc/_chapters/configuration.adoc
	src/main/asciidoc/_chapters/troubleshooting.adoc

Amending-Author: Andrew Purtell <apurtell@apache.org>
2019-01-17 16:59:12 -08:00
Andrew Purtell d6a3e6b404
HBASE-21679 Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1
HBASE-6028 Start/Stop compactions at region server level

Add switching on/off of compactions.

Switching off compactions will also interrupt any currently ongoing compactions.
Adds a "compaction_switch" to hbase shell. Switching off compactions will
interrupt any currently ongoing compactions. State set from shell will be
lost on restart. To persist the changes across region servers modify
hbase.regionserver.compaction.enabled in hbase-site.xml and restart.
2019-01-11 18:08:27 -08:00
Mingliang Liu f3f3798575
HBASE-21164 reportForDuty should do backoff rather than retry
Remove unused methods from Sleeper (its ok, its @Private).
Remove notion of startTime from Sleeper handling (it is is unused).
Allow passing in how long to sleep so can maintain externally.
In HRS, use a RetryCounter to calculate backoff sleep time for when
reportForDuty is failing against a struggling Master.

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java

Amending-Author: Andrew Purtell <apurtell@apache.org>
2019-01-11 16:50:45 -08:00
Saurabh Singh 231e6d56b5
HBASE-20928 Rewrite calculation of midpoint in binarySearch functions to prevent overflow
HBASE-20928 Rewrite calculation of midpoint - addendum (Xu Cang)

Signed-off-by: tedyu <yuzhihong@gmail.com>

Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/RowIndexSeekerV1.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java

Amending-Author: Andrew Purtell <apurtell@apache.org>
2019-01-11 14:40:51 -08:00
Chia-Ping Tsai 2f541f5710
HBASE-21208 Bytes#toShort doesn't work without unsafe
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
Signed-off-by: Reid Chan <reidchan@apache.org>

Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
	hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestBytes.java

Amending-Author: Andrew Purtell <apurtell@apache.org>
2019-01-11 14:40:51 -08:00
Sahil Aggarwal 21cfff857d
HBASE-20716 Unsafe access cleanup
Changes the bytes[] conversion done in Bytes and ByteBufferUtils.
Instead of doing check unsafe_aligned available everytime, choose
the best converter at startup.

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/ByteBufferUtils.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java

Amending-Author: Andrew Purtell <apurtell@apache.org>
2019-01-11 14:40:45 -08:00
huzheng 9cef4e1141 HBASE-21473 RowIndexSeekerV1 may return cell with extra two \x00\x00 bytes which has no tags 2018-11-14 23:29:25 +08:00
Andrew Purtell ebad3ab8ed HBASE-21256 Improve IntegrationTestBigLinkedList for testing huge data
Backport to branch-1

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-10-15 10:53:34 +08:00
Tyler Mi d385ba1a5d HBASE-21098 Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
Test fixes by: Taklon Wu <wutaklon@amazon.com>

Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-10-12 11:13:16 -07:00
Andrew Purtell 1cba67b8ba
HBASE-21000 Default limits for PressureAwareCompactionThroughputController are too low
Based on experimental results adjust default pressure-aware compaction
throughput controller settings to bring compaction time with default
settings in line with those measured on branch-1.4 and below.

hbase.hstore.compaction.throughput.lower.bound = 52428800
hbase.hstore.compaction.throughput.higher.bound = 104857600
2018-10-09 18:46:35 -07:00
Mike Drob ac94011677
HBASE-20764 build broken when latest commit is gpg signed
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-10-08 19:53:34 -05:00
Andrew Purtell 0c6226b56c
HBASE-21162 Revert suspicious change to BoundedByteBufferPool and disable use of direct buffers for IPC reservoir by default
Revert suspicious change to BoundedByteBufferPool made on HBASE-19239 (Fix
findbugs and error-prone issues).

In addition the allocation of direct memory for the server RPC reservoir is
problematic in that tracing native memory or direct buffer leaks to a
particular class or compilation unit is difficult, so allocate the reservoir
on the heap by default instead.
2018-09-10 18:25:27 -07:00
Xu Cang 3a97976eae HBASE-20744 Address FindBugs warnings in branch-1
minor fix

Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-23 16:29:13 -07:00
Mike Drob 2ae61521aa HBASE-20884 Reclassify Base64 as IA.Private 2018-07-13 14:33:23 -05:00
Yu Li 50f7894cc4 HBASE-20691 Change the default WAL storage policy back to "NONE""
This partially reverts HBASE-19858 and adds more doc about why we choose "NONE" as the default.
2018-07-05 15:26:56 +08:00
Xu Cang 2fd69cf93c HBASE-19722 Implement a meta query statistics metrics source
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-06-28 17:17:50 -07:00
Sean Busbey 4b63557682 HBASE-20444 Addendum keep folks from looking at raw version component array.
Signed-off-by: Andrew Purtell <apurtell@apache.org>

 Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
2018-05-31 14:19:14 -05:00
maoling 345567d714 HBASE-20444 Improve comparison logic for HBase version strings
Signed-off-by: Sean Busbey <busbey@apache.org>

 Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/VersionInfo.java

Amended to use commons-lang 2, where isNumeric("") -> true

Amending-Author: Sean Busbey <busbey@apache.org>
2018-05-30 10:25:14 -05:00
Andrew Purtell 210b90604b HBASE-20619 TestWeakObjectPool occasionally times out
Remove too-stringent timeout already removed from this test in master branch.
2018-05-22 12:24:05 -07:00
Lars Hofhansl 6685b62abe HBASE-20459 Majority of scan CPU time in HBase-1 spent in size estimation. 2018-04-21 10:02:07 -07:00
Sean Busbey ba7a936f74 Revert "HBASE-9465 Push entries to peer clusters serially"
This reverts commit 441bc050b9.

 Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java
	hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/WALProtos.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-23 14:42:15 +08:00
Sean Busbey 0a284d2b76 Revert "HBASE-17010 Serial replication should handle daughter regions being assigned to another RS (Phil Yang)"
This reverts commit 97276da9a7.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-23 14:42:04 +08:00
Andrew Purtell f35bcd2fe3 HBASE-19858 Backport HBASE-14061 (Support CF-level Storage Policy) to branch-1
HBASE-14061 Support CF-level Storage Policy
HBASE-14061 Support CF-level Storage Policy (addendum)
HBASE-14061 Support CF-level Storage Policy (addendum2)
HBASE-15172 Support setting storage policy in bulkload
HBASE-17538 HDFS.setStoragePolicy() logs errors on local fs
HBASE-18015 Storage class aware block placement for procedure v2 WALs
HBASE-18017 Reduce frequency of setStoragePolicy failure warnings
Default storage policy if not configured cannot be "NONE"
HBASE-19016 Coordinate storage policy property name for table schema and bulkload
Fix checkstyle warnings
Addressed additional review feedback on backport
2018-02-01 17:10:39 -08:00
Chia-Ping Tsai 71862094d7 HBASE-19877 hbase-common and hbase-zookeeper don't add the log4j.properties to the resource path for testing 2018-01-29 16:52:01 +08:00
Peter Somogyi 528eb10826 HBASE-19545 Replace getBytes(StandardCharsets.UTF_8) with Bytes.toBytes
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-27 20:15:01 +08:00
Andrew Purtell 1fe75f98d3 HBASE-19421 branch-1 does not compile against Hadoop 3.0.0 2017-12-04 15:48:38 -08:00
Michael Stack fb3fee3414 HBASE-19383 [1.2] java.lang.AssertionError: expected:<2> but was:<1> at
org.apache.hadoop.hbase.TestChoreService.testTriggerNowFailsWhenNotScheduled(TestChoreService.java:707)
2017-11-29 20:45:31 -08:00
Andrew Purtell c179d5144f HBASE-19239 Fix findbugs and error-prone issues
Fixes for hbase-common
2017-11-17 15:20:45 -08:00
Lars Hofhansl f9833a7802 HBASE-12091 Optionally ignore edits for dropped tables for replication. 2017-11-14 16:33:12 -08:00
Andrew Purtell ef3b59cb7a HBASE-19232 Fix shaded-check-invariants (check-jar-contents) failure on branch-1 2017-11-13 10:37:20 -08:00
Andrew Purtell 6961526573 HBASE-12350 Backport error-prone build support to branch-1 and branch-2 2017-11-09 15:34:09 -08:00
Andrew Purtell d099263e13 HBASE-19129 TestChoreService is flaky
Increase the delta to compensate for environmental variance.
2017-10-31 00:08:31 +00:00
Andrew Purtell 64328caef0 HBASE-15631 Backport Regionserver Groups (HBASE-6721) to branch-1 (Francis Liu and Andrew Purtell) 2017-10-23 17:10:33 -07:00
Sean Busbey aa50971947 Revert "HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned"
This reverts commit 256fc63007.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-10-09 17:06:14 -05:00
tedyu d5033fd350 HBASE-18743 HFiles in use by a table which has the same name and namespace with a default table cloned from snapshot may be deleted when that snapshot and default table are deleted (wenbang) 2017-09-04 07:52:57 -07:00
Guangxu Cheng 433b42b41c HBASE-18461 Build broken If the username contains a backslash
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-31 08:46:39 -07:00
Sean Mackrory aec4bf6bae HBASE-15947 Classes used only for tests included in main code base
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-30 10:53:31 +08:00
Mike Drob 9c26a42ab1 HBASE-18656 First issues found by error-prone 2017-08-24 13:04:45 -05:00
Andrew Purtell 3feb87b005 HBASE-18431 Mitigate compatibility concerns between branch-1.3 and branch-1.4
- Refactor TableName into its own proto module and fix up users

- Move SnapshotDescription from Snapshot.proto back to HBase.proto

- Restore FastLongHistogram and TestFastLongHistogram;
  deprecate FastLongHistogram

- Move DeleteQueryTracker back to o.a.h.h.regionserver
2017-08-16 11:34:18 -07:00
Josh Elser c7e1d67df7 HBASE-18023 Update row threshold warning from 1k to 5k (addendum) 2017-07-25 18:39:44 -04:00
rgidwani 26247996d2 HBASE-15816 Provide client with ability to set priority on Operations
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-07-21 17:12:25 -07:00