Commit Graph

829 Commits

Author SHA1 Message Date
Peter Somogyi cad3630c70
HBASE-21727 Simplify documentation around client timeout
Signed-off-by: Michael Stack <stack@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java
	src/main/asciidoc/_chapters/configuration.adoc
	src/main/asciidoc/_chapters/troubleshooting.adoc

Amending-Author: Andrew Purtell <apurtell@apache.org>
2019-01-17 16:59:12 -08:00
Andrew Purtell d6a3e6b404
HBASE-21679 Port HBASE-6028 (Start/Stop compactions at region server level) to branch-1
HBASE-6028 Start/Stop compactions at region server level

Add switching on/off of compactions.

Switching off compactions will also interrupt any currently ongoing compactions.
Adds a "compaction_switch" to hbase shell. Switching off compactions will
interrupt any currently ongoing compactions. State set from shell will be
lost on restart. To persist the changes across region servers modify
hbase.regionserver.compaction.enabled in hbase-site.xml and restart.
2019-01-11 18:08:27 -08:00
Mingliang Liu f3f3798575
HBASE-21164 reportForDuty should do backoff rather than retry
Remove unused methods from Sleeper (its ok, its @Private).
Remove notion of startTime from Sleeper handling (it is is unused).
Allow passing in how long to sleep so can maintain externally.
In HRS, use a RetryCounter to calculate backoff sleep time for when
reportForDuty is failing against a struggling Master.

Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
	hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReportForDuty.java

Amending-Author: Andrew Purtell <apurtell@apache.org>
2019-01-11 16:50:45 -08:00
Saurabh Singh 231e6d56b5
HBASE-20928 Rewrite calculation of midpoint in binarySearch functions to prevent overflow
HBASE-20928 Rewrite calculation of midpoint - addendum (Xu Cang)

Signed-off-by: tedyu <yuzhihong@gmail.com>

Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/RowIndexSeekerV1.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java

Amending-Author: Andrew Purtell <apurtell@apache.org>
2019-01-11 14:40:51 -08:00
Chia-Ping Tsai 2f541f5710
HBASE-21208 Bytes#toShort doesn't work without unsafe
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
Signed-off-by: Reid Chan <reidchan@apache.org>

Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
	hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestBytes.java

Amending-Author: Andrew Purtell <apurtell@apache.org>
2019-01-11 14:40:51 -08:00
Sahil Aggarwal 21cfff857d
HBASE-20716 Unsafe access cleanup
Changes the bytes[] conversion done in Bytes and ByteBufferUtils.
Instead of doing check unsafe_aligned available everytime, choose
the best converter at startup.

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/ByteBufferUtils.java
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java

Amending-Author: Andrew Purtell <apurtell@apache.org>
2019-01-11 14:40:45 -08:00
huzheng 9cef4e1141 HBASE-21473 RowIndexSeekerV1 may return cell with extra two \x00\x00 bytes which has no tags 2018-11-14 23:29:25 +08:00
Andrew Purtell ebad3ab8ed HBASE-21256 Improve IntegrationTestBigLinkedList for testing huge data
Backport to branch-1

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-10-15 10:53:34 +08:00
Tyler Mi d385ba1a5d HBASE-21098 Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
Test fixes by: Taklon Wu <wutaklon@amazon.com>

Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-10-12 11:13:16 -07:00
Andrew Purtell 1cba67b8ba
HBASE-21000 Default limits for PressureAwareCompactionThroughputController are too low
Based on experimental results adjust default pressure-aware compaction
throughput controller settings to bring compaction time with default
settings in line with those measured on branch-1.4 and below.

hbase.hstore.compaction.throughput.lower.bound = 52428800
hbase.hstore.compaction.throughput.higher.bound = 104857600
2018-10-09 18:46:35 -07:00
Mike Drob ac94011677
HBASE-20764 build broken when latest commit is gpg signed
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-10-08 19:53:34 -05:00
Andrew Purtell 0c6226b56c
HBASE-21162 Revert suspicious change to BoundedByteBufferPool and disable use of direct buffers for IPC reservoir by default
Revert suspicious change to BoundedByteBufferPool made on HBASE-19239 (Fix
findbugs and error-prone issues).

In addition the allocation of direct memory for the server RPC reservoir is
problematic in that tracing native memory or direct buffer leaks to a
particular class or compilation unit is difficult, so allocate the reservoir
on the heap by default instead.
2018-09-10 18:25:27 -07:00
Xu Cang 3a97976eae HBASE-20744 Address FindBugs warnings in branch-1
minor fix

Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-23 16:29:13 -07:00
Mike Drob 2ae61521aa HBASE-20884 Reclassify Base64 as IA.Private 2018-07-13 14:33:23 -05:00
Yu Li 50f7894cc4 HBASE-20691 Change the default WAL storage policy back to "NONE""
This partially reverts HBASE-19858 and adds more doc about why we choose "NONE" as the default.
2018-07-05 15:26:56 +08:00
Xu Cang 2fd69cf93c HBASE-19722 Implement a meta query statistics metrics source
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-06-28 17:17:50 -07:00
Sean Busbey 4b63557682 HBASE-20444 Addendum keep folks from looking at raw version component array.
Signed-off-by: Andrew Purtell <apurtell@apache.org>

 Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
2018-05-31 14:19:14 -05:00
maoling 345567d714 HBASE-20444 Improve comparison logic for HBase version strings
Signed-off-by: Sean Busbey <busbey@apache.org>

 Conflicts:
	hbase-common/src/main/java/org/apache/hadoop/hbase/util/VersionInfo.java

Amended to use commons-lang 2, where isNumeric("") -> true

Amending-Author: Sean Busbey <busbey@apache.org>
2018-05-30 10:25:14 -05:00
Andrew Purtell 210b90604b HBASE-20619 TestWeakObjectPool occasionally times out
Remove too-stringent timeout already removed from this test in master branch.
2018-05-22 12:24:05 -07:00
Lars Hofhansl 6685b62abe HBASE-20459 Majority of scan CPU time in HBase-1 spent in size estimation. 2018-04-21 10:02:07 -07:00
Sean Busbey ba7a936f74 Revert "HBASE-9465 Push entries to peer clusters serially"
This reverts commit 441bc050b9.

 Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java
	hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/WALProtos.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-23 14:42:15 +08:00
Sean Busbey 0a284d2b76 Revert "HBASE-17010 Serial replication should handle daughter regions being assigned to another RS (Phil Yang)"
This reverts commit 97276da9a7.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-23 14:42:04 +08:00
Andrew Purtell f35bcd2fe3 HBASE-19858 Backport HBASE-14061 (Support CF-level Storage Policy) to branch-1
HBASE-14061 Support CF-level Storage Policy
HBASE-14061 Support CF-level Storage Policy (addendum)
HBASE-14061 Support CF-level Storage Policy (addendum2)
HBASE-15172 Support setting storage policy in bulkload
HBASE-17538 HDFS.setStoragePolicy() logs errors on local fs
HBASE-18015 Storage class aware block placement for procedure v2 WALs
HBASE-18017 Reduce frequency of setStoragePolicy failure warnings
Default storage policy if not configured cannot be "NONE"
HBASE-19016 Coordinate storage policy property name for table schema and bulkload
Fix checkstyle warnings
Addressed additional review feedback on backport
2018-02-01 17:10:39 -08:00
Chia-Ping Tsai 71862094d7 HBASE-19877 hbase-common and hbase-zookeeper don't add the log4j.properties to the resource path for testing 2018-01-29 16:52:01 +08:00
Peter Somogyi 528eb10826 HBASE-19545 Replace getBytes(StandardCharsets.UTF_8) with Bytes.toBytes
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-27 20:15:01 +08:00
Andrew Purtell 1fe75f98d3 HBASE-19421 branch-1 does not compile against Hadoop 3.0.0 2017-12-04 15:48:38 -08:00
Michael Stack fb3fee3414 HBASE-19383 [1.2] java.lang.AssertionError: expected:<2> but was:<1> at
org.apache.hadoop.hbase.TestChoreService.testTriggerNowFailsWhenNotScheduled(TestChoreService.java:707)
2017-11-29 20:45:31 -08:00
Andrew Purtell c179d5144f HBASE-19239 Fix findbugs and error-prone issues
Fixes for hbase-common
2017-11-17 15:20:45 -08:00
Lars Hofhansl f9833a7802 HBASE-12091 Optionally ignore edits for dropped tables for replication. 2017-11-14 16:33:12 -08:00
Andrew Purtell ef3b59cb7a HBASE-19232 Fix shaded-check-invariants (check-jar-contents) failure on branch-1 2017-11-13 10:37:20 -08:00
Andrew Purtell 6961526573 HBASE-12350 Backport error-prone build support to branch-1 and branch-2 2017-11-09 15:34:09 -08:00
Andrew Purtell d099263e13 HBASE-19129 TestChoreService is flaky
Increase the delta to compensate for environmental variance.
2017-10-31 00:08:31 +00:00
Andrew Purtell 64328caef0 HBASE-15631 Backport Regionserver Groups (HBASE-6721) to branch-1 (Francis Liu and Andrew Purtell) 2017-10-23 17:10:33 -07:00
Sean Busbey aa50971947 Revert "HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned"
This reverts commit 256fc63007.

Backing out filterlist regression, see HBASE-18957. Work continuing branch for HBASE-18410.

Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-10-09 17:06:14 -05:00
tedyu d5033fd350 HBASE-18743 HFiles in use by a table which has the same name and namespace with a default table cloned from snapshot may be deleted when that snapshot and default table are deleted (wenbang) 2017-09-04 07:52:57 -07:00
Guangxu Cheng 433b42b41c HBASE-18461 Build broken If the username contains a backslash
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-31 08:46:39 -07:00
Sean Mackrory aec4bf6bae HBASE-15947 Classes used only for tests included in main code base
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-30 10:53:31 +08:00
Mike Drob 9c26a42ab1 HBASE-18656 First issues found by error-prone 2017-08-24 13:04:45 -05:00
Andrew Purtell 3feb87b005 HBASE-18431 Mitigate compatibility concerns between branch-1.3 and branch-1.4
- Refactor TableName into its own proto module and fix up users

- Move SnapshotDescription from Snapshot.proto back to HBase.proto

- Restore FastLongHistogram and TestFastLongHistogram;
  deprecate FastLongHistogram

- Move DeleteQueryTracker back to o.a.h.h.regionserver
2017-08-16 11:34:18 -07:00
Josh Elser c7e1d67df7 HBASE-18023 Update row threshold warning from 1k to 5k (addendum) 2017-07-25 18:39:44 -04:00
rgidwani 26247996d2 HBASE-15816 Provide client with ability to set priority on Operations
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-07-21 17:12:25 -07:00
anoopsamjohn 9036556a33 HBASE-16993 BucketCache throw java.io.IOException: Invalid HFile block magic when configuring hbase.bucketcache.bucket.sizes. 2017-07-20 23:00:48 +05:30
Chia-Ping Tsai 1b8fb0a8d8 HBASE-18365 Eliminate the findbugs warnings for hbase-common 2017-07-13 19:35:18 +08:00
Andrew Purtell 3dd55fa0c0 Set versions on branch-1 to 1.5.0-SNAPSHOT 2017-07-03 18:01:15 -07:00
Phil Yang 3381c6c453 HBASE-17931 Assign system tables to servers with highest version 2017-07-03 11:35:31 +08:00
David Harju 743f454d25 HBASE-18023 Log multi-* requests for more than threshold number of rows
Signed-off-by: Josh Elser <elserj@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
2017-06-30 18:45:36 -07:00
tedyu 940f4107b0 HBASE-18226 Disable reverse DNS lookup at HMaster and use the hostname provided by RegionServer (Duo Xu) 2017-06-21 18:23:30 -07:00
Sean Busbey c6e587c5ac HBASE-16242 Manually resolve transitive Apache Avro dependency to consistently be ver 1.7.7.
Signed-off-by: Jerry He <jerryjch@apache.org>

 Conflicts:
	hbase-common/pom.xml
	hbase-spark/pom.xml
	pom.xml
2017-06-19 15:58:43 -05:00
Andrew Purtell 316e02e3d8 HBASE-18219 Fix typo in constant HConstants.HBASE_CLIENT_MEAT_REPLICA_SCAN_TIMEOUT 2017-06-14 16:03:54 -07:00
huzheng 256fc63007 HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-06-13 20:28:04 +08:00