84b4a61e88
HBASE-22881 Fix non-daemon threads in hbase server implementation ( #512 ) ( #536 )
...
Signed-off-by: stack <stack@apache.org>
2019-08-28 09:48:27 -07:00
c60748cba1
HBASE-22893 Change the comment in HBaseClassTestRule to reflect change in default test timeouts ( #545 )
...
Signed-off-by: Sakthi <sakthi@apache.org>
2019-08-27 01:36:26 -07:00
53522f36cc
HBASE-22810 Initialize an separate ThreadPoolExecutor for taking/restoring snapshot (addendum - use the old config key) ( #517 )
2019-08-22 11:53:46 +08:00
8cbe20c382
HBASE-22841 Add more factory functions to TimeRange
...
These functions make it easier to possible to use
`org.apache.hadoop.hbase.client.Table.CheckAndMutateBuilder#timeRange`
with more interesting ranges, without being forced to use the
deprecated constructors.
Signed-off-by: huzheng <openinx@gmail.com>
2019-08-17 10:12:48 +08:00
6f9db4660a
HBASE-22810 Initialize an separate ThreadPoolExecutor for taking/restoring snapshot ( #486 )
2019-08-15 11:22:08 +08:00
225c6cd5bc
HBASE-22777 Add a multi-region merge (for fixing overlaps)
...
Makes MergeTableRegionsProcedure do more than just two regions at a
time. Compatible as MTRP was done considering one day it'd do more than
two at a time.
Changes hardcoded assumption that merge parent regions are named
mergeA and mergeB in a column on the resultant region. Instead
can have N columns on the merged region, one for each parent
merged. Column qualifiers all being with 'merge'.
Most of code below is undoing the assumption that there are two
parents on a merge only.
2019-08-07 16:58:06 -07:00
3971773189
HBASE-22790 Added deprecation versions for fields in ByteBuffAllocator
...
Signed-off-by: stack <stack@apache.org>
2019-08-05 09:01:03 +02:00
89e2cc6281
HBASE-22677 Add unit tests for org.apache.hadoop.hbase.util.ByteRangeUtils and Classes
...
Co-authored-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-05 03:32:32 +02:00
4565e9f43b
HBASE-22723 Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta
...
Refactor of CatalogJanitor so it generates a
Report on the state of hbase:meta when it runs. Also
refactor so CJ runs even if RIT (previous it would
punt on running if RIT) so it can generate a 'Report'
on the interval regardless. If RIT, it just doesn't
go on to do the merge/split GC as it used to.
If report finds an issue, dump as a WARN message
to the master log.
Follow-on is to make the Report actionable/available
for the Master to pull when it goes to draw the hbck
UI page (could also consider shipping the Report as
part of ClusterMetrics?)
Adds new, fatter Visitor to CJ, one that generates
Report on each run keeping around more findings as
it runs.
Moved some methods around so class reads better;
previous methods were randomly ordered in the class.
M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
Make a few handy methods public.
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
Add utility as defaults on the Inteface; i.e. is this the first region
in table, is it last, does a passed region come next, or does passed
region overlap this region (added tests for this new stuff).
M hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
Bugfix... handle case where buffer passed is null.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
Lots of change, reorg., but mostly adding consistency checking
to the visitor used scanning hbase:meta on a period and the
generation of a Report on what the scan has found traversing
hbase:meta. Added a main so could try the CatalogJanitor against
a running cluster.
A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitorCluster.java
Fat ugly test for CatalogJanitor consistency checking.
M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
Add tests for new functionality in RI.
M hbase-shell/src/main/ruby/hbase/table.rb
Bug fix for case where meta has a null regioninfo; scan was aborting.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-07-29 16:07:10 -07:00
f160dd3c10
HBASE-21773 rowcounter utility should respond to pleas for help
...
Signed-off-by: stack <stack@apache.org>
2019-07-26 21:29:48 +01:00
3a72ccefbd
HBASE-22648 Snapshot TTL ( #371 )
...
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotDescriptionUtils.java
2019-07-22 15:08:20 -07:00
6fb03e63b6
HBASE-22663 The HeapAllocationRatio in WebUI is not accurate because all of the heap allocation will happen in another separated allocator named HEAP ( #365 )
2019-07-20 23:36:21 +08:00
dfebcbec38
HBASE-22700:incorrect timeout in recommended ZooKeeper configuration
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-07-17 22:12:07 +08:00
955e643e1d
HBASE-22586 Javadoc Warnings related to @param tag
...
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-07-13 16:56:53 +02:00
ed44b60cb1
HBASE-22651 ErrorProne issue in TestByteBufferArray
...
Signed-off-by: Michael Stack <stack@apache.org>
2019-07-02 16:37:46 +02:00
8fa7800389
HBASE-22572 Javadoc Warnings: @link reference not found ( #306 )
...
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: stack <stack@apache.org>
2019-07-01 21:18:11 -07:00
345f80ea5f
HBASE-20060 Add details of off heap memstore into book. ( #334 )
2019-06-27 16:54:48 +08:00
6a3aa789d8
HBASE-22612 Address the final overview reviewing comments of HBASE-21879
2019-06-26 15:40:23 +08:00
af0e23c359
HBASE-22547 Align the config keys and add document for offheap read in HBase Book. ( #301 )
2019-06-25 22:25:30 +08:00
d34c4a0481
HBASE-22598 Deprecated the hbase.ipc.server.reservoir.initial.buffer.size & hbase.ipc.server.reservoir.initial.max for HBase2.x compatibility ( #318 )
2019-06-25 22:22:29 +08:00
3199175932
HBASE-22491 Separate the heap HFileBlock and offheap HFileBlock because the heap block won't need refCnt and save into prevBlocks list before shipping ( #268 )
2019-06-25 22:22:16 +08:00
5ff317fe7b
HBASE-22504 Optimize the MultiByteBuff#get(ByteBuffer, offset, len)
2019-06-25 22:22:04 +08:00
4dd728dfff
HBASE-22483 It's better to use 65KB as the default buffer size in ByteBuffAllocator ( #279 )
2019-06-25 22:20:58 +08:00
9e5fc2b379
HBASE-22435 Add a UT to address the HFileBlock#heapSize() in TestHeapSize
2019-06-25 22:14:57 +08:00
b00360731a
HBASE-22412 Improve the metrics in ByteBuffAllocator
2019-06-25 22:14:50 +08:00
d7009d258c
HBASE-21921 Notify users if the ByteBufAllocator is always allocating ByteBuffers from heap which means the increacing GC pressure
2019-06-25 22:14:36 +08:00
39a6a357ff
HBASE-21937 Make the Compression#decompress can accept ByteBuff as input
2019-06-25 22:11:51 +08:00
fc460136e0
HBASE-21957 Unify refCount of BucketEntry and refCount of hbase.nio.ByteBuff into one
2019-06-25 22:07:16 +08:00
bd8260c07e
HBASE-22159 ByteBufferIOEngine should support write off-heap ByteBuff to the bufferArray
2019-06-25 21:28:02 +08:00
081dc03e12
HBASE-22127 Ensure that the block cached in the LRUBlockCache offheap is allocated from heap
2019-06-25 21:23:54 +08:00
0d294d23a9
HBASE-22005 Use ByteBuff's refcnt to track the life cycle of data block
2019-06-25 21:16:48 +08:00
c19592a971
HBASE-21916 Abstract an ByteBuffAllocator to allocate/free ByteBuffer in ByteBufferPool
2019-06-25 21:12:37 +08:00
974e1086d7
HBASE-22617 Recovered WAL directories not getting cleaned up ( #330 )
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-06-25 17:32:39 +08:00
23a2f8abab
HBASE-22615 Make TestChoreService more robust to timing
...
* phrase fudge factor "deltas" in terms of the original period
* increase the delta allowed for chore timing from 5% to 20%
* improve some assertions
Closes #328
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Sakthi <sakthivel.azhaku@gmail.com>
(cherry picked from commit 9aee88e03a
2019-06-22 00:29:04 -05:00
bc737d1ed9
HBASE-21284 Default limits for PressureAwareCompactionThroughputController are too low
...
This is a forward port HBASE-21000 from branch-1.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-06-20 12:44:14 -07:00
6cff25cc23
HBASE-22344 Documented the deprecation of public and limited private APIs ( #208 )
...
Signed-off-by: stack <stack@apache.org>
2019-06-17 22:29:51 +02:00
a2b22a6807
HBASE-22565 Javadoc Warnings: @see cannot be used in inline documentation
2019-06-13 23:14:58 +02:00
38139350a3
HBASE-22458: TestClassFinder fails when run on JDK11
...
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 0445186f1a
2019-06-13 09:02:20 -05:00
35d3612f66
HBASE-22534 TestCellUtil fails when run on JDK11
...
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 9b23e1d00c
2019-06-04 14:51:20 -05:00
1a2632b796
HBASE-22316 Record the stack trace for current thread in FutureUtils.get
2019-05-30 11:27:43 +08:00
ea41133e44
HBASE-22496 UnsafeAccess.unsafeCopy should not copy more than UNSAFE_COPY_THRESHOLD on each iteration
...
Change-Id: I259bc54a0a5b3474d3c455639c3e9cb1e95c8438
Signed-off-by: huzheng <openinx@gmail.com>
2019-05-30 10:21:43 +08:00
e9e680f3d1
HBASE-22469 replace md5 checksum in saveVersion script with sha512
...
Signed-off-by: Josh Elser <elserj@apache.org>
2019-05-28 15:06:37 -04:00
16f18d5c2d
HBASE-22148 Provide a LimitedPrivate(COPROC) API for setting Cell timestamp without copying.
...
Signed-off-by: Josh Elser <elserj@apache.org>
2019-05-28 12:18:48 -05:00
2c55bd9344
HBASE-22449 https everywhere in Maven metadata ( #247 )
2019-05-21 12:38:42 -07:00
b585dc42ba
HBASE-21467 Fix flaky test TestCoprocessorClassLoader.testCleanupOldJars
2019-05-06 09:45:51 -07:00
32462d5690
HBASE-22340 Corrupt KeyValue is silently ignored ( #207 )
2019-05-02 11:22:05 +02:00
018dadc3c1
HBASE-22054: Space Quota: Compaction is not working for super user in case of NO_WRITES_COMPACTIONS
...
Signed-off-by: Josh Elser <elserj@apache.org>
2019-04-30 13:41:08 -04:00
4862a596ef
HBASE-22083 move eclipse settings into a profile.
...
Signed-off-by: stack <stack@apache.org>
Conflicts:
hbase-backup/pom.xml
hbase-hadoop-compat/pom.xml
hbase-protocol/pom.xml
2019-04-25 14:38:38 -05:00
b117d5b4c2
HBASE-22307 Deprecated Preemptive Fail Fast
2019-04-25 14:20:49 +08:00
1bd5b5cf7b
HBASE-22250 The same constants used in many places should be placed in constant classes
...
Signed-off-by: stack <stack@apache.org>
2019-04-23 21:24:46 -07:00
4138fce370
HBASE-22292 PreemptiveFastFailInterceptor clean repeatedFailuresMap issue
...
Signed-off-by: stack <stack@apache.org>
2019-04-23 08:02:48 -07:00
c40e6e2339
HBASE-22231 Removed unused and '*' import
2019-04-23 12:53:52 +02:00
cadf321d53
HBASE-15560 W-TinyLFU based BlockCache
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-04-17 11:10:55 -07:00
f3532e4c26
HBASE-22235 OperationStatus.{SUCCESS|FAILURE|NOT_RUN} are not visible to 3rd party coprocessors
2019-04-16 13:19:56 -07:00
ae4bfabeaa
HBASE-21688 Address WAL filesystem issues
...
Amending-Author: Josh Elser <elserj@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2019-04-03 13:55:48 -04:00
a01c8e1dd6
HBASE-22053 Changed zookeeper URL from http://hadoop.apache.org to https://zookeeper.apache.org Replaced version 3.1.1 with 3.3.3 as 3.1.1 is deprecated.
...
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2019-03-28 00:31:24 +09:00
2c58dc503b
HBASE-22032 KeyValue validation should check for null byte array
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-common/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
2019-03-15 12:45:46 -07:00
d2296a4bad
HBASE-22044 Addendum for checkstyle.
2019-03-13 13:43:39 -05:00
09f9fc7c3e
HBASE-22044 Deprecate ByteBufferUtils; it will be IA.Private in HBase 3.0
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-03-13 13:39:44 -05:00
31b3716e84
HBASE-22001 Polish the Admin interface
...
Signed-off-by: stack <stack@apache.org>
2019-03-11 14:43:12 +08:00
32631f18f5
HBASE-21999 ADDENDUM unknown revisions are okay; make sure we indicate them.
...
(cherry picked from commit a7bbff170a
2019-03-07 13:38:55 -06:00
82618c439d
HBASE-21874 Bucket cache on Persistent memory
...
Signed-off-by: Anoop John <anoop.hbase@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit f67f207965
2019-03-07 08:53:31 -06:00
a5c26653f9
Revert "HBASE-21874 Bucket cache on Persistent memory"
...
This reverts commit f67f207965
2019-03-07 08:52:13 -06:00
f67f207965
HBASE-21874 Bucket cache on Persistent memory
...
Signed-off-by: Anoop John <anoop.hbase@gmail.com>, Sean Busbey
<busbey@apache.org>
2019-03-07 10:52:18 +05:30
41be1fbd8b
HBASE-21999 [DEBUG] Exit if git returns empty revision!
2019-03-06 16:45:55 -08:00
6de61d6bee
HBASE-21481 [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-03-01 23:42:11 +08:00
761aef6d9d
HBASE-20587 Replace Jackson with shaded thirdparty gson
...
Signed-off-by: Michael Stack <stack@apache.org>
2019-02-22 16:40:45 +08:00
f198311556
HBASE-21928 Deprecated HConstants.META_QOS
...
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-02-20 14:16:54 +08:00
dec5bf3087
HBASE-21900 Infinite loop in AsyncMetaRegionLocator if we can not get the location for meta
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-14 19:50:23 +08:00
a33cda4b03
HBASE-21727 Simplify documentation around client timeout - ADDENDUM
...
Add back HBaseConfiguration#getInt with deprecation
Signed-off-by: zhangduo <zhangduo@apache.org>
2019-02-05 18:01:00 +08:00
54a8cc87c6
HBASE-21829 Use FutureUtils.addListener instead of calling whenComplete directly
2019-02-03 15:29:20 +08:00
16665b6e93
HBASE-21799 Update branch-2 version to 2.3.0-SNAPSHOT
2019-01-29 21:53:21 +08:00
aae52f787c
HBASE-20215 - Rename CollectionUtils to ConcurrentMapUtils
2019-01-24 20:55:55 -08:00
67ba050fce
HBASE-21761 Align the methods in RegionLocator and AsyncTableRegionLocator
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-01-24 15:21:26 +08:00
d80556ec85
HBASE-21754 ReportRegionStateTransitionRequest should be executed in priority executor
2019-01-23 21:08:13 +08:00
4eba6b3656
HBASE-21750 Most of KeyValueUtil#length can be replaced by cell#getSerializedSize for better performance because the latter one has been optimized
2019-01-22 20:21:49 +08:00
ff272e8683
HBASE-21726 Add getAllRegionLocations method to AsyncTableRegionLocator
2019-01-19 11:14:40 +08:00
c353d93f69
HBASE-21727 Simplify documentation around client timeout
...
Signed-off-by: Michael Stack <stack@apache.org>
2019-01-16 11:41:40 +01:00
fdfbd9a21c
HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case
2019-01-16 09:47:27 +08:00
f02ac310d2
HBASE-21588 Procedure v2 wal splitting implementation
2019-01-08 17:26:58 +08:00
77ca660389
HBASE-17356 Add replica get support
2019-01-03 08:38:53 +08:00
fe27709e0c
HBASE-20136 TestKeyValue misses ClassRule and Category annotations
2018-12-23 10:17:55 -08:00
46ba4591a4
HBASE-21401 Sanity check when constructing the KeyValue
2018-12-21 18:50:27 +08:00
6ab0fbb1f1
HBASE-21255 [acl] Refactor TablePermission into three classes (Global, Namespace, Table)
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-11-15 11:45:24 +08:00
a765859131
HBASE-21473 RowIndexSeekerV1 may return cell with extra two \x00\x00 bytes which has no tags
2018-11-14 23:13:04 +08:00
98ad5ca6e5
Revert "HBASE-21401 Sanity check in BaseDecoder#parseCell"
...
This reverts commit b6d32e8a10
2018-11-12 11:42:04 +08:00
b6d32e8a10
HBASE-21401 Sanity check in BaseDecoder#parseCell
2018-11-08 20:14:03 +08:00
4f22397ad4
HBASE-21351 The force update thread may have race with PE worker when the procedure is rolling back
2018-11-03 08:24:25 +08:00
ca1b5503e1
HBASE-21292 IdLock.getLockEntry() may hang if interrupted
2018-10-18 14:41:42 -07:00
ad6ca7c95a
HBASE-20716: Changes the bytes[] conversion done in Bytes and ByteBufferUtils. Instead of doing check unsafe_aligned available everytime, choose the best converter at startup.
2018-10-17 21:04:09 -07:00
c67f7f14e2
HBASE-21320 [canary] Cleanup of usage and add commentary
...
Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
2018-10-16 22:19:34 -07:00
2b1716fd8e
HBASE-21256 Improve IntegrationTestBigLinkedList for testing huge data
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-10-12 11:00:03 +08:00
302b77697a
HBASE-20764 build broken when latest commit is gpg signed
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-10-08 19:50:39 -05:00
51854a41fc
HBASE-21227 Implement exponential retrying backoff for Assign/UnassignRegionHandler introduced in HBASE-21217
2018-09-26 20:54:06 +08:00
21fafbaf53
HBASE-20734 Colocate recovered edits directory with hbase.wal.dir
...
Amending-Author: Reid Chan <reidchan@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-09-26 11:49:21 +08:00
0f514ab75a
HBASE-21164 reportForDuty should do backoff rather than retry
...
Remove unused methods from Sleeper (its ok, its @Private).
Remove notion of startTime from Sleeper handling (it is is unused).
Allow passing in how long to sleep so can maintain externally.
In HRS, use a RetryCounter to calculate backoff sleep time for when
reportForDuty is failing against a struggling Master.
2018-09-25 11:32:15 -07:00
df5310fc1e
HBASE-21208 Bytes#toShort doesn't work without unsafe
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-09-25 10:27:08 +08:00
6616fd2e0f
HBASE-21191 Add a holding-pattern if no assign for meta or namespace (Can happen if masterprocwals have been cleared).
...
Add a check for hbase:meta being online before we go to read it.
If not online, move into a holding-pattern until rectified, probably
by external operator.
Incorporates bulk of patch made by Allan Yang over on HBASE-21035.
M hbase-common/src/main/java/org/apache/hadoop/hbase/util/RetryCounterFactory.java
Add a Constructor for case where retries are for ever.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Move stuff around so that the first hbase:meta read is the AM#loadMeta.
Previously, checking table state and/or favored nodes could end up
trying to read a meta that was not onlined holding up master startup.
Do similar for the namespace table. Adds new methods isMeta and
isNamespace which check that the regions/tables are online.. if not,
we wait logging with a back-off that assigns need to be run.
Signed-off-by: Allan Yang <allan163@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-09-16 21:43:48 -07:00
d40348e8cd
HBASE-21098 Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
...
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-09-11 16:31:56 -07:00
737ac48473
HBASE-21083 Introduce a mechanism to bypass the execution of a stuck procedure
2018-08-28 20:19:55 -07:00
linkaline
ravowlga123
openinx
Huon Wilson
stack
Jan Hentschel
Braavos
Wellington Chevreuil
Viraj Jasani
毛蛤丝
syedmurtazahassan
Peter Somogyi
Duo Zhang
Sean Busbey
Mingliang Liu
Sakthi
Wellington Chevreuil
Artem Ervits
Andrew Purtell
OrDTesters
李小保
zouxiangwen
Ben Manes
Vladimir Rodionov
subrat.mishra
Geoffrey Jacoby
ramkrishna
Reid Chan
Guanghao Zhang
Allan Yang
Jingyun Tian
Ted Yu
Sahil Aggarwal
Zephyr Guo
Mike Drob
Zach York
Chia-Ping Tsai
Tyler Mi