Commit Graph

1404 Commits

Author SHA1 Message Date
Jan Hentschel 32434d1200
HBASE-22788 Removed deprecated methods from Bytes
Signed-off-by: stack <stack@apache.org>
2019-08-05 11:12:30 +02:00
Jan Hentschel f6ece8d8e3
HBASE-22790 Added deprecation versions for fields in ByteBuffAllocator
Signed-off-by: stack <stack@apache.org>
2019-08-05 08:59:46 +02:00
Reid Chan a62fdccd3b
HBASE-22609 [Docs] More detail documentation about 'hbase.server.thread.wakefrequency'
Signed-off-by: stack <stack@apache.org>
2019-08-05 14:51:00 +08:00
Braavos d0e5ea1b87 HBASE-22677 Add unit tests for org.apache.hadoop.hbase.util.ByteRangeUtils and Classes
Co-authored-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-05 03:29:51 +02:00
stack 0c80d5b42b HBASE-22723 Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta
Refactor of CatalogJanitor so it generates a
Report on the state of hbase:meta when it runs. Also
refactor so CJ runs even if RIT (previous it would
punt on running if RIT) so it can generate a 'Report'
on the interval regardless. If RIT, it just doesn't
go on to do the merge/split GC as it used to.

If report finds an issue, dump as a WARN message
to the master log.

Follow-on is to make the Report actionable/available
for the Master to pull when it goes to draw the hbck
UI page (could also consider shipping the Report as
part of ClusterMetrics?)

Adds new, fatter Visitor to CJ, one that generates
Report on each run keeping around more findings as
it runs.

Moved some methods around so class reads better;
previous methods were randomly ordered in the class.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Make a few handy methods public.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add utility as defaults on the Inteface; i.e. is this the first region
 in table, is it last, does a passed region come next, or does passed
 region overlap this region (added tests for this new stuff).

M hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
 Bugfix... handle case where buffer passed is null.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Lots of change, reorg., but mostly adding consistency checking
 to the visitor used scanning hbase:meta on a period and the
 generation of a Report on what the scan has found traversing
 hbase:meta. Added a main so could try the CatalogJanitor against
 a running cluster.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitorCluster.java
 Fat ugly test for CatalogJanitor consistency checking.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
 Add tests for new functionality in RI.

M hbase-shell/src/main/ruby/hbase/table.rb
 Bug fix for case where meta has a null regioninfo; scan was aborting.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-07-29 16:00:57 -07:00
Wellington Chevreuil 3708198817 HBASE-21773 rowcounter utility should respond to pleas for help
Signed-off-by: stack <stack@apache.org>
2019-07-26 21:19:17 +01:00
syedmurtazahassan d9493c539f HBASE-22603 Javadoc Warnings related to @link tag
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-07-23 14:14:12 +02:00
Viraj Jasani 9615c644f5 HBASE-22648 Snapshot TTL (#371)
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-07-22 15:03:44 -07:00
Viraj Jasani aa9679ddbc HBASE-22718 Removal of org.apache.hadoop.hbase.util.Counter (#396)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-07-22 13:56:08 +02:00
openinx 00075ea4fc
HBASE-22663 The HeapAllocationRatio in WebUI is not accurate because all of the heap allocation will happen in another separated allocator named HEAP (#365) 2019-07-20 23:30:08 +08:00
毛蛤丝 54b514d62a HBASE-22700:incorrect timeout in recommended ZooKeeper configuration
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-07-17 22:08:52 +08:00
syedmurtazahassan 8defea9561 HBASE-22586 Javadoc Warnings related to @param tag
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-07-13 16:46:18 +02:00
Braavos 53a8c2675f HBASE-22669 Add unit tests for org.apache.hadoop.hbase.util.Strings (#363)
These tests were written using Diffblue Cover.
2019-07-10 10:07:55 +02:00
Peter Somogyi b22459c37c
HBASE-22651 ErrorProne issue in TestByteBufferArray
Signed-off-by: Michael Stack <stack@apache.org>
2019-07-02 16:37:05 +02:00
syedmurtazahassan fabf2b8282 HBASE-22572 Javadoc Warnings: @link reference not found (#306)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: stack <stack@apache.org>
2019-07-01 21:15:00 -07:00
openinx 0198868531
HBASE-20060 Add details of off heap memstore into book. (#334) 2019-06-26 14:58:34 +08:00
Duo Zhang b42614295e
HBASE-22617 Recovered WAL directories not getting cleaned up (#330)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-06-25 16:21:24 +08:00
yuliangwan 841b319e4d HBASE-22611 Close the DataOutputStream object #325 2019-06-24 14:14:05 -07:00
Jan Hentschel 10032a11ba
HBASE-16002 Made constructors of DataType subclasses public
Signed-off-by: stack <stack@apache.org>
2019-06-24 22:32:18 +02:00
openinx 531d2902cb
HBASE-22612 Address the final overview reviewing comments of HBASE-21879 (#331) 2019-06-24 18:04:40 +08:00
openinx 2e414360bd HBASE-22547 Align the config keys and add document for offheap read in HBase Book. (#301) 2019-06-24 10:25:34 +08:00
openinx 686847cb79 HBASE-22598 Deprecated the hbase.ipc.server.reservoir.initial.buffer.size & hbase.ipc.server.reservoir.initial.max for HBase2.x compatibility (#318) 2019-06-24 10:25:34 +08:00
openinx d64c3a2611 HBASE-22491 Separate the heap HFileBlock and offheap HFileBlock because the heap block won't need refCnt and save into prevBlocks list before shipping (#268) 2019-06-24 10:25:33 +08:00
huzheng 97c1158421 HBASE-22504 Optimize the MultiByteBuff#get(ByteBuffer, offset, len) 2019-06-24 10:25:33 +08:00
openinx f83c919587 HBASE-22483 It's better to use 65KB as the default buffer size in ByteBuffAllocator (#279) 2019-06-24 10:25:33 +08:00
huzheng b1fee0ebdd HBASE-22435 Add a UT to address the HFileBlock#heapSize() in TestHeapSize 2019-06-24 10:25:33 +08:00
huzheng 7188a3bd3d HBASE-22412 Improve the metrics in ByteBuffAllocator 2019-06-24 10:25:33 +08:00
huzheng 97476ed2e0 HBASE-21921 Notify users if the ByteBufAllocator is always allocating ByteBuffers from heap which means the increacing GC pressure 2019-06-24 10:25:33 +08:00
huzheng 7dedb5625a HBASE-21937 Make the Compression#decompress can accept ByteBuff as input 2019-06-24 10:25:32 +08:00
huzheng 48aca4db30 HBASE-21957 Unify refCount of BucketEntry and refCount of hbase.nio.ByteBuff into one 2019-06-24 10:25:32 +08:00
huzheng 773c0d6635 HBASE-22159 ByteBufferIOEngine should support write off-heap ByteBuff to the bufferArray 2019-06-24 10:25:32 +08:00
huzheng d1eb6171f9 HBASE-22127 Ensure that the block cached in the LRUBlockCache offheap is allocated from heap 2019-06-24 10:25:32 +08:00
huzheng 35b818606f HBASE-22005 Use ByteBuff's refcnt to track the life cycle of data block 2019-06-24 10:25:31 +08:00
huzheng 532ebfb7a5 HBASE-21916 Abstract an ByteBuffAllocator to allocate/free ByteBuffer in ByteBufferPool 2019-06-24 10:25:31 +08:00
Sean Busbey 9aee88e03a HBASE-22615 Make TestChoreService more robust to timing
* phrase fudge factor "deltas" in terms of the original period
* increase the delta allowed for chore timing from 5% to 20%
* improve some assertions

Closes #328

Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Sakthi <sakthivel.azhaku@gmail.com>
2019-06-22 00:07:10 -05:00
Mingliang Liu 6d08ffcfc6
HBASE-21284 Default limits for PressureAwareCompactionThroughputController are too low
This is a forward port HBASE-21000 from branch-1.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-06-20 12:44:01 -07:00
zhangduo 5d872d3422 HBASE-21526 Use AsyncClusterConnection in ServerManager for getRsAdmin 2019-06-18 08:35:58 +08:00
zhangduo 30c131b5f7 HBASE-21515 Also initialize an AsyncClusterConnection in HRegionServer 2019-06-18 08:35:58 +08:00
Jan Hentschel 5da5deb59d
HBASE-22344 Documented the deprecation of public and limited private APIs (#208)
Signed-off-by: stack <stack@apache.org>
2019-06-17 22:02:23 +02:00
Murtaza Hassan 9ba7651889 HBASE-22565 Javadoc Warnings: @see cannot be used in inline documentation 2019-06-13 23:09:42 +02:00
Sakthi 0445186f1a HBASE-22458: TestClassFinder fails when run on JDK11
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-06-13 08:31:08 -05:00
Sakthi 9b23e1d00c HBASE-22534 TestCellUtil fails when run on JDK11
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-06-04 14:10:37 -05:00
Sean Busbey 51c49a1784 HBASE-22497 Mark HBase 1.2 line as EOM
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-06-03 06:55:58 -05:00
Duo Zhang 1be93fcaec HBASE-22316 Record the stack trace for current thread in FutureUtils.get 2019-05-30 11:10:06 +08:00
Wellington Chevreuil 25c6cd3a4f HBASE-22496 UnsafeAccess.unsafeCopy should not copy more than UNSAFE_COPY_THRESHOLD on each iteration
Change-Id: I259bc54a0a5b3474d3c455639c3e9cb1e95c8438

Signed-off-by: huzheng <openinx@gmail.com>
2019-05-30 10:07:39 +08:00
Artem Ervits 3a3857690a HBASE-22469 replace md5 checksum in saveVersion script with sha512
Signed-off-by: Josh Elser <elserj@apache.org>
2019-05-28 14:53:56 -04:00
Sean Busbey 6ddf893a34 HBASE-22148 Provide a LimitedPrivate(COPROC) API for setting Cell timestamp without copying.
Signed-off-by: Josh Elser <elserj@apache.org>
2019-05-28 12:18:30 -05:00
Andrew Purtell a30b186568
HBASE-22449 https everywhere in Maven metadata (#247) 2019-05-21 12:34:25 -07:00
Peter Somogyi 94122f6a74
HBASE-22340 Corrupt KeyValue is silently ignored (#207) 2019-05-02 11:17:28 +02:00
Sakthi e44fe4964a HBASE-22054: Space Quota: Compaction is not working for super user in case of NO_WRITES_COMPACTIONS
Signed-off-by: Josh Elser <elserj@apache.org>
2019-04-30 12:18:21 -04:00
Sean Busbey bf140acd20 HBASE-22083 move eclipse settings into a profile.
Signed-off-by: stack <stack@apache.org>
2019-04-25 14:17:18 -05:00
Duo Zhang 184568cadd HBASE-22307 Deprecated Preemptive Fail Fast 2019-04-25 14:19:58 +08:00
李小保 7617b71f00 HBASE-22250 The same constants used in many places should be placed in constant classes
Signed-off-by: stack <stack@apache.org>
2019-04-23 21:24:14 -07:00
zouxiangwen 6855143ba9 HBASE-22292 PreemptiveFastFailInterceptor clean repeatedFailuresMap issue
Signed-off-by: stack <stack@apache.org>
2019-04-23 07:56:59 -07:00
Michael Stack 1584d24a33
Merge pull request #100 from OrDTesters/fix-flaky-testCleanupOldJars
HBASE-21467 Fix flaky test TestCoprocessorClassLoader.testCleanupOldJars
2019-04-22 21:56:21 -07:00
Jan Hentschel f30d6c958a HBASE-22231 Removed unused and '*' imports 2019-04-20 22:03:32 +02:00
Ben Manes 8ec93ea193
HBASE-15560 W-TinyLFU based BlockCache
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-04-17 11:10:36 -07:00
Sayed Anisul Hoque 409523b110 HBASE-22248 Removed deprecated CollectionUtils 2019-04-17 10:07:21 +02:00
Andrew Purtell cc9d401015
HBASE-22235 OperationStatus.{SUCCESS|FAILURE|NOT_RUN} are not visible to 3rd party coprocessors 2019-04-16 13:19:59 -07:00
subrat.mishra f22b8ade63 HBASE-22053 Addendum changed zookeeper version from 3.3.3 to 3.4.10. 2019-03-28 08:09:47 -07:00
subrat.mishra 53f449c47e HBASE-22053 Changed zookeeper URL from http://hadoop.apache.org to https://zookeeper.apache.org Replaced version 3.1.1 with 3.3.3 as 3.1.1 is deprecated.
Signed-off-by: Toshihiro Suzuki <brfrn169@gmail.com>
2019-03-28 00:30:54 +09:00
Geoffrey Jacoby d4c37778ee
HBASE-22032 KeyValue validation should check for null byte array
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-03-15 12:34:26 -07:00
Duo Zhang 3c5761e365 HBASE-22002 Remove the deprecated methods in Admin interface
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-03-14 21:42:23 +08:00
Sean Busbey 3f1498ddb4 HBASE-22044 ByteBufferUtils should not be IA.Public API
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-03-13 13:39:22 -05:00
zhangduo 10ca598004 HBASE-22001 Polish the Admin interface
Signed-off-by: stack <stack@apache.org>
2019-03-11 14:38:31 +08:00
Sean Busbey a7bbff170a HBASE-21999 ADDENDUM unknown revisions are okay; make sure we indicate them. 2019-03-07 11:51:40 -06:00
ramkrishna fd6bc5d900 HBASE-21874 Bucket cache on Persistent memory
Signed-off-by: Anoop John <anoop.hbase@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 763202d48e)
2019-03-07 08:50:08 -06:00
Sean Busbey d9793c311c Revert "HBASE-21874 Bucket cache on Persistent memory"
This reverts commit 763202d48e.

bad signed-off-by line
2019-03-07 08:49:41 -06:00
stack 0d69f84bd2 HBASE-21999 [DEBUG] Exit if git returns empty revision! 2019-03-06 16:46:23 -08:00
ramkrishna 763202d48e HBASE-21874 Bucket cache on Persistent memory
Signed-off-by: Anoop John <anoop.hbase@gmail.com>, Sean Busbey
<busbey@apache.org>
2019-03-06 22:37:54 +05:30
Reid Chan f748e489c5 HBASE-21481 [acl] Superuser's permissions should not be granted or revoked by any non-su global admin
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-03-01 23:37:28 +08:00
Duo Zhang f0032c9255 HBASE-20587 Replace Jackson with shaded thirdparty gson
Signed-off-by: Michael Stack <stack@apache.org>
2019-02-22 16:24:51 +08:00
zhangduo e984515b74 HBASE-21928 Deprecated HConstants.META_QOS
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-02-20 14:16:40 +08:00
Duo Zhang 8bed65fa01 HBASE-21900 Infinite loop in AsyncMetaRegionLocator if we can not get the location for meta
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-02-14 19:50:02 +08:00
zhangduo 2b1b79f08b HBASE-21829 Use FutureUtils.addListener instead of calling whenComplete directly 2019-02-03 15:24:54 +08:00
Wellington Chevreuil 54093b0d1f HBASE-20215 - Rename CollectionUtils to ConcurrentMapUtils 2019-01-24 20:55:15 -08:00
zhangduo 416b70f461 HBASE-21761 Align the methods in RegionLocator and AsyncTableRegionLocator
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-01-24 15:21:11 +08:00
Allan Yang b5619a2a26 HBASE-21754 ReportRegionStateTransitionRequest should be executed in priority executor 2019-01-23 20:55:00 +08:00
huzheng 35ed5d6c39 HBASE-21750 Most of KeyValueUtil#length can be replaced by cell#getSerializedSize for better performance because the latter one has been optimized 2019-01-22 19:58:03 +08:00
Duo Zhang 882bd564f4 HBASE-21726 Add getAllRegionLocations method to AsyncTableRegionLocator 2019-01-19 11:14:35 +08:00
Peter Somogyi 51c58e083c HBASE-21727 Simplify documentation around client timeout
Signed-off-by: Michael Stack <stack@apache.org>
2019-01-16 11:32:46 +01:00
huzheng 15031c5bd5 HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case 2019-01-16 09:41:02 +08:00
tianjingyun 281d6429e5 HBASE-21588 Procedure v2 wal splitting implementation 2019-01-07 15:58:15 +08:00
zhangduo db66e6cc9e HBASE-17356 Add replica get support 2019-01-03 08:38:20 +08:00
huzheng 12786f80c1 HBASE-21401 Sanity check when constructing the KeyValue 2018-12-21 18:01:35 +08:00
zhangduo 1acbd36c90 HBASE-21154 Remove hbase:namespace table; fold it into hbase:meta 2018-11-26 14:51:56 +08:00
Reid Chan 130057f137 HBASE-21255 [acl] Refactor TablePermission into three classes (Global, Namespace, Table)
Signed-off-by: Michael Stack <stack@apache.org>
2018-11-15 11:34:16 +08:00
huzheng a8b642067d HBASE-21473 RowIndexSeekerV1 may return cell with extra two \x00\x00 bytes which has no tags 2018-11-14 23:00:12 +08:00
huzheng 362b5dd259 Revert "HBASE-21401 Sanity check in BaseDecoder#parseCell"
This reverts commit f17382792f.
2018-11-12 11:37:46 +08:00
OrDTesters 3af7d02d1d HBASE-21467 Fix flaky test TestCoprocessorClassLoader.testCleanupOldJars 2018-11-11 13:13:44 -06:00
maoling 5e84997f2f HBASE-13468 hbase.zookeeper.quorum supports ipv6 address
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-11-10 06:09:29 -08:00
huzheng f17382792f HBASE-21401 Sanity check in BaseDecoder#parseCell 2018-11-08 20:07:04 +08:00
zhangduo 62fe365934 HBASE-21351 The force update thread may have race with PE worker when the procedure is rolling back 2018-11-03 08:24:11 +08:00
Allan Yang 05d22ed960
HBASE-21292 IdLock.getLockEntry() may hang if interrupted 2018-10-18 14:42:10 -07:00
Sahil Aggarwal e520399201
HBASE-20716: Changes the bytes[] conversion done in Bytes and ByteBufferUtils. Instead of doing check unsafe_aligned available everytime, choose the best converter at startup. 2018-10-17 21:04:27 -07:00
Michael Stack 8cc56bd18c
HBASE-21320 [canary] Cleanup of usage and add commentary
Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
2018-10-16 22:20:00 -07:00
Zephyr Guo da63ebb2c0 HBASE-21256 Improve IntegrationTestBigLinkedList for testing huge data
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-10-12 10:59:35 +08:00
Mike Drob c9213f752e
HBASE-20764 build broken when latest commit is gpg signed
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-10-08 19:49:36 -05:00
zhangduo 71be251ef2 Revert "HBASE-21188 Print heap and gc informations in our junit ResourceChecker"
This reverts commit dc1dedb073.
2018-09-28 21:03:24 +08:00
zhangduo d7e08317d2 HBASE-21227 Implement exponential retrying backoff for Assign/UnassignRegionHandler introduced in HBASE-21217 2018-09-26 20:53:56 +08:00