Commit Graph

1334 Commits

Author SHA1 Message Date
openinx 53522f36cc HBASE-22810 Initialize an separate ThreadPoolExecutor for taking/restoring snapshot (addendum - use the old config key) (#517) 2019-08-22 11:53:46 +08:00
Huon Wilson 8cbe20c382 HBASE-22841 Add more factory functions to TimeRange
These functions make it easier to possible to use
`org.apache.hadoop.hbase.client.Table.CheckAndMutateBuilder#timeRange`
with more interesting ranges, without being forced to use the
deprecated constructors.

Signed-off-by: huzheng <openinx@gmail.com>
2019-08-17 10:12:48 +08:00
openinx 6f9db4660a HBASE-22810 Initialize an separate ThreadPoolExecutor for taking/restoring snapshot (#486) 2019-08-15 11:22:08 +08:00
stack 225c6cd5bc HBASE-22777 Add a multi-region merge (for fixing overlaps)
Makes MergeTableRegionsProcedure do more than just two regions at a
time. Compatible as MTRP was done considering one day it'd do more than
two at a time.

Changes hardcoded assumption that merge parent regions are named
mergeA and mergeB in a column on the resultant region. Instead
can have N columns on the merged region, one for each parent
merged. Column qualifiers all being with 'merge'.

Most of code below is undoing the assumption that there are two
parents on a merge only.
2019-08-07 16:58:06 -07:00
Jan Hentschel 3971773189
HBASE-22790 Added deprecation versions for fields in ByteBuffAllocator
Signed-off-by: stack <stack@apache.org>
2019-08-05 09:01:03 +02:00
Braavos 89e2cc6281
HBASE-22677 Add unit tests for org.apache.hadoop.hbase.util.ByteRangeUtils and Classes
Co-authored-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-05 03:32:32 +02:00
stack 4565e9f43b HBASE-22723 Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta
Refactor of CatalogJanitor so it generates a
Report on the state of hbase:meta when it runs. Also
refactor so CJ runs even if RIT (previous it would
punt on running if RIT) so it can generate a 'Report'
on the interval regardless. If RIT, it just doesn't
go on to do the merge/split GC as it used to.

If report finds an issue, dump as a WARN message
to the master log.

Follow-on is to make the Report actionable/available
for the Master to pull when it goes to draw the hbck
UI page (could also consider shipping the Report as
part of ClusterMetrics?)

Adds new, fatter Visitor to CJ, one that generates
Report on each run keeping around more findings as
it runs.

Moved some methods around so class reads better;
previous methods were randomly ordered in the class.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Make a few handy methods public.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add utility as defaults on the Inteface; i.e. is this the first region
 in table, is it last, does a passed region come next, or does passed
 region overlap this region (added tests for this new stuff).

M hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
 Bugfix... handle case where buffer passed is null.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Lots of change, reorg., but mostly adding consistency checking
 to the visitor used scanning hbase:meta on a period and the
 generation of a Report on what the scan has found traversing
 hbase:meta. Added a main so could try the CatalogJanitor against
 a running cluster.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitorCluster.java
 Fat ugly test for CatalogJanitor consistency checking.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
 Add tests for new functionality in RI.

M hbase-shell/src/main/ruby/hbase/table.rb
 Bug fix for case where meta has a null regioninfo; scan was aborting.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-07-29 16:07:10 -07:00
Wellington Chevreuil f160dd3c10 HBASE-21773 rowcounter utility should respond to pleas for help
Signed-off-by: stack <stack@apache.org>
2019-07-26 21:29:48 +01:00
Viraj Jasani 3a72ccefbd
HBASE-22648 Snapshot TTL (#371)
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>

Conflicts:
	hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotDescriptionUtils.java
2019-07-22 15:08:20 -07:00
openinx 6fb03e63b6 HBASE-22663 The HeapAllocationRatio in WebUI is not accurate because all of the heap allocation will happen in another separated allocator named HEAP (#365) 2019-07-20 23:36:21 +08:00
毛蛤丝 dfebcbec38 HBASE-22700:incorrect timeout in recommended ZooKeeper configuration
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-07-17 22:12:07 +08:00
syedmurtazahassan 955e643e1d
HBASE-22586 Javadoc Warnings related to @param tag
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-07-13 16:56:53 +02:00
Peter Somogyi ed44b60cb1 HBASE-22651 ErrorProne issue in TestByteBufferArray
Signed-off-by: Michael Stack <stack@apache.org>
2019-07-02 16:37:46 +02:00
syedmurtazahassan 8fa7800389 HBASE-22572 Javadoc Warnings: @link reference not found (#306)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: stack <stack@apache.org>
2019-07-01 21:18:11 -07:00
openinx 345f80ea5f HBASE-20060 Add details of off heap memstore into book. (#334) 2019-06-27 16:54:48 +08:00
huzheng 6a3aa789d8 HBASE-22612 Address the final overview reviewing comments of HBASE-21879 2019-06-26 15:40:23 +08:00
openinx af0e23c359 HBASE-22547 Align the config keys and add document for offheap read in HBase Book. (#301) 2019-06-25 22:25:30 +08:00
openinx d34c4a0481 HBASE-22598 Deprecated the hbase.ipc.server.reservoir.initial.buffer.size & hbase.ipc.server.reservoir.initial.max for HBase2.x compatibility (#318) 2019-06-25 22:22:29 +08:00
openinx 3199175932 HBASE-22491 Separate the heap HFileBlock and offheap HFileBlock because the heap block won't need refCnt and save into prevBlocks list before shipping (#268) 2019-06-25 22:22:16 +08:00
huzheng 5ff317fe7b HBASE-22504 Optimize the MultiByteBuff#get(ByteBuffer, offset, len) 2019-06-25 22:22:04 +08:00
openinx 4dd728dfff HBASE-22483 It's better to use 65KB as the default buffer size in ByteBuffAllocator (#279) 2019-06-25 22:20:58 +08:00
huzheng 9e5fc2b379 HBASE-22435 Add a UT to address the HFileBlock#heapSize() in TestHeapSize 2019-06-25 22:14:57 +08:00
huzheng b00360731a HBASE-22412 Improve the metrics in ByteBuffAllocator 2019-06-25 22:14:50 +08:00
huzheng d7009d258c HBASE-21921 Notify users if the ByteBufAllocator is always allocating ByteBuffers from heap which means the increacing GC pressure 2019-06-25 22:14:36 +08:00
huzheng 39a6a357ff HBASE-21937 Make the Compression#decompress can accept ByteBuff as input 2019-06-25 22:11:51 +08:00
huzheng fc460136e0 HBASE-21957 Unify refCount of BucketEntry and refCount of hbase.nio.ByteBuff into one 2019-06-25 22:07:16 +08:00
huzheng bd8260c07e HBASE-22159 ByteBufferIOEngine should support write off-heap ByteBuff to the bufferArray 2019-06-25 21:28:02 +08:00
huzheng 081dc03e12 HBASE-22127 Ensure that the block cached in the LRUBlockCache offheap is allocated from heap 2019-06-25 21:23:54 +08:00
huzheng 0d294d23a9 HBASE-22005 Use ByteBuff's refcnt to track the life cycle of data block 2019-06-25 21:16:48 +08:00
huzheng c19592a971 HBASE-21916 Abstract an ByteBuffAllocator to allocate/free ByteBuffer in ByteBufferPool 2019-06-25 21:12:37 +08:00
Duo Zhang 974e1086d7 HBASE-22617 Recovered WAL directories not getting cleaned up (#330)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-06-25 17:32:39 +08:00
Sean Busbey 23a2f8abab HBASE-22615 Make TestChoreService more robust to timing
* phrase fudge factor "deltas" in terms of the original period
* increase the delta allowed for chore timing from 5% to 20%
* improve some assertions

Closes #328

Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Sakthi <sakthivel.azhaku@gmail.com>
(cherry picked from commit 9aee88e03a)
2019-06-22 00:29:04 -05:00
Mingliang Liu bc737d1ed9
HBASE-21284 Default limits for PressureAwareCompactionThroughputController are too low
This is a forward port HBASE-21000 from branch-1.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-06-20 12:44:14 -07:00
Jan Hentschel 6cff25cc23
HBASE-22344 Documented the deprecation of public and limited private APIs (#208)
Signed-off-by: stack <stack@apache.org>
2019-06-17 22:29:51 +02:00
Murtaza Hassan a2b22a6807
HBASE-22565 Javadoc Warnings: @see cannot be used in inline documentation 2019-06-13 23:14:58 +02:00
Sakthi 38139350a3 HBASE-22458: TestClassFinder fails when run on JDK11
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 0445186f1a)
2019-06-13 09:02:20 -05:00
Sakthi 35d3612f66 HBASE-22534 TestCellUtil fails when run on JDK11
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 9b23e1d00c)
2019-06-04 14:51:20 -05:00
Duo Zhang 1a2632b796 HBASE-22316 Record the stack trace for current thread in FutureUtils.get 2019-05-30 11:27:43 +08:00
Wellington Chevreuil ea41133e44 HBASE-22496 UnsafeAccess.unsafeCopy should not copy more than UNSAFE_COPY_THRESHOLD on each iteration
Change-Id: I259bc54a0a5b3474d3c455639c3e9cb1e95c8438

Signed-off-by: huzheng <openinx@gmail.com>
2019-05-30 10:21:43 +08:00
Artem Ervits e9e680f3d1 HBASE-22469 replace md5 checksum in saveVersion script with sha512
Signed-off-by: Josh Elser <elserj@apache.org>
2019-05-28 15:06:37 -04:00
Sean Busbey 16f18d5c2d HBASE-22148 Provide a LimitedPrivate(COPROC) API for setting Cell timestamp without copying.
Signed-off-by: Josh Elser <elserj@apache.org>
2019-05-28 12:18:48 -05:00
Andrew Purtell 2c55bd9344
HBASE-22449 https everywhere in Maven metadata (#247) 2019-05-21 12:38:42 -07:00
OrDTesters b585dc42ba HBASE-21467 Fix flaky test TestCoprocessorClassLoader.testCleanupOldJars 2019-05-06 09:45:51 -07:00
Peter Somogyi 32462d5690 HBASE-22340 Corrupt KeyValue is silently ignored (#207) 2019-05-02 11:22:05 +02:00
Sakthi 018dadc3c1 HBASE-22054: Space Quota: Compaction is not working for super user in case of NO_WRITES_COMPACTIONS
Signed-off-by: Josh Elser <elserj@apache.org>
2019-04-30 13:41:08 -04:00
Sean Busbey 4862a596ef HBASE-22083 move eclipse settings into a profile.
Signed-off-by: stack <stack@apache.org>

 Conflicts:
	hbase-backup/pom.xml
	hbase-hadoop-compat/pom.xml
	hbase-protocol/pom.xml
2019-04-25 14:38:38 -05:00
Duo Zhang b117d5b4c2 HBASE-22307 Deprecated Preemptive Fail Fast 2019-04-25 14:20:49 +08:00
李小保 1bd5b5cf7b HBASE-22250 The same constants used in many places should be placed in constant classes
Signed-off-by: stack <stack@apache.org>
2019-04-23 21:24:46 -07:00
zouxiangwen 4138fce370 HBASE-22292 PreemptiveFastFailInterceptor clean repeatedFailuresMap issue
Signed-off-by: stack <stack@apache.org>
2019-04-23 08:02:48 -07:00
Jan Hentschel c40e6e2339 HBASE-22231 Removed unused and '*' import 2019-04-23 12:53:52 +02:00