0b9dc14dbc
HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by increasing scanner timeout.
2018-01-11 10:31:45 -08:00
71a1192d67
HBASE-19751 Use RegionInfo directly instead of an identifier and a namespace when getting WAL
2018-01-11 15:47:34 +08:00
8bf6adbe75
HBASE-19749 Revisit logic of UserScanQueryMatcher#mergeFilterResponse method
2018-01-11 12:05:02 +08:00
b568cf4ebd
HBASE-19740 Repeated error message for NamespaceExistException
2018-01-10 18:25:45 -08:00
ee3accb370
Add more flexibility for input directory structure to LoadIncrementalHFiles
2018-01-10 13:37:14 -08:00
67a2c62a6a
HBASE-19748 TestRegionReplicaFailover and TestRegionReplicaReplicationEndpoint UT hangs
2018-01-10 18:17:35 +08:00
09c11b0ca4
HBASE-19743 Disable TestMemstoreLABWithoutPool
...
This is a revert of a revert, i.e., a reapplication only this time
I'm pushing w/ the right JIRA number
2018-01-09 18:21:38 -08:00
88864f3d5d
Revert "HBASE-19742 Disable TestMemstoreLABWithoutPool"
...
Revert because have wrong JIRA number.
This reverts commit 1454c44bcb
2018-01-09 18:21:20 -08:00
669b08f8ec
HBASE-19742 Disable TestMemstoreLABWithoutPool
2018-01-09 14:33:29 -08:00
81ea657ed1
HBASE-19483 Add proper privilege check for rsgroup commands addendum
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-09 07:58:20 -08:00
f458b89c05
HBASE-19139 Create Async Admin methods for Clear Block Cache
2018-01-09 14:43:01 +08:00
eb88b69845
HBASE-19719 Fix checkstyle issues
2018-01-09 13:11:01 +08:00
2d5267331e
HBASE-19711 TestReplicationAdmin.testConcurrentPeerOperations hangs
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
ec364d0d6c
HBASE-19707 Race in start and terminate of a replication source after we async start replicatione endpoint
2018-01-09 13:11:01 +08:00
01f1fc8284
HBASE-19636 All rs should already start work with the new peer change when replication peer procedure is finished
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
1a89793ee8
HBASE-19634 Add permission check for executeProcedures in AccessController
2018-01-09 13:11:01 +08:00
8eeedf0403
HBASE-19697 Remove TestReplicationAdminUsingProcedure
2018-01-09 13:11:01 +08:00
a5cfc51cbe
HBASE-19661 Replace ReplicationStateZKBase with ZKReplicationStorageBase
2018-01-09 13:11:01 +08:00
368db315a6
HBASE-19687 Move the logic in ReplicationZKNodeCleaner to ReplicationChecker and remove ReplicationZKNodeCleanerChore
2018-01-09 13:11:01 +08:00
d95ee41583
HBASE-19544 Add UTs for testing concurrent modifications on replication peer
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
d357920988
HBASE-19686 Use KeyLocker instead of ReentrantLock in PeerProcedureHandlerImpl
2018-01-09 13:11:01 +08:00
a85a215f36
HBASE-19623 Create replication endpoint asynchronously when adding a replication source
2018-01-09 13:11:01 +08:00
19707a85dd
HBASE-19633 Clean up the replication queues in the postPeerModification stage when removing a peer
2018-01-09 13:11:01 +08:00
f89920a60f
HBASE-19622 Reimplement ReplicationPeers with the new replication storage interface
2018-01-09 13:11:01 +08:00
62a4f5bb46
HBASE-19635 Introduce a thread at RS side to call reportProcedureDone
2018-01-09 13:11:01 +08:00
f4703c6ed3
HBASE-19617 Remove ReplicationQueues, use ReplicationQueueStorage directly
2018-01-09 13:11:01 +08:00
5fc90244a8
HBASE-19642 Fix locking for peer modification procedure
2018-01-09 13:11:01 +08:00
fdc907140b
HBASE-19592 Add UTs to test retry on update zk failure
2018-01-09 13:11:01 +08:00
1025388da6
HBASE-19630 Add peer cluster key check when add new replication peer
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
65eb91f84e
HBASE-19573 Rewrite ReplicationPeer with the new replication storage interface
2018-01-09 13:11:01 +08:00
c4fa568b47
HBASE-19599 Remove ReplicationQueuesClient, use ReplicationQueueStorage directly
2018-01-09 13:11:01 +08:00
5e6c303528
HBASE-19543 Abstract a replication storage interface to extract the zk specific code
2018-01-09 13:11:01 +08:00
8f5e54a456
HBASE-19525 RS side changes for moving peer modification from zk watcher to procedure
2018-01-09 13:11:01 +08:00
7afae59323
HBASE-19580 Use slf4j instead of commons-logging in new, just-added Peer Procedure classes
2018-01-09 13:11:01 +08:00
17762c60cc
HBASE-19520 Add UTs for the new lock type PEER
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
62496b5f36
HBASE-19564 Procedure id is missing in the response of peer related operations
2018-01-09 13:11:01 +08:00
7f4bd0d371
HBASE-19524 Master side changes for moving peer modification from zk watcher to procedure
2018-01-09 13:11:01 +08:00
f17198ff19
HBASE-19216 Implement a general framework to execute remote procedure on RS
2018-01-09 13:11:01 +08:00
7ddf79946d
HBASE-19483 Add proper privilege check for rsgroup commands
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-08 19:48:37 -08:00
c5277d5f88
HBASE-19729 UserScanQueryMatcher#mergeFilterResponse should return INCLUDE_AND_SEEK_NEXT_ROW when filterResponse is INCLUDE_AND_SEEK_NEXT_ROW
2018-01-09 07:19:40 +08:00
2509a150c0
HBASE-19731 TestFromClientSide#testCheckAndDeleteWithCompareOp and testNullQualifier are flakey
2018-01-08 10:35:36 -08:00
2af61718b1
HBASE-19541 Removed unnecessary semicolons in hbase-server
2018-01-08 16:43:43 +01:00
afc2cdbaff
HBASE-19684 BlockCacheKey toString Performance
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-08 17:06:21 +08:00
7378dad5a9
HBASE-19712 Fix TestSnapshotQuotaObserverChore#testSnapshotSize
2018-01-08 08:47:18 +08:00
5a66eb978c
HBASE-19696 Filter returning INCLUDE_AND_NEXT_COL doesn't skip remaining versions when scan has explicit columns (Ankit Singhal)
2018-01-07 07:50:00 -08:00
45e46bb242
HBASE-19709 Ensure that we don't set a poolSize of 0
...
In some situations, Runtime.getRuntime().getAvailableProcessors()
may return 0 which would result in calculatePoolSize returning 0
which will trigger an exception. Guard against this case.
Signed-off-by: Reid Chan <reidddchan@outlook.com>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-01-05 18:44:35 -05:00
a30d9fe8d9
HBASE-19651 Remove LimitInputStream
2018-01-05 16:59:29 -06:00
2bf2b5b2c1
Revert "Revert "Revert "HBASE-19651 Remove LimitInputStream"""
...
This reverts commit 032701c96c
2018-01-05 16:58:26 -06:00
032701c96c
Revert "Revert "HBASE-19651 Remove LimitInputStream""
...
This reverts commit e5f7030ded
2018-01-05 16:55:59 -06:00
e5f7030ded
Revert "HBASE-19651 Remove LimitInputStream"
...
Revert to put Beluga in place as author.
This reverts commit 032fdc53de
2018-01-05 16:55:01 -06:00
9cd0e1a490
HBASE-17436 Add column to master web UI for displaying region states when listing regions of a table.
...
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-01-05 14:32:27 -08:00
5ce9351129
HBASE-19483 Add proper privilege check for rsgroup commands - revert due to pending review comment
2018-01-05 12:06:31 -08:00
fc7736eb00
HBASE-19483 Add proper privilege check for rsgroup commands
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-05 11:18:07 -08:00
032fdc53de
HBASE-19651 Remove LimitInputStream
...
Signed-off-by: Beluga Behr <dam6923@gmail.com>
2018-01-05 08:31:11 -06:00
bff937a767
HBASE-19688 TimeToLiveProcedureWALCleaner should extends BaseLogCleanerDelegate
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-04 10:26:23 -08:00
8119acfca7
HBASE-19596 RegionMetrics/ServerMetrics/ClusterMetrics should apply to all public classes
2018-01-04 13:13:34 +08:00
338a74e737
HBASE-19490 Rare failure in TestRateLimiter
2018-01-04 10:46:43 +08:00
301062566a
HBASE-19613 Miscellaneous changes to WALSplitter.
...
* Use ArrayList instead LinkedList
* Use Apache Commons where appropriate
* Parameterize and improve logging
2018-01-03 18:30:10 -08:00
6e136f26bf
HBASE-18806 VerifyRep by snapshot need not to restore snapshot for each mapper
2018-01-04 10:10:03 +08:00
9a98bb4ce9
HBASE-19691 Removes Global(A) requirement for getClusterStatus
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-03 18:47:53 -05:00
d8ef30c5e1
HBASE-19654 Remove misleading and chatty debug message in ReplicationLogCleaner
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-03 06:58:32 -08:00
ab0e459693
HBASE-19620 Add UT to confirm the race in MasterRpcServices.getProcedureResult
2018-01-03 20:29:07 +08:00
d84c4ec29f
HBASE-19666 TestDefaultCompactSelection test failed
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-03 19:08:50 +08:00
a47afc84cd
HBASE-19675 Miscellaneous HStore Class Improvements.
...
* Use HashSet instead of List for a variable which is only used for lookups
* Remove logging code guards in favor of slf4j parameters
* Use CollectionsUtils.isEmpty() consistently
* Small check-style fixes
2018-01-02 15:16:53 -08:00
32e0107751
HBASE-19667 Get rid of MasterEnvironment#supportGroupCPs
2018-01-03 04:59:38 +08:00
f6f57d38f7
HBASE-19358 Improve the stability of splitting log when do fail over
...
Signed-off-by: Yu Li <liyu@apache.org>
2018-01-03 04:11:40 +08:00
9c2a35542f
HBASE-19676 CleanerChore logging improvements (BELUGA BEHR)
2018-01-02 06:26:29 -08:00
9ac95419dc
HBASE-19658: Fixing small problem and enabling test
2018-01-02 12:51:20 +02:00
6a0e6fefd3
HBASE-19671 Fix TestMultiParallel#testActiveThreadsCount
2018-01-02 16:57:14 +08:00
6708d54478
HBASE-19679 Superusers Logging and Data Structures (BELUGA BEHR)
2018-01-01 14:18:21 -08:00
73ab51e946
HBASE-19678 HBase Admin security capabilities should be represented as a Set - revert due to wrong issue
2018-01-01 14:16:01 -08:00
6c2aa4c9cc
HBASE-19678 HBase Admin security capabilities should be represented as a Set (BELUGA BEHR)
2017-12-31 11:31:44 -08:00
360d465a4a
HBASE-19677 Miscellaneous HFileCleaner Improvements (BELUGA BEHR)
2017-12-31 09:47:12 -08:00
0cd6050d09
HBASE-19672 Correct comments for default values of major compaction in SortedCompactionPolicy#getNextMajorCompactTime()
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-30 16:27:20 -08:00
0d0964aa6a
HBASE-19649 Use singleton feature for ImmutableSegment
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-30 06:34:45 +08:00
a21eb68f95
HBASE-8518 Get rid of hbase.hstore.compaction.complete setting
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-30 04:23:18 +08:00
e23f7afe57
HBASE-19428 Deprecate the compareTo(Row)
2017-12-29 20:03:39 +08:00
8d0da1a77f
HBASE-19282: Making CellChunkMap the default index (CellSet delegatee) for ImmutableSegments, when MSLAB is used.
...
In order to avoid additional user settings. If no MSLAB is requested the index is going to be CellArrayMap
Signed-off-by: Anastasia Braginsky <anastas@yahoo-inc.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-28 16:26:38 -08:00
b3a4fca48f
HBASE-19660 Up default retries from 10 to 15 and blocking store files limit from 10 to 16
2017-12-28 14:59:40 -08:00
2dae9d16e2
Revert "HBASE-19651 Remove LimitInputStream"
...
This reverts commit 28eaf715e8
2017-12-28 14:31:44 -08:00
28eaf715e8
HBASE-19651 Remove LimitInputStream
...
Signed-off-by: Beluga Behr <dam6923@gmail.com>
2017-12-28 13:04:41 -08:00
0c4b520d97
HBASE-19133 Transfer big cells or upserted/appended cells into
...
MSLAB upon flattening to CellChunkMap
Signed-off-by: Gali Sheffi <gsheffi@oath.com>
2017-12-28 10:51:44 -08:00
c3b4f788b1
HBASE-19552 find-and-replace thirdparty offset
2017-12-28 11:52:32 -06:00
ea7d5fc884
HBASE-19552 Update hbase-thirdparty version
...
Some manual cleanup of changing package names in pom files and getting
rid of the no-longer-needed netty system property.
This commit will break compilation, package renames in source code are
done in follow-on commits using straightforward find and replace.
's/org.apache.hadoop.hbase.shaded.com.google/org.apache.hbase.thirdparty.com.google/'
's/org.apache.hadoop.hbase.shaded.io.netty/org.apache.hbase.thirdparty.io.netty/'
2017-12-28 11:41:42 -06:00
8580ba7b6f
HBASE-19643 Need to update cache location when get error in AsyncBatchRpcRetryingCaller
2017-12-28 14:33:24 +08:00
53233be5a9
HBASE-19626 Rename Cell.DataType to Cell.Type
2017-12-28 13:04:29 +08:00
1050936ce7
HBASE-19652 Turn down CleanerChore logging; too chatty
2017-12-27 20:49:26 -08:00
2c65f039ed
HBASE-19647 Logging cleanups; emit regionname when RegionTooBusyException inside RetriesExhausted... make netty connect/disconnect TRACE-level; ADDENDUM
2017-12-27 14:14:58 -08:00
01b1f48ccd
HBASE-19644 add the checkstyle rule to reject the illegal imports
2017-12-28 04:10:42 +08:00
16cd916976
HBASE-19647 Logging cleanups; emit regionname when RegionTooBusyException inside RetriesExhausted... make netty connect/disconnect TRACE-level
2017-12-27 11:55:42 -08:00
6b39062e86
HBASE-19628 ByteBufferCell should extend ExtendedCell
2017-12-27 22:26:40 +08:00
467a4667d8
HBASE-19629 RawCell#getTags should return the Iterator<Tag> in order to avoid iterating through whole tag array at once
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-27 21:33:42 +08:00
0e85a880fb
HBASE-19615 CompositeImmutableSegment ArrayList Instead of LinkedList
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-27 10:04:53 +08:00
0e7fff5829
HBASE-19624 TestIOFencing hangs
2017-12-27 09:33:53 +08:00
7ce1943ef3
HBASE-19550 Wrap the cell passed via Mutation#add(Cell) to be of ExtendedCell
2017-12-26 16:39:51 +08:00
1556939236
HBASE-19621 Revisit the methods in ReplicationPeerConfigBuilder
2017-12-26 14:40:31 +08:00
2ce5dc8927
HBASE-19618 Remove replicationQueuesClient.class/replicationQueues.class config and remove table based ReplicationQueuesClient/ReplicationQueues implementation
2017-12-26 14:39:41 +08:00
27c56860da
HBASE-19608 Race in MasterRpcServices.getProcedureResult
2017-12-25 11:52:34 +08:00
2f25589422
HBASE-19602 Cleanup the usage of ReplicationAdmin from document
2017-12-23 18:55:54 +08:00
0dadb45ad7
HBASE-19576 Introduce builder for ReplicationPeerConfig and make it immutable
2017-12-23 18:55:29 +08:00
c0c802f62b
HBASE-19514 TestJMXListener now uses random port
...
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2017-12-22 18:40:58 +01:00
c0598dcb10
HBASE-19593 Possible NPE if wal is closed during waledit append.(Rajeshabbu)
2017-12-22 17:20:45 +05:30
448ba3a78f
HBASE-19496 Reusing the ByteBuffer in rpc layer corrupt the ServerLoad and RegionLoad
2017-12-22 18:54:25 +08:00
001bce54e7
HBASE-19590 Remove the duplicate code in deprecated ReplicationAdmin
2017-12-22 18:32:15 +08:00
8ec0aa0d70
HBASE-19589 New regions should always be added with state CLOSED (followup of HBASE-19530)
2017-12-21 22:57:53 -08:00
37bf54a516
HBASE--17248 fix javadoc in SimpleRegionNormalizer
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-21 21:35:08 -08:00
bcaf2fd20c
HBASE-19148 Reevaluate default values of configurations; minor ADDENDUM
2017-12-21 14:14:07 -08:00
4d6b928682
HBASE-19148 Reevaluate default values of configurations
...
Removed unused:
<name>hbase.fs.tmp.dir</name>
Added hbase.master.loadbalance.bytable
Edit of description text. Moved stuff around to put configs beside each
other.
M hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerCommandLine.java
Emit some hbase configs in log on startup.
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-21 14:07:16 -08:00
59baf12c03
HBASE-19575 add copy constructor to Mutation
2017-12-22 03:24:43 +08:00
35728acd21
HBASE-19578 MasterProcWALs cleaning is incorrect
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-21 09:38:25 -08:00
00b0288c96
HBASE-10092 Addendum fix TestAssignProcedure and TestWALEntrySinkFilter
2017-12-21 21:43:17 +08:00
9a07e07e55
HBASE-19559 Fix TestLogRolling.testLogRollOnDatanodeDeath
2017-12-21 15:20:04 +08:00
f572c4b80e
HBASE-10092 Move up on to log4j2
...
Changes:
- replaced commons-logging to slf4j everywhere
- log.XXX(Throwable) calls were replaced with log.XXX(t.toString(), t)
- log.XXX(Object) calls were replaced with log.XXX(Objects.toString(obj))
- log.fatal() calls were replaced with log.error(HBaseMarkers.FATAL, ...)
- programmatic log4j configuration was removed from the unit test
This commit does not affect the current logging configurations, because log4j
is still on the classpath. slf4j-log4j12 binds log4j to slf4j.
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-20 22:21:33 -08:00
7f938dd980
HBASE-19218 Master stuck thinking hbase:namespace is assigned after restart preventing intialization
...
Signed-off-by: Li Xiang <easyliangjob@gmail.com>
2017-12-20 21:47:10 -08:00
c811d7f965
HBASE-19542 fix TestSafemodeBringsDownMaster
2017-12-21 10:32:14 +08:00
4d71f4e32f
HBASE-19567 ClassNotFoundException:
...
org.apache.hadoop.hbase.KeyValue$RawBytesComparator starting 2.0.0 over
a 0.98.25 data.
2017-12-20 11:21:06 -08:00
084e324fb9
HBASE-19554 For debug: Modify HTU.waitUntilAllRegionsAssigned to handle the case where we do not have entries for the given table
2017-12-20 10:15:18 -08:00
55fefd4b5a
HBASE-19561 maxCacheSize in CacheEvictionStats can't be accumulated repeatedly When dealing with each region
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-20 07:05:05 -08:00
cae5c2706b
HBASE-19492 (addendum) Add EXCLUDE_NAMESPACE and EXCLUDE_TABLECFS support to replication peer config
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-12-20 20:49:05 +08:00
d2534fc570
HBASE-19468 FNFE during scans and flushes (Ram)
2017-12-20 11:09:12 +05:30
3c78ad5215
HBASE-19558 TestRegionsOnMasterOptions hack so it works reliably
2017-12-19 15:59:02 -08:00
70608acf28
HBASE-19556: Remove TestAssignmentManager#testGoodSplit, which no longer make sense
2017-12-19 13:55:05 -08:00
32f6fd41c2
HBASE-19494 Create simple WALKey filter that can be plugged in on the Replication Sink
...
Implement new WALEntrySinkFilter (as opposed to WALEntryFilter) and
specify the implmentation (with a no-param constructor) in config
using property hbase.replication.sink.walentrysinkfilter
Signed-off-by: wolfgang hoschek whoscheck@cloudera.com
2017-12-19 13:48:59 -08:00
dbe409e2c7
HBASE-19555 TestSplitTransactionOnCluster is flaky
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-19 09:25:57 -08:00
03e79b7994
HBASE-19492 Add EXCLUDE_NAMESPACE and EXCLUDE_TABLECFS support to replication peer config
2017-12-19 16:53:43 +08:00
7a7e55b601
HBASE-19532 AssignProcedure#COMPARATOR may produce incorrect sort order
2017-12-18 18:32:24 -08:00
b4056d267a
HBASE-19122 Suspect methods on Cell to be deprecated
2017-12-18 15:20:44 -08:00
9d0c7c6dfb
HBASE-19531 Remove needless volatile declaration
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-19 00:34:28 +08:00
c8bf03f5fb
HBASE-19521 HBase mob compaction needs to check hfile version
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-18 08:32:18 -08:00
62b591bdc7
HBASE-19546 TestMasterReplication.testCyclicReplication2 uses wrong assertion
2017-12-18 19:59:14 +08:00
988ea870ed
HBASE-18309 (addendum) fix NPE in LogCleaner
2017-12-17 17:33:09 +08:00
979767824d
HBASE-19522 The complete order may be wrong in AsyncBufferedMutatorImpl
2017-12-16 14:00:59 +08:00
59529a78f0
HBASE-19498 Fix findbugs and error-prone warnings in hbase-client (branch-2)
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-12-15 15:55:51 -08:00
20b42d2d70
HBASE-18352 Enable TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas disabled by Proc-V2 AM in HBASE-14614
...
Reenables the test. Adds facility to HBaseTestingUtility so
you can pass in ports a restarted cluster should use. This
is needed so retention of region placement, on which this
test depends, can come trigger (this is why it was broke
on AMv2 commit... region placement retention is done
different in AMv2).
2017-12-15 12:47:43 -08:00
89e2869e22
HBASE-19272 Deal with HBCK tests disabled by HBASE-14614 AMv2 when HBCK
...
works again...
I removed them
2017-12-15 12:19:49 -08:00
75f512bd71
HBASE-18838 Fix hadoop3 check-shaded-invariants
2017-12-15 11:19:47 -06:00
010012cbcb
HBASE-18946 Stochastic load balancer assigns replica regions to the same RS
...
Added new bulk assign createRoundRobinAssignProcedure to complement
the existing createAssignProcedure. The former asks the balancer for
target servers to set into the created AssignProcedures. The latter
sets no target server into AssignProcedure. When no target server
is specified, we make effort at assign-time at trying to deploy the
region to its old location if there was one.
The new round robin assign procedure creator does not do this. Use
the new round robin method on table create or reenabling offline
regions. Use the old assign in ServerCrashProcedure or in
EnableTable so there is a chance we retain locality.
Bulk preassigning passing all to-be-assigned to the balancer in one
go is good for ensuring good distribution especially when read
replicas in the mix.
The old assign was single-assign scoped so region replicas could
end up on the same server.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
Cleanup around forceNewPlan. Was confusing.
Added a Comparator to sort AssignProcedures so meta and system tables
come ahead of user-space tables.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Remove the forceNewPlan argument on createAssignProcedure. Didn't make
sense given we were creating a new AssignProcedure; the arg had no
effect.
(createRoundRobinAssignProcedures) Recast to feed all regions to the balancer in
bulk and to sort the return so meta and system tables take precedence.
Miscellaneous fixes including keeping the Master around until all
RegionServers are down, documentation on how assignment retention
works, etc.
2017-12-15 08:53:41 -08:00
11e82de8a1
HBASE-15536 Make AsyncFSWAL as our default WAL
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-12-15 20:45:29 +08:00
018a65b947
HBASE-19505 Disable ByteBufferPool by default at HM.
2017-12-15 15:43:10 +05:30
deba43b156
HBASE-19516 IntegrationTestBulkLoad and IntegrationTestImportTsv run into 'java.lang.RuntimeException: DistributedHBaseCluster@1bb564e2 not an instance of MiniHBaseCluster' (Ankit)
2017-12-14 18:37:09 -08:00
661491b56b
HBASE-19513 Fix the wrapped AsyncFSOutput implementation
2017-12-15 09:39:36 +08:00
6ab8ce9829
Revert "HBASE-18352 Enable TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas"
...
Pushed prematurely
This reverts commit 1a173f820b
2017-12-14 14:17:20 -08:00
70f02dbc7c
HBASE-19462 Deprecate all addImmutable methods in Put
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-14 14:15:54 -08:00
1a173f820b
HBASE-18352 Enable TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas
...
disabled by Proc-V2 AM in HBASE-14614
Reenables the test. Adds facility to HBaseTestingUtility so
you can pass in ports a restarted cluster should use. This
is needed so retention of region placement, on which this
test depends, can come trigger (this is why it was broke
on AMv2 commit... region placement retention is done
different in AMv2).
2017-12-14 14:13:55 -08:00
4a1c3b4210
HBASE-19267 Remove compiler-plugin mapping executions as it breaks Java8 detection
...
It seems like the original reason this execution filter was added is no
longer an issue for 2.0. Actually, these entries actually preclude
Eclipse from correctly using the Java8 source/target version that we
have specified (which creates numerous compilation errors in Eclipse)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-12-14 15:11:19 -05:00
2c9ef8a471
HBASE-19289 Add flag to disable stream capability enforcement
...
Signed-off-by: Josh Elser <elserj@apache.org>
2017-12-14 12:19:22 -06:00
d5aefbd2c7
HBASE-19511 Splits causes blocks to be cached again and so such blocks cannot be evicted from bucket cache (Ram)
...
Signed-off Duo Zhang, Anoop Sam John <palomino219@gmail.com, anoop.hbase@gmail.com>
2017-12-15 00:35:20 +05:30
fe316fe7ac
HBASE-19512 Move EventType and ExecutorType from hbase-client to hbase-server
2017-12-14 20:15:44 +08:00
7466e64abb
HBASE-19510 TestDistributedLogSplitting is flakey for AsyncFSWAL
2017-12-14 15:59:41 +08:00
d1fb987265
HBASE-19287 Revert "Revert "master hangs forever if RecoverMeta
...
send assign meta region request to target server fail""
This is a revert of a revert; i.e a reapplication with the
log message fixed up and some added javadoc.
This reverts commit 9ef115163b
2017-12-13 23:08:28 -08:00
9ef115163b
Revert "master hangs forever if RecoverMeta send assign meta region request to target server fail"
...
Reverting because missing issue JIRA #
This reverts commit d3aeaeffa4
2017-12-13 23:06:23 -08:00
d3aeaeffa4
master hangs forever if RecoverMeta send assign meta region request to target server fail
2017-12-13 22:29:19 -08:00
ba5f9ac380
HBASE-19503 Fix TestWALOpenAfterDNRollingStart for AsyncFSWAL
2017-12-14 09:41:12 +08:00
d28e126b29
HBASE-19508 ReadOnlyConfiguration throws exception if any Configuration in current context calls addDefautlResource; ADDENDUM -- fix license
2017-12-13 10:28:55 -08:00
736dd93c31
HBASE-19508 ReadOnlyConfiguration throws exception if any Configuration in current context calls addDefautlResource
...
Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
2017-12-13 10:23:50 -08:00
d0c2329aa8
HBASE-19456 RegionMover's region server hostname option is no longer case insensitive
...
Signed-off-by: Josh Elser <elserj@apache.org>
2017-12-13 12:03:14 -05:00
86043ef629
HBASE-19000 Group multiple block cache clear requests per server
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-13 07:47:09 -08:00
7f586995a8
HBASE-19502 Make sure we have closed all StoreFileScanner if we fail to open any StoreFileScanners
2017-12-13 21:30:08 +08:00
677c1f2c63
HBASE-19427 Add TimeRange support into Append to optimize for counters
2017-12-13 15:35:49 +08:00
7e5f3a516c
HBASE-19213 Align check and mutate operations in Table and AsyncTable
...
- Deprecates old checkAnd*() operations in Table
- Adds Table#CheckAndMutateBuilder and implements it in HTable
Commiter note: When committing the patch, noticed redundant {@inheritDoc} being added in HTable.
Removed new and olds ones.
2017-12-12 20:37:24 -08:00
11467ef111
HBASE-19394 Support multi-homing env for the publication of RS status with multicast (hbase.status.published) (Toshihiro Suzuki)
2017-12-12 07:38:15 -08:00
2e813f106f
HBASE-19493 Make TestWALMonotonicallyIncreasingSeqId also work with AsyncFSWAL
2017-12-12 20:56:58 +08:00
856ee283fa
HBASE-19371 Running WALPerformanceEvaluation against asyncfswal throws exceptions
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-11 13:38:02 -08:00
eb67ee0d0f
HBASE-19485 Minor improvement to TestCompactedHFilesDischarger (Ram)
...
Signed off by : Ted Yu <yuzhihong@gmail.com>
2017-12-11 22:53:46 +05:30
c98bab51de
HBASE-17425 Fixed deprecated calls in TestUpdateConfiguration
2017-12-09 14:42:43 +01:00
e880946f53
HBASE-19463 Make CPEnv#getConnection return a facade that throws Unsupported if CP calls #close
2017-12-09 20:20:18 +08:00
7092b814bd
HBASE-19457 Debugging flaky TestTruncateTableProcedure
...
- Adds debug logging for future ease
- Removes 60s timeout since testRecoveryAndDoubleExecutionPreserveSplits is only halfway after a minute.
- Adds some comments
- Logging change: Some places report "regionState=" while others just "state=".
State machine procs also have "state=" in their logs. Let me change all region related logging to "regionState=" so that
1) it's consistent everywhere, 2) more filtered results when searching through logs.
2017-12-08 17:25:16 -08:00
a5a77ae3d5
HBASE-19134 Make WALKey an Interface; expose Read-Only version to CPs
...
Created a new WALKey Interface and a WALKeyImpl. The WALKey Interface
is surfaced to Coprocessors and throughout most of the code base.
WALKeyImpl is used internally by WAL and by Replication which need
access to WALKey setters.
Methods that were deprecated in WALObserver because they were exposing
Private audience Classes have been undeprecated now we have WALKey.
Moved over to use SequenceId#getSequenceId throughout. Changed
SequenceId#getSequenceId removing the IOE.
2017-12-08 15:12:31 -08:00
033e64a8b1
HBASE-19454 Debugging TestDistributedLogSplitting#testThreeRSAbort
...
- Changed testThreeRSAbort to kill the RSs intead of aborting. Simple aborting will close the regions, we want extreme failure testing here.
- Adds some logging for easier debugging.
- Refactors TestDistributedLogSplitting to use standard junit rules.
2017-12-07 19:03:52 -08:00
66db0006eb
HBASE-19349 Introduce wrong version depencency of servlet-api jar
...
Move the hadoop-hdfs guava exclude in modules up to the top pom.
Looks like an exclude in a module is not additive but rather exclusive
blanking out the top level set of exclusions.
Tested by looking in lib dir of the built tarball.
2017-12-07 17:50:30 -08:00
428e5672e7
HBASE-19163 Maximum lock count exceeded from region server's batch processing
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-12-07 13:57:22 -08:00
f55e81e6c0
HBASE-19435 Reopen Files for ClosedChannelException in BucketCache
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-07 10:52:55 -08:00
638433358b
HBASE-19448 Replace StringBuffer with StringBuilder for hbase-server
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-07 22:45:50 +08:00
f90420b3a5
HBASE-19301 Provide way for CPs to create short circuited connection
...
with custom configurations; ADDENDUM -- adds warning to powerful new,
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
2017-12-07 06:37:00 -08:00
464e4e8058
HBASE-12444 Total number of requests overflow because it's int
2017-12-07 18:39:12 +08:00
ba4f9f8349
HBASE-19357 Bucket cache no longer L2 for LRU cache.
2017-12-07 15:03:53 +05:30
d2b1578b73
HBASE-19430 Remove the SettableTimestamp and SettableSequenceId
2017-12-07 13:28:02 +08:00
97d17ae195
HBASE-19447 INFO level logging of GetClusterStatus from HMaster is too chatty
2017-12-06 17:06:34 -08:00
4a2e8b852d
HBASE-19295 The Configuration returned by CPEnv should be read-only.
...
Adds a ReadOnlyConfiguration that delegates gets but throws exception
on sets/adds, etc.
2017-12-06 15:18:09 -08:00
00750fe79a
HBASE-15628 Implement an AsyncOutputStream which can work with any FileSystem implementation; ADDENDUM -- Change Log output
2017-12-06 11:35:54 -08:00
eabad8a91c
HBASE-19023 Replace hbase-server with hbase-mapreduce for HBase and MapReduce chapter
...
RowCounter and other related HBase's MapReduce classes have been moved
to hbase-mapreduce component by HBASE-18640, related chapter was
out-of-date and this fix replaced hbase-server with hbase-mapreduce
to correct those commands
Also this change moved RowCounter_Counters.properties to
hbase-mapreduce package as well
JIRA https://issues.apache.org/jira/browse/HBASE-19023
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-06 09:01:19 -08:00
27ed4d8add
HBASE-19417 Remove boolean return value from postBulkLoadHFile hook
2017-12-06 07:06:28 -08:00
ebd8841e0e
HBASE-18112 (addendum) fix the out-of-bounds index
2017-12-06 21:54:45 +08:00
75cdbb5700
HBASE-19410 Move zookeeper related UTs to hbase-zookeeper and mark them as ZKTests
2017-12-06 16:38:34 +08:00
d107e33fcd
Revert "HBASE-19357 Bucket cache no longer L2 for LRU cache."
...
This reverts commit d34e30cad3
2017-12-06 11:37:09 +05:30
d34e30cad3
HBASE-19357 Bucket cache no longer L2 for LRU cache.
2017-12-06 11:25:08 +05:30
ed60e4518d
HBASE-19323 Make netty engine default in hbase2
2017-12-05 15:29:29 -08:00
8e3714e772
HBASE-19426 Move has() and setTimestamp() to Mutation (Chia-Ping Tsai)
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-05 09:40:55 -08:00
1856237e2d
HBASE-19384 Results returned by preAppend hook in a coprocessor are replaced with
...
null from other coprocessor even on bypass
If 'bypass' is set by a Coprocessor, skip out on calling any subsequent
Coprocessors that might be chained to a bypassable method.
This patch restores some of the 'complete' behavior removed by
HBASE-19123 only 'bypass' now triggers 'complete'.
2017-12-05 08:06:41 -08:00
3c02d9681b
HBASE-19056 TestCompactionInDeadRegionServer is top of the flakies charts!
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-12-04 14:12:59 +08:00
7a5b078306
HBASE-19399 Purge curator dependency from hbase-client
2017-12-03 08:30:30 +08:00
fa9ab3955c
HBASE-19408 Remove WALActionsListener.Base
2017-12-03 01:05:10 +08:00
be4f158afd
HBASE-19339 Eager policy results in the negative size of memstore
2017-12-03 00:53:36 +08:00
d6a4a3dc33
HBASE-19396 Fix flaky test TestHTableMultiplexerFlushCache
2017-12-02 20:12:38 +08:00
c0da74f7c1
HBASE-19406 Fix CompactionRequest equals and hashCode
2017-12-01 19:15:11 -08:00
835d15bf97
HBASE-19285 Implements table-level latency histograms
...
For a egionserver's view of a table (the regions
that belong to a table hosted on a regionserver),
this change tracks the latencies of operations that
affect the regions for this table.
Tracking at the per-table level avoids the memory bloat
and performance impact that accompanied the previous
per-region latency metrics while still providing important
details for operators to consume.
Signed-Off-By: Andrew Purtell <apurtell@apache.org>
2017-12-01 17:46:09 -05:00
2e8bd0036d
HBASE-19392 TestReplicaWithCluster#testReplicaGetWithPrimaryAndMetaDown failure in master
...
Signed-off-by: Huaxiang Sun <hsun@apache.org>
2017-11-30 22:45:39 -08:00
49a9fe4883
HBASE-19344 improve asyncWAL by using Independent thread for netty #IO in FanOutOneBlockAsyncDFSOutput
2017-12-01 11:19:03 +08:00
cc3f804b07
HBASE-19326 Remove decommissioned servers from rsgroup
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-30 16:10:28 -08:00
e0dd26de8e
HBASE-19388 - Incorrect value is being set for Compaction Pressure in RegionLoadStats object inside HRegion class
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-11-30 08:47:23 -08:00
Apekshit Sharma
zhangduo
huzheng
tedyu
Rahul Gidwani
Michael Stack
Guangxu Cheng
Guanghao Zhang
Jan Hentschel
BELUGA BEHR
Chia-Ping Tsai
Josh Elser
Janos Gub
Reid Chan
Balazs Meszaros
Jingyun Tian
anastas
Xiang Li
brandboat
gsheffi
Mike Drob
Vasudevan
Rajeshbabu Chintaguntla
Daisuke
Peter Somogyi
Yi Liang
Yun Chi-Shih
QilinCao
anoopsamjohn
ramkrish86
Yi Liang
Sergey Soldatov
huaxiangsun
Zach York
Andrew Purtell
TAK LON WU
Harshal Deepakkumar Jain