11193d7cc1
HBASE-21590 Optimize trySkipToNextColumn in StoreScanner a bit.
2018-12-13 11:56:39 -08:00
9b139ca011
HBASE-21582 If call HBaseAdmin#snapshotAsync but forget call isSnapshotFinished, then SnapshotHFileCleaner will skip to run every time
2018-12-12 21:53:37 +08:00
b8405bfea4
HBASE-21568 Use CacheConfig.DISABLED where we don't expect to have blockcache running
...
This includes removing the "old way" of disabling blockcache in favor of the
new API.
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-12-11 11:37:35 -05:00
9981e3f843
HBASE-21570 Add write buffer periodic flush support for AsyncBufferedMutator
2018-12-11 15:29:39 +08:00
0094335c52
HBASE-21560 Return a new TableDescriptor for MasterObserver#preModifyTable to allow coprocessor modify the TableDescriptor
2018-12-08 16:03:30 +08:00
7b5b99e243
HBASE-21554 Show replication endpoint classname for replication peer on master web UI
2018-12-07 16:57:52 +08:00
f86d311f76
HBASE-21413 Empty meta log doesn't get split when restart whole cluster
...
Signed-off-by: stack <stack@apache.org>
2018-12-06 21:20:55 -08:00
2ebf5ab02f
Backport "HBASE-21126 Add ability for HBase Canary to ignore a configurable number of ZooKeeper down nodes" to branch-2.0
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-12-06 21:20:51 -08:00
5cb8c3e9c7
HBASE-21559 The RestoreSnapshotFromClientTestBase related UT are flaky
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-12-07 08:25:49 +08:00
1a1a65b565
HBASE-21551 Memory leak when use scan with STREAM at server side
2018-12-06 11:20:38 +08:00
96f8e0cbe9
HBASE-21534 TestAssignmentManager is flakey
2018-12-06 09:54:18 +08:00
7e8ce1c5cc
HBASE-21550 Add a new method preCreateTableRegionInfos for MasterObserver which allows CPs to modify the TableDescriptor
2018-12-06 08:30:37 +08:00
5648337545
HBASE-21541 Move MetaTableLocator.verifyRegionLocation to hbase-rsgroup module
2018-12-04 22:27:49 +08:00
4118bc688e
HBASE-21479 Individual tests in TestHRegionReplayEvents class are failing
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-12-01 09:48:19 +01:00
9d5df33be5
HBASE-21518 TestMasterFailoverWithProcedures is flaky
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-11-30 09:13:38 +01:00
1be42994a5
HBASE-21504 If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
2018-11-30 10:03:46 +08:00
ad8fbf3fd3
HBASE-18735 Provide an option to kill a MiniHBaseCluster without waiting on shutdown
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-11-29 17:05:04 -05:00
aaafafe63b
HBASE-21300 Fix the wrong reference file path when restoring snapshots for tables with MOB columns
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-11-29 13:57:33 +08:00
5688944721
HBASE-21517 Move the getTableRegionForRow method from HMaster to TestMaster
2018-11-28 09:41:45 +08:00
3c043440a8
HBASE-21507 Compaction failed when execute AbstractMultiFileWriter.beforeShipped() method
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
2018-11-27 20:58:24 +08:00
beeb38d4dc
Revert "HBASE-21507 Compaction failed when execute AbstractMultiFileWriter.beforeShipped() method"
...
This reverts commit 6aa57f71f0
2018-11-27 20:58:14 +08:00
6aa57f71f0
HBASE-21507 Compaction failed when execute AbstractMultiFileWriter.beforeShipped() method
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
2018-11-27 20:31:42 +08:00
958e84d023
HBASE-21498 Master OOM when SplitTableRegionProcedure new CacheConfig and instantiate a new BlockCache
2018-11-27 14:23:27 +08:00
33c4da5558
HBASE-21508 Ignore the reportRegionStateTransition call from a dead server
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-11-26 14:32:33 +08:00
7f02645741
HBASE-21511 Remove in progress snapshot check in SnapshotFileCache#getUnreferencedFiles
2018-11-25 18:22:53 -08:00
1731666b43
HBASE-21387 Addendum fix TestSnapshotFileCache
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-11-24 17:00:31 +08:00
cc26c8716f
HBASE-21387 Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files
2018-11-23 11:51:58 +08:00
248b8a6f56
HBASE-21503 Replication normal source can get stuck due potential race conditions between source wal reader and wal provider initialization threads.
...
Found and analysed by Wellington Chevreuil
2018-11-21 17:59:06 +08:00
d590d6e472
HBASE-21034 Add new throttle type: read/write capacity unit
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-11-21 09:50:07 +08:00
6a64811f44
HBASE-21490 WALProcedure may remove proc wal files still with active procedures
...
Signed-off-by: Allan Yang <allan163@apache.org>
2018-11-19 08:21:09 -08:00
7d16cf0977
HBASE-21480 Taking snapshot when RS crashes prevent we bring the regions online
2018-11-18 21:07:27 +08:00
37540b541d
HBASE-21485 Add more debug logs for remote procedure execution
2018-11-16 14:47:24 +08:00
6ab0fbb1f1
HBASE-21255 [acl] Refactor TablePermission into three classes (Global, Namespace, Table)
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-11-15 11:45:24 +08:00
17055cdc70
HBASE-21472 Should not persist the dispatched field for RegionRemoteProcedureBase
2018-11-15 09:41:43 +08:00
a765859131
HBASE-21473 RowIndexSeekerV1 may return cell with extra two \x00\x00 bytes which has no tags
2018-11-14 23:13:04 +08:00
fbf5e9e0c4
HBASE-21465 Retry on reportRegionStateTransition can lead to unexpected errors
2018-11-14 18:55:27 +08:00
a44515aea7
Revert "HBASE-21465 Retry on reportRegionStateTransition can lead to unexpected errors"
...
This reverts commit a27eea0807
2018-11-13 22:24:02 +08:00
f8970bdf04
Revert "HBASE-21465 Addendum change TestReportRegionStateTransitionRetry to MediumTests"
...
This reverts commit 3153aae128
2018-11-13 22:23:25 +08:00
3153aae128
HBASE-21465 Addendum change TestReportRegionStateTransitionRetry to MediumTests
2018-11-13 14:59:48 +08:00
a27eea0807
HBASE-21465 Retry on reportRegionStateTransition can lead to unexpected errors
2018-11-13 11:52:12 +08:00
ea3b2dfaeb
HBASE-21463 The checkOnlineRegionsReport can accidentally complete a TRSP
2018-11-13 11:17:52 +08:00
61f1d9735b
HBASE-21466 WALProcedureStore uses wrong FileSystem if wal.dir is not under rootdir
2018-11-12 17:02:45 -08:00
98ad5ca6e5
Revert "HBASE-21401 Sanity check in BaseDecoder#parseCell"
...
This reverts commit b6d32e8a10
2018-11-12 11:42:04 +08:00
f1d9f59bfe
HBASE-21376 Add some verbose log to MasterProcedureScheduler
2018-11-10 17:17:24 +08:00
6f602db5f2
HBASE-21410 A helper page that help find all problematic regions and procedures
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-11-09 14:02:57 +08:00
f936a1045d
Revert "HBASE-21410 A helper page that help find all problematic regions and procedures"
...
This reverts commit a2f650a0ec
2018-11-09 14:02:38 +08:00
a2f650a0ec
HBASE-21410 A helper page that help find all problematic regions and procedures
2018-11-09 13:56:43 +08:00
13b68abb58
HBASE-21439 RegionLoads aren't being used in RegionLoad cost functions
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
2018-11-08 14:08:52 -08:00
dcdebbffdc
HBASE-20604 ProtobufLogReader#readNext can incorrectly loop to the same position in the stream until the the WAL is rolled
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-11-08 10:22:11 -08:00
b6d32e8a10
HBASE-21401 Sanity check in BaseDecoder#parseCell
2018-11-08 20:14:03 +08:00
399fbb2aa6
HBASE-21247 Custom Meta WAL Provider doesn't default to custom WAL Provider whose configuration value is outside the enums in Providers
2018-11-06 12:35:31 -08:00
d544b7adea
HBASE-21421 Do not kill RS if reportOnlineRegions fails
2018-11-06 14:56:33 +08:00
a0bba8db46
HBASE-21425 2.1.1 fails to start over 1.x data; namespace not assigned
2018-11-03 09:45:57 -07:00
4f22397ad4
HBASE-21351 The force update thread may have race with PE worker when the procedure is rolling back
2018-11-03 08:24:25 +08:00
1bd98bb608
HBASE-21407 Resolve NPE in backup Master UI
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-11-02 11:46:47 -07:00
943f65f3b5
HBASE-21422 NPE in TestMergeTableRegionsProcedure.testMergeWithoutPONR
2018-11-02 20:55:36 +08:00
3d3dd04651
HBASE-19682 Use Collections.emptyList() For Empty List Values
2018-11-01 08:59:14 -07:00
b9c9f1d61f
HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService (addendum for branch-2)
2018-11-01 18:05:01 +08:00
2d0810a006
HBASE-21388 No need to instantiate MemStoreLAB for master which not carry table
2018-11-01 16:20:49 +08:00
857abfb082
HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-29 20:56:11 -07:00
eb0f9e15d1
HBASE-21375 Revisit the lock and queue implementation in MasterProcedureScheduler
2018-10-29 20:01:53 +08:00
6fbd70a611
HBASE-21325 Force to terminate regionserver when abort hang in somewhere
2018-10-29 10:12:21 +08:00
4ded55702d
HBASE-21391 RefreshPeerProcedure should also wait master initialized before executing
2018-10-26 21:45:03 +08:00
f71768308c
HBASE-21385 HTable.delete request use rpc call directly instead of AsyncProcess
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-25 16:22:18 -07:00
8c1b534ee8
HBASE-21372) Set hbase.assignment.maximum.attempts to Long.MAX
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Allan Yang <allan163@apache.org>
2018-10-24 09:07:24 -07:00
f3da04a552
HBASE-21349 Do not run CatalogJanitor or Nomalizer when cluster is shutting down
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-23 14:44:45 -07:00
390c3227e2
HBASE-21342 FileSystem in use may get closed by other bulk load call in secure bulkLoad
...
Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
2018-10-23 15:52:05 -05:00
d4ce9e534e
HBASE-21338 Warn if balancer is an ill-fit for cluster size
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-23 13:23:03 -07:00
bf2ce5488d
HBASE-21356 bulkLoadHFile API should ensure that rs has the source hfile's write permissionls
2018-10-23 10:25:06 +08:00
328555a374
HBASE-21336 Addendum remove unused code in HBTU
2018-10-22 20:27:11 +08:00
ff2b332aef
HBASE-21355 (addendum) replace the expensive reload storefiles with reading the merge result of compacted storefiles and current storefiles
2018-10-22 19:33:36 +08:00
0cefe7312c
HBASE-21336 Simplify the implementation of WALProcedureMap
2018-10-22 18:36:16 +08:00
69e65f1bd7
HBASE-21334 TestMergeTableRegionsProcedure is flakey
2018-10-22 14:18:59 +08:00
6dce9cd218
HBASE-21355 HStore's storeSize is calculated repeatedly which causing the confusing region split
2018-10-22 10:21:57 +08:00
46fe6c0e92
HBASE-21200 Memstore flush doesn't finish because of seekToPreviousRow() in memstore scanner.
2018-10-20 08:35:41 +09:00
444f15785c
HBASE-21073 Redo concept of maintenance mode
...
Instead of being an ephemeral state set by hbck, maintenance mode is now
an explicit toggle set by either configuration property or environment
variable. In maintenance mode, master will host system tables and not
assign any user-space tables to RSs. This gives operators the ability to
affect repairs to meta table with fewer moving parts.
2018-10-19 10:04:35 -05:00
4a609db30c
HBASE-21269 Forward-port HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign
2018-10-18 06:22:22 -07:00
20a9b8ad00
HBASE-21055 NullPointerException when balanceOverall() but server balance info is null
...
Signed-off-by: huzheng <openinx@gmail.com>
2018-10-18 14:06:08 +08:00
d7d3beb6bc
HBASE-21330 ReopenTableRegionsProcedure will enter an infinite loop if we schedule a TRSP at the same time
2018-10-18 11:34:43 +08:00
3563f48900
HBASE-21281 Update bouncycastle dependency - addendum adds dependency for hbase-server module
2018-10-17 16:00:28 -07:00
f32d3e1e2c
HBASE-21327 Fix minor logging issue where we don't report servername if no associated SCP
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-10-17 09:38:26 -07:00
c67f7f14e2
HBASE-21320 [canary] Cleanup of usage and add commentary
...
Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
2018-10-16 22:19:34 -07:00
1a0e1039a2
HBASE-21311 Split TestRestoreSnapshotFromClient
2018-10-17 11:18:34 +08:00
6511dae07a
HBASE-21263 Mention compression algorithm along with other storefile details
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
2018-10-16 12:47:11 -07:00
26292ab3d6
HBASE-21310 Split TestCloneSnapshotFromClient
2018-10-16 15:33:43 +08:00
c79927bc22
HBASE-21278 Do not rollback successful sub procedures when rolling back a procedure
2018-10-16 15:12:55 +08:00
3097bbc730
HBASE-21266 Not running balancer because processing dead regionservers, but empty dead rs list
2018-10-15 22:28:16 -07:00
5a6b68012d
HBASE-21290 No need to instantiate BlockCache for master which not carry table
2018-10-15 17:25:07 +08:00
287697bfdf
HBASE-21260 The whole balancer plans might be aborted if there are more than one plans to move a same region
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-10-15 15:54:25 +08:00
f01ea9b094
HBASE-21309 Increase the waiting timeout for TestProcedurePriority
2018-10-15 15:27:01 +08:00
9d6afa9d72
HBASE-21299 List counts of actual region states in master UI tables section
2018-10-12 11:02:16 -07:00
2ab7e13dfa
HBASE-21289 Remove the log "'hbase.regionserver.maxlogs' was deprecated." in AbstractFSWAL
2018-10-12 20:38:19 +08:00
9da4c1393d
HBASE-21254 Need to find a way to limit the number of proc wal files
2018-10-12 11:05:21 +08:00
33d72e4db4
HBASE-21281 Upgrade bouncycastle to latest
...
BC 1.47 introduced some incompatible API changes which came in via
a new Maven artifact. We don't use any changed API in HBase. This
also removes some unnecessary dependencies on bcprov in other
modules (presumably, they are vestiges)
Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Ted Yu <tedyu@apache.org>
2018-10-11 10:53:02 -04:00
81adb704c8
HBASE-21287 Allow configuring test master initialization wait time.
2018-10-11 09:50:37 -05:00
91aceeb28d
HBASE-21251 Refactor RegionMover
2018-10-10 15:14:57 +08:00
701fe7a0c8
HBASE-21280 Add anchors for each heading in UI
...
Signed-off-by: Ted Yu <tedyu@apache.org>
2018-10-09 22:49:54 -07:00
17bd935301
HBASE-21250 Addendum remove unused modification in hbase-server module
2018-10-08 14:56:25 +08:00
e1c89e5f8e
HBASE-21250 Refactor WALProcedureStore and add more comments for better understanding the implementation
2018-10-07 17:12:27 +08:00
9e3f3fdc1f
HBASE-21185 - WALPrettyPrinter: Additional useful info to be printed by wal printer tool, for debugability purposes
...
Signed-off-by: Allan Yang <allan163@apache.org>
2018-10-04 03:31:15 -07:00
8a5537b5f5
HBASE-18549 Add metrics for failed replication queue recovery
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-10-01 18:39:03 -07:00
44b01d5908
HBASE-19275 TestSnapshotFileCache never worked properly
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-10-01 17:12:18 -07:00
1728a20be2
HBASE-21248 Implement exponential backoff when retrying for ModifyPeerProcedure
2018-09-29 13:35:04 +08:00
3d1a2dbe68
HBASE-21196 HTableMultiplexer clears the meta cache after every put operation
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-28 16:35:53 -07:00
f9e18cf31b
HBASE-20857 balancer status tag in jmx metrics
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-28 16:12:04 -07:00
40c1f072b4
HBASE-21207 Add client side sorting functionality in master web UI for table and region server details
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-28 15:29:49 -07:00
5fe40089f3
HBASE-19418 configurable range of delay in PeriodicMemstoreFlusher
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-28 14:27:28 -07:00
c9c861dbb9
HBASE-18451 PeriodicMemstoreFlusher should inspect the queue before adding a delayed flush request, fix logging
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-28 11:50:19 -07:00
1109ebb617
HBASE-21244 Skip persistence when retrying for assignment related procedures
2018-09-28 21:09:41 +08:00
7b71903baf
HBASE-21228 Memory leak since AbstractFSWAL caches Thread object and never clean later
2018-09-27 16:11:47 +08:00
38d6829a62
HBASE-21232 Show table state in Tables view on Master home page
2018-09-26 10:57:53 -07:00
51854a41fc
HBASE-21227 Implement exponential retrying backoff for Assign/UnassignRegionHandler introduced in HBASE-21217
2018-09-26 20:54:06 +08:00
f3c3c7c422
HBASE-21212 Wrong flush time when update flush metric
2018-09-26 19:17:35 +08:00
21fafbaf53
HBASE-20734 Colocate recovered edits directory with hbase.wal.dir
...
Amending-Author: Reid Chan <reidchan@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-09-26 11:49:21 +08:00
0f514ab75a
HBASE-21164 reportForDuty should do backoff rather than retry
...
Remove unused methods from Sleeper (its ok, its @Private).
Remove notion of startTime from Sleeper handling (it is is unused).
Allow passing in how long to sleep so can maintain externally.
In HRS, use a RetryCounter to calculate backoff sleep time for when
reportForDuty is failing against a struggling Master.
2018-09-25 11:32:15 -07:00
fa2888ebf0
HBASE-21217 Revisit the executeProcedure method for open/close region
2018-09-25 18:10:47 +08:00
20974513d9
Amend HBASE-20704 Sometimes some compacted storefiles are not archived on region close
...
Forward port small logging improvements from branch-1 version of this change.
2018-09-21 16:12:44 -07:00
fd68e7593e
HBASE-20636 Introduce two bloom filter type : ROWPREFIX and ROWPREFIX_DELIMITED
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
2018-09-21 16:06:34 -07:00
29214c77a6
HBASE-21214 [hbck2] setTableState just sets hbase:meta state, not in-memory state
2018-09-21 16:04:42 -07:00
648d58bc23
HBASE-21206 Scan with batch size may return incomplete cells
2018-09-20 22:24:17 +08:00
460cf6ff47
HBASE-21204 NPE when scan raw DELETE_FAMILY_VERSION and codec is not set
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-09-20 06:58:53 -07:00
899fddb4e7
HBASE-21023 Added bypassProcedure() API to HbckService
2018-09-19 15:17:52 -07:00
8876f12c0c
HBASE-21156 [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
...
Adds 'raw' assigns and unassigns methods to Hbck Service.
Fixes HbckService so it works when cluster is Kerberized.
2018-09-19 09:47:32 -07:00
84896deb1c
HBASE-21102 ServerCrashProcedure should select target server where no other replicas exist for the current region - addendum fixes NPE
2018-09-19 07:17:55 -07:00
13581e527d
HBASE-21102 - ServerCrashProcedure should select target server where no
...
other replicas exist for the current region - fix flaky test (Ram)
2018-09-18 11:30:49 +05:30
6616fd2e0f
HBASE-21191 Add a holding-pattern if no assign for meta or namespace (Can happen if masterprocwals have been cleared).
...
Add a check for hbase:meta being online before we go to read it.
If not online, move into a holding-pattern until rectified, probably
by external operator.
Incorporates bulk of patch made by Allan Yang over on HBASE-21035.
M hbase-common/src/main/java/org/apache/hadoop/hbase/util/RetryCounterFactory.java
Add a Constructor for case where retries are for ever.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Move stuff around so that the first hbase:meta read is the AM#loadMeta.
Previously, checking table state and/or favored nodes could end up
trying to read a meta that was not onlined holding up master startup.
Do similar for the namespace table. Adds new methods isMeta and
isNamespace which check that the regions/tables are online.. if not,
we wait logging with a back-off that assigns need to be run.
Signed-off-by: Allan Yang <allan163@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-09-16 21:43:48 -07:00
7a1b4d449d
HBASE-20704 Sometimes some compacted storefiles are not archived on region close
2018-09-16 18:28:24 -07:00
966e3751c2
HBASE-21097 Flush pressure assertion may fail in testFlushThroughputTuning
...
Amending-Author: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-09-15 18:36:33 +08:00
e9d23d5d25
HBASE-21168 Insecure Randomness in BloomFilterUtil
...
Flagged by Fortify static analysis
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-09-12 09:51:57 -05:00
d40348e8cd
HBASE-21098 Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
...
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-09-11 16:31:56 -07:00
c0acad92d2
HBASE-21021 Result returned by Append operation should be ordered
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-11 14:55:32 -07:00
6e699b0694
HBASE-21172 Addendum fix comment
2018-09-11 17:32:38 +08:00
55e1297d96
HBASE-21172 Reimplement the retry backoff logic for ReopenTableRegionsProcedure
2018-09-11 15:14:02 +08:00
52d50cc426
HBASE-21173 Remove the duplicate HRegion#close in TestHRegion
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-09-11 11:20:23 +08:00
1a7dba44ce
HBASE-21158 Empty qualifier cell is always returned when using QualifierFilter
2018-09-10 21:51:48 +08:00
2be9196ba7
HBASE-21102 - ServerCrashProcedure should select target server where no
...
other replicas exist for the current region (ram)
2018-09-10 13:31:36 +05:30
d35e2f35a4
HBASE-21144 AssignmentManager.waitForAssignment is not stable
2018-09-10 12:19:04 +08:00
49501cd5b5
HBASE-21052 After restoring a snapshot, table.jsp page for the table gets stuck
2018-09-09 10:25:10 +09:00
1b65c2f161
HBASE-21001 ReplicationObserver fails to load in HBase 2.0.0
2018-09-07 23:44:03 +08:00
3f8c15ad2b
HBASE-21155 Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir; ADDENDUM
...
Address review comments.
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-09-06 17:25:20 -07:00
62919a3791
HBASE-21155 Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
2018-09-06 16:45:53 -07:00
bbd9131c59
HBASE-21138 Close HRegion instance at the end of every test in TestHRegion
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-09-06 15:01:19 -07:00
1798032ace
HBASE-21161 Enable the test added in HBASE-20741 that was removed
...
accidentally (Ram)
2018-09-06 16:53:16 +05:30
83c871add8
HBASE-21107 add a metrics for netty direct memory
2018-09-05 16:10:17 -07:00
dee13ea401
HBASE-20892 [UI] Start / End keys are empty on table.jsp
2018-09-05 09:21:38 +08:00
1a5aedab34
HBASE-20741 Split of a region with replicas creates all daughter regions
...
and its replica in same server (Ram)
Signed-off-by: Huaxiang Sun, Michael Stack, Duo Zhang
2018-09-04 16:26:11 +05:30
0409c54ba7
HBASE-21129 Clean up duplicate codes in #equals and #hashCode methods of Filter
...
Signed-off-by Ted Yu <yuzhihong@gmail.com>
2018-09-04 11:15:57 +08:00
58365c8dd2
HBASE-21126 Configurable number of allowed failures for ZooKeeper Canary
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-09-03 19:35:28 -07:00
68c5313ca4
HBASE-21136 NPE in MetricsTableSourceImpl.updateFlushTime
2018-09-02 18:32:24 +08:00
cf59f2a1d2
HBASE-21128 TestAsyncRegionAdminApi.testAssignRegionAndUnassignRegion is broken
2018-08-30 10:09:49 +08:00
72a29211cc
HBASE-15728 Add Per-Table metrics back
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableSource.java
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperAggregate.java
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableSourceImpl.java
hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperStub.java
hbase-hadoop2-compat/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsTableSourceImpl.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServer.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsTableWrapperAggregateImpl.java
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMetricsTableAggregate.java
2018-08-29 16:27:15 -07:00
737ac48473
HBASE-21083 Introduce a mechanism to bypass the execution of a stuck procedure
2018-08-28 20:19:55 -07:00
22254cf4ce
HBASE-20942 Fix ArrayIndexOutOfBoundsException for RpcServer TRACE logging
...
Also makes the trace log message length configurable.
Signed-off-by: Josh Elser <elserj@apache.org>
2018-08-28 11:02:18 -04:00
fc21dc854b
HBASE-21017 Revisit the expected states for open/close
2018-08-28 14:49:47 +08:00
cdf5cfa396
HBASE-20941 Created and implemented HbckService in master
...
Added API setTableStateInMeta() to update table state only in Meta. This will be used by hbck2 tool.
2018-08-27 12:10:33 -07:00
a0cbfb8ea6
HBASE-21071 HBaseTestingUtility::startMiniCluster() to use builder pattern
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-08-27 10:22:04 -07:00
f9790b01d9
HBASE-21088 HStoreFile should be closed in HStore#hasReferences
2018-08-27 09:38:03 -07:00
4f32883e95
HBASE-21072 Addendum do not write lock file when running TestHBaseFsckReplication
2018-08-27 21:04:48 +08:00
2369afd7e5
HBASE-21084 When cloning a snapshot including a split parent region, the split parent region of the cloned table will be online
2018-08-27 09:58:52 +09:00
404d4d56cc
HBASE-21095 The timeout retry logic for several procedures are broken after master restarts
2018-08-26 18:17:20 +08:00
f533f01a3a
HBASE-20881 Introduce a region transition procedure to handle all the state transition for a region
2018-08-26 18:08:06 +08:00
2b7ef2e623
HBASE-20772 Controlled shutdown fills Master log with the disturbing message 'No matching procedure found for rit=OPEN, location=ZZZZ, table=YYYYY, region=XXXX transition to CLOSED'
...
Look for the particular case where RS does the close of region w/o
involving Master and log special message in this case. Dodgy. But
until we have Master run shutdown of all regions, better than
the message we currently show.
2018-08-25 14:30:27 +08:00
a83073aff0
HBASE-21078 [amv2] CODE-BUG NPE in RTP doing Unassign
2018-08-24 13:22:45 -07:00
6047fb05f5
HBASE-21113 Apply the branch-2 version of HBASE-21095, The timeout retry logic for several procedures are broken after master restarts
...
I applied the patch HBASE-21095 and then reverted it so could apply the
patch as HBASE-21113 (by reverting the HBASE-21095 revert but pushing
with this message!).
Revert "Revert "HBASE-21095 The timeout retry logic for several procedures are broken after master restarts""
This reverts commit a220566b98
2018-08-24 12:32:33 -07:00
a220566b98
Revert "HBASE-21095 The timeout retry logic for several procedures are broken after master restarts"
...
This reverts commit 09be1d0d57
2018-08-24 12:24:17 -07:00
09be1d0d57
HBASE-21095 The timeout retry logic for several procedures are broken after master restarts
2018-08-24 12:21:01 -07:00
2ea45e67d0
HBASE-21072 Block out HBCK1 in hbase2
...
Write the hbase-1.x hbck1 lock file to block out hbck1 instances writing
state to an hbase-2.x cluster (could do damage).
Set hbase.write.hbck1.lock.file to false to disable this writing.
2018-08-24 09:23:54 -07:00
c33af1e854
HBASE-21101 Remove the waitUntilAllRegionsAssigned call after split in TestTruncateTableProcedure
2018-08-24 10:35:05 +08:00
b318311dfd
HBASE-20193 Move TestCreateTableProcedure.testMRegions to a separated file
2018-08-24 10:09:23 +08:00
87f9b4accd
HBASE-21097 Flush pressure assertion may fail in testFlushThroughputTuning
2018-08-23 11:49:47 -07:00
a7a281a644
HBASE-19008 Add missing equals or hashCode method(s) to stock Filter implementations
...
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-08-24 00:39:35 +08:00
cc6fc1f04f
HBASE-20194 Remove the explicit timeout config for TestTruncateTableProcedure
2018-08-23 06:27:37 +08:00
a7da458c5b
HBASE-21041 Memstore's heap size will be decreased to minus zero after flush
2018-08-22 23:03:20 +08:00
77e1a66f1e
HBASE-21031 Memory leak if replay edits failed during region opening
2018-08-22 22:22:35 +08:00
b5ae4e5f9e
HBASE-20666 Unsuccessful table creation leaves entry in hbase:rsgroup table
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-08-21 18:55:46 -07:00
d1b5113511
HBASE-21032 ScanResponses contain only one cell each
...
Amending-Author: Duo Zhang <zhangduo@apache.org>
2018-08-21 13:31:49 -07:00
3787c72953
HBASE-20917 MetaTableMetrics#stop references uninitialized requestsMap for non-meta region - addendum
2018-08-19 14:25:12 -07:00
52ba33aa73
HBASE-20940 HStore.cansplit should not allow split to happen if it has references (Vishal Khandelwal)
2018-08-17 15:02:22 -07:00
1fa67725c5
HBASE-21047 Object creation of StoreFileScanner thru constructor and close may leave refCount to -1 (Vishal Khandelwal)
2018-08-16 11:42:49 -07:00
75939775af
HBASE-20705 Having RPC quota on a table now no longer prevents Space Quota to be recreate/removed
...
Just added 2 test cases as the subtasks of this jira solves the issue
Signed-off-by: Josh Elser <elserj@apache.org>
2018-08-16 11:16:56 -04:00
cbe2fc113d
HBASE-21062 Correctly use the defaultProvider value on the Providers enum when constructing a WALProvider
2018-08-16 10:18:39 -04:00
d5abb92088
HBASE-20469 Directory used for sidelining old recovered edits files should be made configurable
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-08-15 18:01:24 -07:00
bee8566f2c
HBASE-20985 add two attributes when we do normalization
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-13 16:49:37 +08:00
d979f70142
HBASE-21025 Addendum missed a 'succ = true'
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-13 10:54:49 +08:00
51df2887c4
HBASE-21012 Revert the change of serializing TimeRangeTracker
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-08-11 22:25:10 +08:00
2311fa86d4
HBASE-21025 Add cache for TableStateManager
2018-08-10 21:11:45 +08:00
fb9aa8a95a
HBASE-18201 add UT and docs for DataBlockEncodingTool
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-08-10 11:16:36 +08:00
28c3336bfe
HBASE-20965 Separate region server report requests to new handlers
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-09 18:15:22 +08:00
6a8bd0fe84
HBASE-20813 Removed RPC quotas when the associated table/Namespace is dropped off
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-08-07 13:43:27 -04:00
1aacde4839
HBASE-20986 Separate the config of block size when we do log splitting and write Hlog
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-07 14:00:31 +08:00
b2cefab0e1
HBASE-20885 Removed entry for RPC quota from hbase:quota when RPC quota is removed
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-08-03 11:06:56 -04:00
06a92a3d20
HBASE-19036 Add action in Chaos Monkey to restart Active Namenode
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-08-02 05:00:16 -07:00
690d29bae7
HBASE-20856 PITA having to set WAL provider in two places
...
With this change if hbase.wal.meta_provider is not explicitly set,
it uses whatever set with hbase.wal.provider. this change avoids a use
case of unexpectedly using two different providers when only
hbase.wal.provider is set to non-default but not hbase.wal.meta_provider.
This change also include document (architecture.adoc) update
Also, this is a port from master to branch-2
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <Apache9@apache.org>
2018-08-01 14:36:29 -07:00
fc0c4660fa
HBASE-20989 Minor, miscellaneous logging fixes
...
Signed-off-by: Zach York <zyork@amazon.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-08-01 11:20:26 -07:00
bfb9c71e3b
HBASE-20794 add INFO level log to createTable operation
2018-08-01 11:04:18 -07:00
477d567fd7
HBASE-20893 Data loss if splitting region while ServerCrashProcedure executing ADDENDUM: Rather than rollback, just do region reopens.
...
In split, reopen the parent if recovered.edits and in merge, reopen the
parent region or regions that happened to have recovered.edits on close.
2018-08-01 00:33:33 -07:00
777f3e88e2
HBASE-20935 HStore.removeCompactedFiles should log in case it is unable to delete a file (Vishal Khandelwal)
...
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
2018-07-31 15:59:53 -07:00
99714da30f
HBASE-20950 Helper method to configure secure DFS cluster for tests
...
Create a helper method HBaseKerberosUtils#setSecuredConfiguration().
TestSecureExport, TestSaslFanOutOneBlockAsyncDFSOutput,
SecureTestCluster and TestThriftSpnegoHttpServer uses this new helper
method.
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-31 13:37:10 -07:00
e14b60a539
HBASE-20886 [Auth] Support keytab login in hbase client
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-07-31 11:10:34 +08:00
584093c23f
HBASE-20538 Upgrade our hadoop versions to 2.7.7 and 3.0.3
2018-07-29 20:20:01 +08:00
5957fceda4
Revert "HBASE-20538 Upgrade our hadoop-two.version to 2.7.7 and 3.0.3"
...
This reverts commit 71a4c17d99
2018-07-29 20:19:16 +08:00
71a4c17d99
HBASE-20538 Upgrade our hadoop-two.version to 2.7.7 and 3.0.3
2018-07-29 20:04:53 +08:00
ab18440107
HBASE-19369 Switch to Builder Pattern In WAL
...
This patch switches to the builder pattern by adding a helper method.
It also checks to ensure that the pattern is available (i.e. that
HBase is running on a hadoop version that supports it).
Amending-Author: Mike Drob <mdrob@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-07-27 23:42:53 -05:00
51b0d0edd0
HBASE-20939 There will be race when we call suspendIfNotReady and then throw ProcedureSuspendedException
2018-07-27 21:25:22 +08:00
611ca11c9f
HBASE-20921 Possible NPE in ReopenTableRegionsProcedure
2018-07-27 09:34:25 +08:00
bb6f24cd59
HBASE-20932 Effective MemStoreSize::hashCode()
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-25 14:42:47 -07:00
3b84c9c8d2
HBASE-20649 Validate HFiles do not have PREFIX_TREE DataBlockEncoding
...
Amending-Author: Balazs Meszaros <balazs.meszaros@cloudera.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-07-25 09:12:17 -05:00
ac4b789f5f
HBASE-20867 RS may get killed while master restarts
2018-07-25 18:13:50 +08:00
0b283dae0d
HBASE-20846 Restore procedure locks when master restarts
2018-07-25 14:37:31 +08:00
097ae55708
HBASE-20928 Rewrite calculation of midpoint in binarySearch functions to prevent overflow
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-24 08:17:16 -07:00
99701968c8
HBASE-20565 ColumnRangeFilter combined with ColumnPaginationFilter can produce incorrect result
2018-07-24 10:33:33 +08:00
aaf5c78445
HBASE-20878 Data loss if merging regions while ServerCrashProcedure executing
2018-07-24 09:57:47 +08:00
786fd9137a
HBASE-20893 Data loss if splitting region while ServerCrashProcedure executing
2018-07-23 14:46:32 +08:00
654480326b
HBASE-20917 MetaTableMetrics#stop references uninitialized requestsMap for non-meta region
2018-07-22 20:51:58 -07:00
dd0325a740
HBASE-20401 Make MAX_WAIT and waitIfNotFinished in CleanerContext configurable (Contributed by Stephen Wu)
2018-07-23 10:32:17 +08:00
b15efbf8ed
HBASE-19893 restore_snapshot is broken in master branch when region splits
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-21 07:47:50 -07:00
e8a7a4472f
HBASE-20908 Infinite loop on regionserver if region replica are reduced
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-20 13:05:43 -07:00
810473b277
HBASE-20914 Trim Master memory usage
...
Add (weak reference) interning of ServerNames.
Correct Balancer regions x racks matrix.
Make smaller defaults when creating ArrayDeques.
2018-07-20 10:08:37 -07:00
1a6fae74b5
HBASE-20870 Wrong HBase root dir in ITBLL's Search Tool
2018-07-20 11:22:03 +08:00
37e5f5fb5f
HBASE-20672 New metrics ReadRequestRate and WriteRequestRate
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-07-19 15:41:53 -07:00
cf1f63bf91
HBASE-20901 Reducing region replica has no effect
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-19 13:39:44 -07:00
35a3c605f2
HBASE-6028 Start/Stop compactions at region server level
...
Add switching on/off of compactions.
Switching off compactions will also interrupt any currently ongoing compactions.
Adds a "compaction_switch" to hbase shell. Switching off compactions will
interrupt any currently ongoing compactions. State set from shell will be
lost on restart. To persist the changes across region servers modify
hbase.regionserver.compaction.enabled in hbase-site.xml and restart.
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-07-19 06:20:12 -07:00
e464230bde
HBASE-20907 Fix Intermittent failure on TestProcedurePriority
2018-07-19 12:01:16 +08:00
532d84d6d0
HBASE-20875 MemStoreLABImp::copyIntoCell uses 7% CPU when writing
...
Make the #copyCellInto method smaller so it inlines; we do it by
checking for the common type early and then taking a code path
that presumes ByteBufferExtendedCell -- avoids checks.
2018-07-18 20:41:05 -07:00
3078acb83c
HBASE-20860 Merged region's RIT state may not be cleaned after master restart
2018-07-17 17:33:11 +08:00
07fcdf9418
HBASE-20879 Compacting memstore config should handle lower case
2018-07-13 11:22:06 -07:00
d3bb364595
HBASE-20865 CreateTableProcedure is stuck in retry loop in CREATE_TABLE_WRITE_FS_LAYOUT state
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-13 08:26:05 -07:00
d7f4b06206
HBASE-19572 RegionMover should use the configured default port number and not the one from HConstants
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-07-13 10:46:01 +08:00
59830b0078
HBASE-20847 Addendum use addFront instead of addBack to add sub procedure
2018-07-12 08:31:21 +08:00
430b056ade
HBASE-20847 The parent procedure of RegionTransitionProcedure may not have the table lock
2018-07-11 17:37:21 +08:00
176afd998a
HBASE-20542-ADDENDUM: fix TestHStore
2018-07-11 11:31:28 +03:00
7990283d50
HBASE-20697 Can't cache All region locations of the specify table by calling table.getRegionLocator().getAllRegionLocations()
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-07-11 11:07:04 +08:00
8decae9b92
HBASE-20784 Will lose the SNAPSHOT suffix if we get the version of RS from ServerManager
2018-07-10 10:00:11 +08:00
e6403c18ff
HBASE-20833 Modify pre-upgrade coprocessor validator to support table level coprocessors
...
- -jar parameter now accepts multiple jar files and directories of jar files.
- observer classes can be verified by -class option.
- -table parameter was added to check table level coprocessors.
- -config parameter was added to obtain the coprocessor classes from
HBase cofiguration.
- -scan option was removed.
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-07-09 11:15:18 -05:00
ae22d57201
HBASE-20806 Split style journal for flushes and compactions
2018-07-09 18:42:30 +05:30
8a394ade4c
HBASE-20822 TestAsyncNonMetaRegionLocator is flakey
2018-07-09 14:56:42 +08:00
1da2fa3baa
HBASE-20808 (Addendum) Remove duplicate calls for cancelling of chores
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-07-07 00:18:22 +08:00
473c71edb1
HBASE-20808 Wrong shutdown order between Chores and ChoreService
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-07-06 11:37:13 +08:00
4653d4ac6b
HBASE-20691 Change the default WAL storage policy back to "NONE""
...
This reverts commit 564c193d61
2018-07-04 13:45:16 +08:00
341fa55367
HBASE-20474 Show non-RPC tasks on master/regionserver Web UI by default
2018-07-04 10:56:46 +08:00
5ef9d8f451
HBASE-20839 Fallback to FSHLog if we can not instantiated AsyncFSWAL when user does not specify AsyncFSWAL explicitly
2018-07-04 10:29:30 +08:00
c612642b2b
HBASE-20829 Remove the addFront assertion in MasterProcedureScheduler.doAdd
2018-07-04 09:40:52 +08:00
021eae48ca
HBASE-20244 NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-07-03 22:15:48 +08:00
8b1e6a81ff
HBASE-20789 TestBucketCache#testCacheBlockNextBlockMetadataMissing is flaky
2018-07-03 17:59:25 +08:00
30664b1a57
HBASE-20193 Basic Replication Web UI - Regionserver
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-07-03 16:02:28 +08:00
33d002f445
HBASE-20826 Truncate really long RpcServer warnings unless TRACE is on
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-07-03 10:14:52 +08:00
ad7e453817
HBASE-20450 Provide metrics for number of total active, priority and replication rpc handlers
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-07-02 11:38:02 -07:00
3847def1c2
HBASE-20817 Infinite loop when executing ReopenTableRegionsProcedure
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-07-02 21:28:53 +08:00
7868f0ef97
HBASE-20542: Better heap utilization for IMC with MSLABs
2018-07-01 15:52:30 +03:00
490ca0ce20
HBASE-20825 Fix pre and post hooks of CloneSnapshot and RestoreSnapshot for Access checks
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
2018-06-29 16:33:34 -04:00
a16e5523fc
HBASE-20357 AccessControlClient API Enhancement
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-06-29 07:54:14 -07:00
02235f4e32
HBASE-20792 info:servername and info:sn inconsistent for OPEN region
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-06-29 11:11:09 +08:00
58ccd3dc7e
HBASE-19722 Meta query statistics metrics source
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-06-28 17:17:20 -07:00
f264b79ec1
HBASE-20781 Save recalculating families in a WALEdit batch of Cells
...
Pass the Set of families through to the WAL rather than recalculate
a Set already known.
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-06-27 22:04:40 -07:00
32e0767dfd
HBASE-20732 Shutdown scan pool when master is stopped
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-06-28 11:51:52 +08:00
e61507b9a0
HBASE-19164: Remove UUID.randomUUID in tests.
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-27 10:36:48 -05:00
557985ee47
HBASE-20194 Basic Replication WebUI - Master
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-06-26 18:27:50 +08:00
a1b297e673
HBASE-20777 RpcConnection could still remain opened after we shutdown the NettyRpcServer
2018-06-26 09:07:58 +08:00
40cd76af8a
HBASE-20780 ServerRpcConnection logging cleanup Get rid of one of the logging lines in ServerRpcConnection by amalgamating all into one new-style log line.
2018-06-25 16:43:43 -07:00
4e16ecd7f4
HBASE-20770 WAL cleaner logs way too much; gets clogged when lots of work to do
...
General log cleanup; setting stuff that can flood the log to TRACE.
2018-06-25 12:12:43 -07:00
bf65e95466
HBASE-20403. Fix race between prefetch task and non-pread HFile reads
...
With prefetch-on-open enabled, the task doing the prefetching was using
non-positional (i.e. streaming) reads. If the main (non-prefetch) thread
was also using non-positional reads, these two would conflict, because
inputstreams are not thread-safe for non-positional reads.
In the case of an encrypted filesystem, this could cause JVM crashes,
etc, as underlying cipher buffers were freed underneath the racing
threads. In the case of a non-encrypted filesystem, less severe errors
would be thrown. The included unit test reproduces the latter case.
(cherry picked from commit 025ddce868
2018-06-25 12:11:36 -07:00
e915024e3c
HBASE-20778 Make it so WALPE runs on DFS
2018-06-23 23:34:17 -07:00
2f4b0a4a42
HBASE-20775 TestMultiParallel is flakey
2018-06-24 08:26:26 +08:00
dde042cc93
HBASE-20776 Update branch-2 version to 2.2.0-SNAPSHOT
2018-06-22 22:15:18 +08:00
108e636247
HBASE-18569 Add prefetch support for async region locator
2018-06-22 18:20:08 +08:00
a86141b625
HBASE-20752 Make sure the regions are truly reopened after ReopenTableRegionsProcedure
2018-06-22 14:06:29 +08:00
6cebe06225
HBASE-20767 Always close hbaseAdmin along with connection in HBTU
2018-06-22 10:20:06 +08:00
28d0d8c5cc
HBASE-20642 Clients should re-use the same nonce across DDL operations
...
Also changes modify table operations to help the case where a MTP spans
two master, avoiding the sanity-checks propagating back to the client
unnecessarily.
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-06-20 15:10:52 -07:00
c8b76eb3f1
HBASE-20706 Prevent MTP from trying to reopen non-OPEN regions
...
ModifyTableProcedure is using MoveRegionProcedure in a way
that was unintended from the original implementation. As such,
we have to guard against certain usages of it. We know we can
re-open OPEN regions, but regions in OPENING will similarly
soon be OPEN (thus, we want to reopen those regions too).
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-06-20 14:30:36 -07:00
5c2cb15e0b
HBASE-20739 Add priority for SCP
2018-06-20 15:18:47 +08:00
83969b0da1
HBASE-20742 Always create WAL directory for region server
2018-06-20 14:21:28 +08:00
9eeb501825
HBASE-20745 Log when master proc wal rolls
2018-06-19 19:53:29 -07:00
3e33aecea2
HBASE-20708 Remove the usage of RecoverMetaProcedure in master startup
2018-06-19 15:09:11 +08:00
ee84a8f243
HBASE-20332 shaded mapreduce module shouldn't include hadoop
...
* modify the jar checking script to take args; make hadoop stuff optional
* separate out checking the artifacts that have hadoop vs those that don't.
* * Unfortunately means we need two modules for checking things
* * put in a safety check that the support script for checking jar contents is maintained in both modules
* * have to carve out an exception for o.a.hadoop.metrics2. :(
* fix duplicated class warning
* clean up dependencies in hbase-server and some modules that depend on it.
* allow Hadoop to have its own htrace where it needs it
* add a precommit check to make sure we're not using old htrace imports
Conflicts:
hbase-backup/pom.xml
hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-18 14:02:48 -07:00
b04c976fe6
HBASE-20478 Update checkstyle to v8.2
...
Cannot go to latest (8.9) yet due to
https://github.com/checkstyle/checkstyle/issues/5279
* move hbaseanti import checks to checkstyle
* implment a few missing equals checks, and ignore one
* fix lots of javadoc errors
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-06-18 14:02:40 -07:00
8edd5d948a
HBASE-20737 put collection into ArrayList instead of addAll function
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-06-17 11:16:16 +08:00
b2afba580b
HBASE-20723 Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them
2018-06-16 01:34:53 -07:00
b68746c0b2
HBASE-20695 Implement table level RegionServer replication metrics
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-06-15 10:45:13 +08:00
bde9f08a83
HBASE-20625 refactor some WALCellCodec related code
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-06-14 19:46:33 +08:00
161dc7c7f3
HBASE-20722 Make RegionServerTracker only depend on children changed event
2018-06-14 08:38:53 +08:00
075523dd1e
HBASE-20561 The way we stop a ReplicationSource may cause the RS down
2018-06-13 18:05:27 +08:00
d44e8a7aff
HBASE-20656 Validate pre-2.0 coprocessors against HBase 2.0+
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-11 10:32:40 -05:00
4b0bbd839e
HBASE-20707 Move MissingSwitchDefault case check
...
Perform this check using error-prone instead of checkstyle because the
former can handle enum switches somewhat more intelligently.
2018-06-11 10:13:29 -05:00
6befdc43ba
HBASE-20700 Move meta region when server crash can cause the procedure to be stuck
2018-06-11 15:28:21 +08:00
4d971d0f48
HBASE-20698 (addendum) Master don't record right server version until new started region server call regionServerReport method
2018-06-10 08:32:01 +08:00
9d15e16946
HBASE-20698 Master don't record right server version until new started region server call regionServerReport method
2018-06-09 14:47:07 +08:00
4a5fe54d94
HBASE-20699 QuotaCache should cancel the QuotaRefresherChore service inside its stop()
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-06-08 07:22:52 -07:00
858eee20ec
HBASE-20702 Processing crash, skip ONLINE'ing empty rows
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-06-07 09:54:32 -07:00
271d93dc73
HBASE-20665: Changed log level of HBASE-8547 warning to debug
...
Closes #77
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-06-07 11:34:52 -04:00
00289b8ffa
HBASE-20683 Incorrect return value for PreUpgradeValidator
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-06-06 20:04:22 +02:00
d7b09de854
HBASE-20670 NPE in HMaster#isInMaintenanceMode
2018-06-04 15:19:45 -07:00
063eefe3b0
HBASE-20634 Reopen region while server crash can cause the procedure to be stuck; ADDENDUM
2018-06-04 12:38:56 -07:00
27e2c8c86b
HBASE-20628 SegmentScanner does over-comparing when one flushing
...
Signed-off-by: eshcar <eshcar@oath.com>
Signed-off-by: anoopsjohn <anoopsamjohn@gmail.com>
2018-06-04 09:50:13 -07:00
d834859404
HBASE-20634 Reopen region while server crash can cause the procedure to be stuck
...
A reattempt at fixing HBASE-20173 [AMv2] DisableTableProcedure concurrent to ServerCrashProcedure can deadlock
The scenario is a SCP after processing WALs, goes to assign regions that
were on the crashed server but a concurrent Procedure gets in there
first and tries to unassign a region that was on the crashed server
(could be part of a move procedure or a disable table, etc.). The
unassign happens to run AFTER SCP has released all RPCs that
were going against the crashed server. The unassign fails because the
server is crashed. The unassign used to suspend itself only it would
never be woken up because the server it was going against had already
been processed. Worse, the SCP could not make progress because the
unassign was suspended with the lock on a region that it wanted to
assign held making it so it could make no progress.
In here, we add to the unassign recognition of the state where it is
running post SCP cleanup of RPCs. If present, unassign moves to finish
instead of suspending itself.
Includes a nice unit test made by Duo Zhang that reproduces nicely the
hung scenario.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/FailedRemoteDispatchException.java
Moved this class back to hbase-procedure where it belongs.
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/NoNodeDispatchException.java
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/NoServerDispatchException.java
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/NullTargetServerDispatchException.java
Specializiations on FRDE so we can be more particular when we say there
was a problem.
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/RemoteProcedureDispatcher.java
Change addOperationToNode so we throw exceptions that give more detail
on issue rather than a mysterious true/false
M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
Undo SERVER_CRASH_HANDLE_RIT2. Bad idea (from HBASE-20173)
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
Have expireServer return true if it actually queued an expiration. Used
later in this patch.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Hide methods that shouldn't be public. Add a particular check used out
in unassign procedure failure processing.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
Check that server we're to move from is actually online (might
catch a few silly move requests early).
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
Add doc on ServerState. Wasn't being used really. Now we actually stamp
a Server OFFLINE after its WAL has been split. Means its safe to assign
since all WALs have been processed. Add methods to update SPLITTING
and to set it to OFFLINE after splitting done.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
Change logging to be new-style and less repetitive of info.
Cater to new way in which .addOperationToNode returns info (exceptions
rather than true/false).
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
Add looking for the case where we failed assign AND we should not
suspend because we will never be woken up because SCP is beyond
doing this for all stuck RPCs.
Some cleanup of the failure processing grouping where we can proceed.
TODOs have been handled in this refactor including the TODO that
wonders if it possible that there are concurrent fails coming in
(Yes).
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
Doc and removing the old HBASE-20173 'fix'.
Also updating ServerStateNode post WAL splitting so it gets marked
OFFLINE.
A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestServerCrashProcedureStuck.java
Nice test by Duo Zhang.
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Duo Zhang <palomino219@gmail.com>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-04 09:26:36 -07:00
4c95b82b61
HBASE-19761:Fix Checkstyle errors in hbase-zookeeper
...
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2018-06-02 10:17:27 +02:00
f46569a742
HBASE-20667 Rename TestGlobalThrottler to TestReplicationGlobalThrottler
2018-06-01 17:01:14 -07:00
d3e2248f12
HBASE-18116 Replication source in-memory accounting should not include bulk transfer hfiles
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-06-01 11:16:16 -07:00
53d29d53c4
HBASE-20592 Create a tool to verify tables do not have prefix tree encoding
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-01 19:22:49 +02:00
b22409d51d
Revert "HBASE-18116 fix replication source in-memory calculation by excluding bulk load file"
...
This reverts commit 050fae501a
2018-05-31 15:28:37 -07:00
050fae501a
HBASE-18116 fix replication source in-memory calculation by excluding bulk load file
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-31 14:22:12 -07:00
fc9743c17a
HBASE-20444 Addendum keep folks from looking at raw version component array.
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-31 14:17:41 -05:00
aaec02e0f5
HBASE-20646 TestWALProcedureStoreOnHDFS failing on branch-1
2018-05-30 14:44:54 -07:00
15bb234d51
Revert "TestWALProcedureStoreOnHDFS failing on branch-1"
...
This reverts commit 694e79a67e
2018-05-30 14:44:49 -07:00
694e79a67e
TestWALProcedureStoreOnHDFS failing on branch-1
2018-05-30 13:46:08 -07:00
b785896cbd
HBASE-20659 Implement a reopen table regions procedure
2018-05-30 20:03:35 +08:00
856a3ac154
HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint - revert due to pending discussion
2018-05-29 19:58:32 -07:00
2dc51934f4
HBASE-20597 Serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
2018-05-29 11:29:12 -07:00
7f154dc20e
Revert "HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint"
...
This reverts commit 60dcef289b
2018-05-29 11:24:30 -07:00
d36cce1574
HBASE-20633 Dropping a table containing a disable violation policy fails to remove the quota upon table delete
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-05-29 11:50:40 -04:00
aa00391140
HBASE-20390 ADDENDUM 2: fix TestHRegionWithInMemoryFlush OOME
2018-05-29 16:24:27 +03:00
cf1928aaca
HBASE-20390-ADDENDUM: fix TestHRegionWithInMemoryFlush OOME
2018-05-29 13:01:07 +03:00
c8fd6e0fb6
HBASE-20533 Fix the flaky TestAssignmentManagerMetrics
2018-05-29 09:50:04 +08:00
0455e75edd
HBASE-20648 HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-28 08:27:41 -07:00
bc72fcd8c5
HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-27 18:43:08 -07:00
f40c10a211
HBASE-20518 Need to serialize the enabled field for UpdatePeerConfigProcedure
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-05-25 14:45:49 +08:00
d1cbd561df
HBASE-20548 Master fails to startup on large clusters, refreshing block distribution
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-24 15:47:22 -07:00
db8789ab22
HBASE-20616 TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-24 15:17:01 -07:00
55c4791a63
HBASE-20588 Space quota change after quota violation doesn't seem to take in effect
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-05-24 12:50:22 -04:00
079e08d7c5
HBASE-20390: IMC Default Parameters for 2.0.0
2018-05-24 18:10:10 +03:00
bfab1e2f92
HBASE-20589 Don't need to assign meta to a new RS when standby master become active
2018-05-24 11:45:59 +08:00
a94c6dbadb
HBASE-20624 Race in ReplicationSource which causes walEntryFilter being null when creating new shipper
2018-05-24 10:48:35 +08:00
60dcef289b
HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
2018-05-23 16:46:20 -07:00
afddf6b1c2
HBASE-20620 HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; part 2
...
Adds new stripped-down, faster ByteBufferKeyValue comparator
(BBKV is the base Cell-type in hbase2). Creates an instance
of new Comparator each time we create new memstore rather
than use the universal CellComparator.
Remove unused and unneeded Interfaces from Cell base type.
2018-05-23 13:20:29 -07:00
fbda502435
HBASE-20612 TestReplicationKillSlaveRSWithSeparateOldWALs sometimes fail because it uses an expired cluster conn
2018-05-23 12:07:54 +08:00
c3c9a4a595
HBASE-20579 Improve snapshot manifest copy in ExportSnapshot
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-18 06:43:53 -07:00
d06673cf3e
HBASE-20583 SplitLogWorker should handle FileNotFoundException when split a wal
2018-05-18 14:30:40 +08:00
39ea1efa88
HBASE-20571 JMXJsonServlet generates invalid JSON if it has NaN in metrics
...
- CacheStats won't generate NaN metrics.
- JSONBean class will serialize special floating point values as
"NaN", "Infinity" or "-Infinity"
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-16 12:20:37 -07:00
61f2b5f071
HBASE-20567 Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace.
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-05-16 14:03:36 -05:00
77eaff0e10
HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; ADDENDUM
...
Add method the CellComparator Interface. Add implementation to
meta comparator so we don't fall back to the default comparator.
Includes a nothing change to hbase-server/pom.xml just to provoke
build.
2018-05-16 09:42:51 -07:00
5c4685e56e
HBASE-20520 Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
2018-05-16 09:18:06 -07:00
82e3011166
HBASE-20585 Need to clear peer map when clearing MasterProcedureScheduler
2018-05-16 08:46:34 +08:00
60b8344cf1
HBASE-20457 Return immediately for a scan rpc call when we want to switch from pread to stream
2018-05-15 21:09:04 +08:00
b7def9b690
HBASE-20447 Only fail cacheBlock if block collisions aren't related to next block metadata
...
When we pread, we don't force the read to read all of the next block header.
However, when we get into a race condition where two opener threads try to
cache the same block and one thread read all of the next block header and
the other one didn't, it will fail the open process. This is especially important
in a splitting case where it will potentially fail the split process.
Instead, in the caches, we should only fail if the required blocks are different.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-14 16:09:14 -07:00
4b0ac73f51
HBASE-20560 Revisit the TestReplicationDroppedTables ut
2018-05-14 19:33:51 +08:00
be3df29cef
HBASE-20128 Add new UTs which extends the old replication UTs but set replication scope to SERIAL
2018-05-14 19:32:39 +08:00
5ac7740896
HBASE-20411 Ameliorate MutableSegment synchronize
...
Change the MemStore size accounting so we don't synchronize across three
volatiles applying deltas. Instead:
+ Make MemStoreSize, a datastructure of our memstore size longs, immutable.
+ Undo MemStoreSizing being an instance of MemStoreSize; instead it has-a.
+ Make two MemStoreSizing implementations; one thread-safe, the other not.
+ Let all memory sizing longs run independent, untied by
synchronize (Huaxiang and Anoop suggestion) using atomiclongs.
+ Review all use of MemStoreSizing. Many are single-threaded and do
not need to be synchronized; use the non-thread safe counter.
TODO: Use this technique accounting at the global level too.
2018-05-12 02:16:19 +01:00
1f10ef553e
HBASE-20545 Improve performance of BaseLoadBalancer.retainAssignment
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-10 10:45:17 -07:00
61f96b6ffa
HBASE-20544 Make HBTU default to random ports.
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Josh Elser <elserj@apache.org>
Conflicts:
hbase-backup/src/test/resources/hbase-site.xml
hbase-spark-it/src/test/resources/hbase-site.xml
hbase-spark/src/test/resources/hbase-site.xml
2018-05-09 23:45:39 -07:00
c430016cf9
HBASE-20554 "WALs outstanding" message from CleanerChore is noisy
2018-05-09 19:11:50 -07:00
cba8d2fb8d
HBASE-20204 Add locking to RefreshFileConnections in BucketCache
...
This is a follow-up to HBASE-20141 where Anoop suggested adding locking
for refreshing channels.
2018-05-09 14:23:27 -07:00
de25f8b209
HBASE-20539 Disable IMC; part 2
...
Set default for in-memory compaction to NONE. PE was using
this value creating its table. Add test to ensure expected
default.
2018-05-09 06:29:56 -07:00
55badefa38
HBASE-20485 Copy constructor of Scan doesn't copy the readType and replicaId
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-05-09 16:45:39 +08:00
d6d209564c
HBASE-20536 Make TestRegionServerAccounting stable and it should not use absolute number
2018-05-08 08:21:30 +08:00
fd0840c3e4
HBASE-20538 TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
...
Disable test TestSaslFanOutOneBlockAsyncDFSOutput for the moment.
Depends on HDFS-13494
2018-05-07 09:54:07 -07:00
de71cb5915
HBASE-20481 Replicate entries from same region serially in ReplicationEndpoint for serial replication
2018-05-04 15:52:43 +08:00
5f260451d5
HBASE-20378 Provide a hbck option to cleanup replication barrier for a table
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-05-04 15:20:17 +08:00
eaafdff761
HBASE-20524 Need to clear metrics when ReplicationSourceManager refresh replication sources
2018-05-04 14:34:07 +08:00
e92580e201
HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer
2018-05-03 12:26:54 -07:00
f89c5b08df
HBASE-20414 TestLockProcedure#testMultipleLocks may fail on slow machine
2018-05-03 07:12:11 -07:00
984fb5bd05
HBASE-20169 NPE when calling HBTU.shutdownMiniCluster (TestAssignmentManagerMetrics is flakey); AMENDMENT
2018-05-02 16:14:38 -07:00
b0719ec11b
HBASE-20514 On Master restart if table is stuck in DISABLING state, CLOSED regions should not be considered stuck in-transition
...
On startup CLOSED regions of DISABLED and DISABLING tables are treated the same way as not in-transition.
2018-05-02 12:18:04 -07:00
b43cde2ce1
HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer
2018-05-02 10:27:00 +08:00
da3e06afab
HBASE-20492 UnassignProcedure is stuck in retry loop on region stuck in OPENING state
...
Add backoff when stuck in RegionTransitionProcedure, the subclass of
AssignProcedure and UnassignProcedure. Can happen when we go to
transition but the current Region state is not what we expect.
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
Add doc on being able to suspend and wait on a timeout.
M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
Add 'attempt' counter so we can do backoff when we get stuck.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
Add persistence of new 'attempt' counter
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
Doc data members that are persisted by subclasses given this is 'odd'.
Add a counter for 'attempts' used when 'stuck' to implement backoff.
Add suspend with timeout when 'stuck'. Add callback when timeout is
exhausted which does wakeup of this procedure.
A hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestUnexpectedStateException.java
Test of backoff.
2018-04-30 17:58:27 -07:00
d38a104c4d
HBASE-20497 The getRecoveredQueueStartPos always return 0 in RecoveredReplicationSourceShipper
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-28 20:50:47 +08:00
72093178fb
HBASE-20475 Fix the flaky TestReplicationDroppedTables unit test.
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-28 18:46:41 +08:00
e8603e1d7c
HBASE-20476 Open sequence number could go backwards in AssignProcedure
2018-04-28 16:55:06 +08:00
20e2ac132e
HBASE-20040 Fixed: Master UI should include "Cluster Key" needed to use the cluster as a replication sink
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-26 06:46:07 -05:00
75004408b0
HBASE-20466 Consistently use override mechanism for exempt classes in CoprocessorClassloader
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-23 07:48:13 -07:00
45c087d988
HBASE-20470 [2.0.0RC1] has broken unit tests...
...
Fix test that depended upon flush being slow and one family only.
Fix MemStoreSize compare to allow passing alternate implementation
(needed when IMC was no longer default everywhere).
2018-04-22 15:32:56 -07:00
298ce96246
HBASE-19924 hbase rpc throttling does not work for multi() with request count rater.
2018-04-20 16:55:23 -07:00
22e7ae0311
HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-20 11:22:49 -05:00
24eb141bac
HBASE-20442 clean up incorrect use of commons-collections 3
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
Conflicts:
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/master/BackupLogCleaner.java
2018-04-20 08:17:22 -05:00
9740168fa8
HBASE-20440 Clean up incorrect use of commons-lang 2.y
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierFactoryImpl.java
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierImpl.java
2018-04-20 08:17:22 -05:00
944ecc729c
HBASE-20439 Clean up incorrect use of commons-logging in hbase-server
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierImpl.java
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeReportingChore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeStoreImpl.java
Additions:
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCellBasedHFileOutputFormat2.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCellBasedImportExport2.java
2018-04-20 08:04:17 -05:00
af4dd3eb3c
HBASE-18059 remove scanner order related code
2018-04-18 19:37:12 -07:00
3ac0ffde4c
HBASE-18792 Disabled destructive (mostly write) operations in hbck for HBase 2.0+
...
HBase server versioned is checked after connecting to the server and then following operations are not allowed:
-fix, -fixAssignments, -fixMeta, -fixHdfsHoles, -fixHdfsOrphans, -fixTableOrphans, -fixHdfsOverlaps, -maxMerge
-sidelineBigOverlaps, -maxOverlapsToSideline, -fixSplitParents, -removeParents, -fixEmptyMetaCells
-repair, -repairHoles
2018-04-18 15:22:03 -07:00
36680da549
HBASE-19994 Create a new class for RPC throttling exception, make it retryable
2018-04-17 10:45:44 -07:00
1c8d9d788f
HBASE-20404 Fixes to CleanChore correctness and operability.
...
* Make CleanerChore less chatty: move WARN message to DEBUG when we expect non-empty dirs
* Make CleanerChore less chatty: move IOE we'll retry to INFO
* CleanerChore should treat IOE for FileStatus as a failure
* Add tests asserting assumptions in above
Signed-off-by: Reid Chan <reidddchan@outlook.com>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-17 11:53:19 -05:00
d951675df5
HBASE-20398 Redirect doesn't work on web UI
2018-04-17 09:19:13 -07:00
46c7808393
HBASE-20399 Fix merge layout
2018-04-17 09:16:58 -07:00
ecd9914dc9
HBASE-19963 TestFSHDFSUtils assumes wrong default port for Hadoop 3.0.1+
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-16 12:00:08 -05:00
f73986d110
HBASE-20417 Do not read wal entries when peer is disabled
2018-04-16 22:07:48 +08:00
1cb05a18bc
HBASE-20419 Fix potential NPE in ZKUtil#listChildrenAndWatchForNewChildren callers
...
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-16 16:07:59 +08:00
4fea862963
HBASE-20409 Set hbase.client.meta.operation.timeout in TestClientOperationTimeout
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-04-15 10:12:34 +02:00
f5d970eba1
HBASE-20294 Also cleanup last pushed sequence id in ReplicationBarrierCleaner
2018-04-14 14:56:51 +08:00
ae8a21204d
HBASE-20377 Deal with table in enabling and disabling state when modifying serial replication peer
2018-04-13 20:33:25 +08:00
b0701434e3
HBASE-20243 [Shell] Add shell command to create a new table by cloning the existent table
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-11 19:34:30 -07:00
0f6ff895e9
HBASE-20350 NullPointerException in Scanner during close()
2018-04-11 14:34:23 -07:00
0773209dd3
HBASE-20310 Fixed false inconsistency shown by hbck -metaonly option on HBase 2
...
When -metaonly option is specified then tablesInfo is not populated for all tables from hdfs causing false inconsistency of dangling state in meta.
2018-04-11 12:50:55 -07:00
f3089704d2
HBASE-20219 An error occurs when scanning with reversed=true and loadColumnFamiliesOnDemand=true
2018-04-11 12:27:28 -07:00
b52bf4052f
HBASE-20382 If RSGroups not enabled, rsgroup.jsp prints stack trace
2018-04-11 09:49:50 -07:00
c78908cff3
HBASE-20384 [AMv2] Logging format improvements; use encoded name rather than full region name marking transitions
2018-04-11 08:40:49 -07:00
3eee7d37f4
HBASE-20367 Write a replication barrier for regions when disabling a table
2018-04-11 20:36:59 +08:00
4bcb560e22
HBASE-15291 FileSystem not closed in secure bulkLoad
...
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-11 12:11:41 +05:30
2487b4603c
HBASE-20363 TestNamespaceAuditor.testRegionMerge is flaky
2018-04-09 21:02:42 +08:00
74ab10c353
HBASE-20296 Remove last pushed sequence ids when removing tables from a peer
2018-04-09 15:18:44 +08:00
ead569c951
HBASE-20285 Delete all last pushed sequence ids when removing a peer or removing the serial flag for a peer
2018-04-09 15:18:44 +08:00
8de9596355
HBASE-20127 Add UT for serial replication after failover
2018-04-09 15:18:44 +08:00
6f13b28c19
HBASE-20271 ReplicationSourceWALReader.switched should use the file name instead of the path object directly
2018-04-09 15:18:44 +08:00
9369cf26eb
HBASE-20147 Serial replication will be stuck if we create a table with serial replication but add it to a peer after there are region moves
2018-04-09 15:18:44 +08:00
aec43bb207
HBASE-20242 The open sequence number will grow if we fail to open a region after writing the max sequence id file
2018-04-09 15:18:44 +08:00
16a4dd6b8f
HBASE-20206 WALEntryStream should not switch WAL file silently
2018-04-09 15:18:44 +08:00
644bfe36b2
HBASE-20117 Cleanup the unused replication barriers in meta table
2018-04-09 15:18:44 +08:00
2d0d6a3ba1
HBASE-20167 Optimize the implementation of ReplicationSourceWALReader
2018-04-09 15:18:44 +08:00
cea5199ea1
HBASE-20148 Make serial replication as a option for a peer instead of a table
2018-04-09 15:18:44 +08:00
6b5412759f
HBASE-20125 Add UT for serial replication after region split and merge
2018-04-09 15:18:44 +08:00
8b61a061d3
HBASE-20129 Add UT for serial replication checker
2018-04-09 15:18:44 +08:00
f29bf1d778
HBASE-20115 Reimplement serial replication based on the new replication storage layer
2018-04-09 15:18:44 +08:00
1d11cdb26c
HBASE-20050 Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-09 15:18:44 +08:00
39c1ddc6e3
HBASE-20362 TestMasterShutdown.testMasterShutdownBeforeStartingAnyRegionServer is flaky
2018-04-09 14:55:42 +08:00
dcc1d9e36a
HBASE-20182 Can not locate region after split and merge
2018-04-07 13:37:59 +08:00
e78a8e08f0
HBASE-20095 Redesign single instance pool in CleanerChore - addendum simplifies onConfigurationChange
2018-04-05 10:25:09 -07:00
039bc73571
HBASE-19488 Move to using Apache commons CollectionUtils
...
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-04-04 14:16:33 -07:00
9a34880724
HBASE-16499 slow replication for small HBase clusters
...
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-04 09:59:50 +05:30
ed21f26171
HBASE-20298 Doc change in read/write/total accounting metrics
2018-04-03 17:02:27 -07:00
d4e115bf34
HBASE-20259 Doc configs for in-memory-compaction and add detail to
...
in-memory-compaction logging
Adds logging of CompactingMemStore configuration on construction.
Add logging of detail about Store on creation including memstore type.
Add chapter to refguide on new in-memory compaction feature.
2018-04-02 09:56:32 -07:00
4b909b890d
HBASE-17631 Changed Canary interval to 60 seconds
2018-04-01 13:22:37 +02:00
468d4fcd73
HBASE-20282 Clean up tooling docs/help
2018-03-30 09:00:46 -07:00
4c9167a25c
HBASE-20159 Support using separate ZK quorums for client - addendum fixes flaky TestSeparateClientZKCluster
2018-03-30 07:23:25 -07:00
1687dc5ddb
HBASE-20197 Review of ByteBufferWriterOutputStream.java (BELUGA BEHR)
...
Signed-off-by: Vasudevan <ramkrishna.s.vasudevan@gmail.com>
2018-03-30 12:20:09 +05:30
b72e19e3b9
HBASE-20159 Support using separate ZK quorums for client
2018-03-29 03:20:52 +08:00
5712fd0451
HBASE-20199 Add a unit test to verify flush and snapshot permission requirements aren't excessive
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-27 20:10:28 -04:00
6f1aa0edff
HBASE-20280 Fix possibility of deadlocking in refreshFileConnections
...
When prefetch on open is specified, there is a deadlocking case
where if the prefetch is cancelled, the PrefetchExecutor interrupts
the threads if necessary, when that happens in FileIOEngine, it
causes an ClosedByInterruptException which is a subclass of
ClosedChannelException. If we retry all ClosedChannelExceptions,
this will lock as this access is expected to be interrupted.
This change removes calling refreshFileConnections for
ClosedByInterruptExceptions.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-03-27 16:52:59 -07:00
a601c57f97
HBASE-20111 A region's splittable state now includes the configuration splitPolicy
...
The Master asks a RegionServer whether a Region can be split or not, primarily to
verify that the region is not closing, opening, etc. This change has the RegionServer
also consult the configured RegionSplitPolicy.
Signed-off-by: Josh Elser <elserj@apache.org>
2018-03-27 14:58:58 -04:00
e1afc5932a
HBASE-20229 ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on
2018-03-26 15:13:13 -07:00
c3d82a283d
HBASE-20223 Update to hbase-thirdparty 2.1.0
...
Remove commons-cli and commons-collections4 use. Account
for the newer internal protobuf version of 3.5.1.
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-03-26 16:07:39 -04:00
7c03ea0436
HBASE-20095 Redesign single instance pool in CleanerChore
2018-03-26 12:49:22 -05:00
b5881dbd3f
HBASE-17819 Reduce the heap overhead for BucketCache.
2018-03-25 16:39:21 +05:30
c8c670b561
HBASE-20272 TestAsyncTable#testCheckAndMutateWithTimeRange fails due to TableExistsException
2018-03-24 12:42:49 -07:00
a6debe4845
Revert "HBASE-19504 Add TimeRange support into checkAndMutate; ADDENDUM to fix failing unit test"
...
Reverting an addendum in favor of patch at HBASE-20272.
This reverts commit 5da92ce352
2018-03-24 12:41:26 -07:00
5da92ce352
HBASE-19504 Add TimeRange support into checkAndMutate; ADDENDUM to fix failing unit test
2018-03-24 12:29:45 -07:00
52014b9423
HBASE-13300 Fixed casing in set/getTimeStamp for Mutations
2018-03-23 17:23:55 -07:00
e962e1ac2f
HBASE-20135 Fixed NullPointerException during reading bloom filter when upgraded from hbase-1 to hbase-2
2018-03-23 15:20:43 -07:00
e6d9e04511
HBASE-20261 Table page (table.jsp) in Master UI does not show replicaIds for hbase meta table
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-03-23 13:34:08 -04:00
6aba045aae
HBASE-19504 Add TimeRange support into checkAndMutate
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-24 00:05:22 +08:00
96d63fee11
HBASE-20224 Web UI is broken in standalone mode
...
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
2018-03-22 20:28:08 -07:00
79e4c9d925
Revert "HBASE-20224 Web UI is broken in standalone mode"
...
Broke shell tests.
This reverts commit dd9fe813ec
2018-03-22 10:47:47 -07:00
dd9fe813ec
HBASE-20224 Web UI is broken in standalone mode
...
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
2018-03-22 06:52:51 -07:00
dd9e46bbf5
HBASE-20212 Make all Public classes have InterfaceAudience category
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-22 18:09:54 +08:00
0a94c9c250
HBASE-17449 Add explicit document on different timeout settings
2018-03-22 12:45:06 +05:30
2bc99e4b5e
HBASE-20237 Put back getClosestRowBefore and throw UnsupportedOperation instead... for asynchbase client Throw exception if an old client connects.
2018-03-21 21:48:15 -07:00
7a130dcc33
HBASE-20090 Properly handle Preconditions check failure in MemStoreFlusher.run
2018-03-21 09:09:50 -07:00
4fcc1ac7f7
HBASE-19024 Provide a configurable option to hsync WAL edits to the disk for better durability (Harshal Jain)
2018-03-20 17:30:39 -07:00
fabb1d97cc
HBASE-20169 NPE when calling HBTU.shutdownMiniCluster
...
Adds a prepare step to RecoverMetaProcedure in which we test for
cluster up and master being up. If not up, we fail the run.
Modified hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java
Modified hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java
Minor log cleanup.
Modified hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RecoverMetaProcedure.java
Add pepare step.
Modified hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerMetrics.java
Debug for the failing test....
Added hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRecoverMetaProcedure.java
Test the prepare step goes down if master or cluster are down.
2018-03-20 13:09:43 -07:00
75ff2504ec
HBASE-20232 [LOGGING] Formatting around close and flush
2018-03-20 10:33:07 -07:00
3f1c86786c
HBASE-20213 [LOGGING] Aligning formatting and logging less (compactions,
...
in-memory compactions)
Log less. Log using same format as used elsewhere in log.
Align logs in HFileArchiver with how we format elsewhere. Removed
redundant 'region' qualifiers, tried to tighten up the emissions so
easier to read the long lines.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java
Add a label for each of the chunkcreators we make (I was confused by
two chunk creater stats emissions in log file -- didn't know that one
was for data and the other index).
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java
Formatting. Log less.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactionStrategy.java
Make the emissions in here trace-level. When more than a few regions,
log is filled with this stuff.
2018-03-16 13:07:34 -07:00
6bf967adfb
[HBASE-20141] Fix TooManyFiles exception when RefreshingChannels
...
HBASE-19435 implements a fix for reopening file channels when they are unnexpected closed
to avoid disabling the BucketCache. However, it was missed that the the channels might not
actually be completely closed (the write or read channel might still be open
(see https://docs.oracle.com/javase/7/docs/api/java/nio/channels/ClosedChannelException.html )
This commit closes any open channels before creating a new channel.
2018-03-16 10:54:43 -07:00
79d47dd57a
HBASE-20202 [AMv2] Don't move region if its a split parent or offlined
...
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/DoNotRetryRegionException.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/exceptions/MergeRegionException.java
Allow passing cause to Constructor.
M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
Add prepare step to move procedure.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
Add check that regions to merge are actually online to the Constructor
so we can fail fast if they are offline
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
Add prepare step. Check regions and context and skip move if not right.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
Add check parent region is online to constructor.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java
Add generic check region is online utility function for use by subclasses.
M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMove.java
Add test that we fail if we try to move an offlined region.
2018-03-16 09:34:15 -07:00
e0bdc87b27
HBASE-20190 Fix default for MIGRATE_TABLE_STATE_FROM_ZK_KEY
2018-03-15 10:34:31 -07:00
95596e8ba7
HBASE-20119 Introduce a pojo class to carry coprocessor information in order to make TableDescriptorBuilder accept multiple cp at once
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-16 01:26:08 +08:00
24ea9d1e48
HBASE-20146 Addendum Regions are stuck while opening when WAL is disabled
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-03-15 10:13:55 +08:00
63ce0c0733
HBASE-20178 [AMv2] Throw exception if hostile environment
...
Add Fail-Fast to Procedures by throwing exception out of Procedure
constructor so if move but table is disabled or if master is going
down, etc., we can give notice before the procedure is scheduled.
Will help guard against scheduling Procedures that will have a hard
time succeeding; e.g. a move when table is offline.
Also fixed bug around table state where we presumed ENABLED though no
entry in hbase:meta (we were using this mechanism for whether a table
existed or not).
M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMove.java
Test stolen from HBASE-20131
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableState.java
Add convenience isEnabled/isDisabled
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Promote assert state to throw exception.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
Add isClusterUp
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Move constructor now throws exception
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RestoreSnapshotProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java
Do environment check at construction and fail-fast if hostile.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java
Add preflightCheck utility method.
M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
Removed setting time setting table state; broke when using other than
default environment edge masked by presumption that no state meant
active.
2018-03-14 14:43:34 -07:00
5e2f8e4023
HBASE-20180 Avoid Class::newInstance
2018-03-14 12:23:38 -05:00
ad425e8603
HBASE-20185 Fix ACL check for MasterRpcServices#execProcedure
2018-03-14 19:08:17 +05:30
47b227b55b
HBASE-19389 Limit concurrency of put with dense (hundreds) columns to prevent write handler exhausted
...
Signed-off-by: Yu Li <liyu@apache.org>
2018-03-14 18:36:58 +08:00
c8ecfc5461
Revert "HBASE-19389 Limit concurrency of put with dense (hundreds) columns to prevent write handler exhausted"
...
This reverts commit c6d89b494c
2018-03-14 18:35:22 +08:00
c6d89b494c
HBASE-19389 Limit concurrency of put with dense (hundreds) columns to prevent write handler exhausted
...
Signed-off-by: Yu Li <liyu@apache.org>
2018-03-14 18:33:39 +08:00
a9d9fa35a2
HBASE-17165 Make use of retry setting in LoadIncrementalHFiles & fix test
2018-03-13 14:59:56 -07:00
3757915dac
HBASE-19075: Fix the 'tasks' table on master info page to not scroll up on clicking the tab
2018-03-13 14:24:56 -07:00
260ee0da60
HBASE-20173 [AMv2] DisableTableProcedure concurrent to ServerCrashProcedure can deadlock
...
Allow that DisableTableProcedue can grab a region lock before
ServerCrashProcedure can. Cater to this cricumstance where SCP
was not unable to make progress by running the search for RIT
against the crashed server a second time, post creation of all
crashed-server assignemnts. The second run will uncover such as
the above DisableTableProcedure unassign and will interrupt its
suspend allowing both procedures to make progress.
M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
Add new procedure step post-assigns that reruns the RIT finder method.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Make this important log more specific as to what is going on.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
Better explanation as to what is going on.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
Add extra step and run handleRIT a second time after we've queued up
all SCP assigns. Also fix a but. SCP was adding an assign of a RIT
that was actually trying to unassign (made the deadlock more likely).
2018-03-13 05:44:43 -07:00
4033405b36
HBASE-19449 Minor logging change in HFileArchiver
...
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-03-12 22:12:27 +05:30
842678c991
HBASE-20120 Removed unused classes/ java files from hbase-server
...
deleted: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NoOpHeapMemoryTuner.java
deleted: hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseWALEntryFilter.java
deleted: hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSMapRUtils.java
deleted: hbase-server/src/main/java/org/apache/hadoop/hbase/util/ProtoUtil.java
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-03-10 12:59:48 +08:00
1728b48c82
HBASE-20132 Change the "KV" to "Cell" for web UI
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-03-10 12:36:29 +08:00
55e3dda25d
HBASE-20024 Fixed flakyness of TestMergeTableRegionsProcedure
...
We assumed that we can run for loop from 0 to lastStep sequentially. MergeTableRegionProcedure skips step 2. So, when i is 0 the procedure is already at step 3.
Added a method StateMachineProcedure#getCurrentStateId that can be used from test code only.
2018-03-09 12:44:49 -08:00
cdf7be8929
HBASE-20146 Regions are stuck while opening when WAL is disabled
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 21:14:42 +08:00
d4e0fc74d4
HBASE-20082 Fix findbugs errors only on master which are introduced by HBASE-19397
2018-03-09 20:55:48 +08:00
55d0d3026a
HBASE-19923 Reset peer state and config when refresh replication source failed
2018-03-09 20:55:48 +08:00
d1e775e35e
HBASE-19936 Introduce a new base class for replication peer procedure
2018-03-09 20:55:48 +08:00
abaa80f26e
HBASE-19873 addendum add missing rule for new tests
2018-03-09 20:55:48 +08:00
db4cb01a9e
HBASE-19783 Change replication peer cluster key/endpoint from a not-null value to null is not allowed
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
a7dce36766
HBASE-19748 TestRegionReplicaFailover and TestRegionReplicaReplicationEndpoint UT hangs
2018-03-09 20:55:48 +08:00
1c32241a00
HBASE-19719 Fix checkstyle issues
2018-03-09 20:55:48 +08:00
1e69d7fa24
HBASE-19711 TestReplicationAdmin.testConcurrentPeerOperations hangs
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
0165455d34
HBASE-19707 Race in start and terminate of a replication source after we async start replicatione endpoint
2018-03-09 20:55:48 +08:00
d36aacdf9e
HBASE-19636 All rs should already start work with the new peer change when replication peer procedure is finished
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
53b18fe0ac
HBASE-19634 Add permission check for executeProcedures in AccessController
2018-03-09 20:55:48 +08:00
029406c45b
HBASE-19697 Remove TestReplicationAdminUsingProcedure
2018-03-09 20:55:48 +08:00
5655b3c0ca
HBASE-19661 Replace ReplicationStateZKBase with ZKReplicationStorageBase
2018-03-09 20:55:48 +08:00
b84fbde175
HBASE-19687 Move the logic in ReplicationZKNodeCleaner to ReplicationChecker and remove ReplicationZKNodeCleanerChore
2018-03-09 20:55:48 +08:00
13318dd350
HBASE-19544 Add UTs for testing concurrent modifications on replication peer
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
b0de0ec439
HBASE-19686 Use KeyLocker instead of ReentrantLock in PeerProcedureHandlerImpl
2018-03-09 20:55:48 +08:00
5d922a5e0c
HBASE-19623 Create replication endpoint asynchronously when adding a replication source
2018-03-09 20:55:48 +08:00
525fef572e
HBASE-19633 Clean up the replication queues in the postPeerModification stage when removing a peer
2018-03-09 20:55:48 +08:00
1e36a84afc
HBASE-19622 Reimplement ReplicationPeers with the new replication storage interface
2018-03-09 20:55:48 +08:00
4c6942df58
HBASE-19635 Introduce a thread at RS side to call reportProcedureDone
2018-03-09 20:55:48 +08:00
d9b5eb3abb
HBASE-19617 Remove ReplicationQueues, use ReplicationQueueStorage directly
2018-03-09 20:55:48 +08:00
b40c426806
HBASE-19642 Fix locking for peer modification procedure
2018-03-09 20:55:48 +08:00
ac07e07988
HBASE-19592 Add UTs to test retry on update zk failure
2018-03-09 20:55:48 +08:00
712b5a80dc
HBASE-19630 Add peer cluster key check when add new replication peer
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
eae251d203
HBASE-19573 Rewrite ReplicationPeer with the new replication storage interface
2018-03-09 20:55:48 +08:00
15055a421b
HBASE-19599 Remove ReplicationQueuesClient, use ReplicationQueueStorage directly
2018-03-09 20:55:48 +08:00
1de08ded58
HBASE-19543 Abstract a replication storage interface to extract the zk specific code
2018-03-09 20:55:48 +08:00
3fd417600e
HBASE-19525 RS side changes for moving peer modification from zk watcher to procedure
2018-03-09 20:55:48 +08:00
da07870995
HBASE-19580 Use slf4j instead of commons-logging in new, just-added Peer Procedure classes
2018-03-09 20:55:48 +08:00
9ead5934c0
HBASE-19520 Add UTs for the new lock type PEER
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
7c2a0d7e21
HBASE-19564 Procedure id is missing in the response of peer related operations
2018-03-09 20:55:48 +08:00
5e410d8140
HBASE-19524 Master side changes for moving peer modification from zk watcher to procedure
2018-03-09 20:55:48 +08:00
95af14fea6
HBASE-19216 Implement a general framework to execute remote procedure on RS
2018-03-09 20:55:48 +08:00
3576eb6bd8
HBASE-19598 Fix TestAssignmentManagerMetrics flaky test
2018-03-09 20:31:45 +08:00
b4e297f3f6
HBASE-20160 TestRestartCluster.testRetainAssignmentOnRestart uses the wrong condition to decide whether the assignment is finished
2018-03-09 11:08:57 +08:00
78c02c5081
HBASE-20114 Fix IllegalFormatConversionException in rsgroup.jsp
...
Repplication. Was reverted earlier.
2018-03-08 09:29:01 -08:00
71cc7869db
HBASE-20155 update branch-2 version to 2.1.0-SNAPSHOT
...
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2018-03-08 08:44:30 -08:00
25efd37f32
HBASE-20144 The shutdown of master will hang if there are no live region server
2018-03-08 15:54:40 +08:00
96a42b7359
Revert "HBASE-20137 TestRSGroups is flakey"
...
Revert. Fix is not right.
This reverts commit 6d1740d498
2018-03-07 09:25:02 -08:00
6d1740d498
HBASE-20137 TestRSGroups is flakey
...
On failed RPC we expire the server and suspend expecting the
resultant ServerCrashProcedure to wake us back up again. In tests,
TestRSGroup hung because it failed to schedule a server expiration
because the server was already expired undergoing processing (the
test was shutting down). Deal with this case by having expire
servers return false if unable to expire. Callers will then know
where a ServerCrashProcedure has been scheduled or not.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
Have expireServer return true if successful.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
The log that included an exception whose message was the current
procedure as a String totally baffled me. Make it more obvious what
exception is.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
If failed expire of a server, wake our procedure -- do not suspend --
and presume ok to move region to CLOSED state (because going down or
concurrent crashed server processing ongoing).
2018-03-05 22:53:27 -08:00
9e9b347d66
HBASE-20114 Fix IllegalFormatConversionException in rsgroup.jsp - revert premature commit
2018-03-02 12:07:16 -08:00
4ec2eeef63
HBASE-20114 Fix IllegalFormatConversionException in rsgroup.jsp
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-02 09:01:05 -08:00
9927c2e14a
HBASE-20070 refactor website generation
...
* rely on git plumbing commands when checking if we've built the site for a particular commit already
* switch to forcing '-e' for bash
* add command line switches for: path to hbase, working directory, and publishing
* only export JAVA/MAVEN HOME if they aren't already set.
* add some docs about assumptions
* Update javadoc plugin to consistently be version 3.0.0
* avoid duplicative site invocations on reactor modules
* update use of cp command so it works both on linux and mac
* manually skip enforcer plugin during build
* still doing install of all jars due to MJAVADOC-490, but then skip rebuilding during aggregate reports.
* avoid the pager on git-diff by teeing to a log file, which also helps later reviewing in the case of big changesets.
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
Conflicts:
hbase-backup/pom.xml
hbase-spark-it/pom.xml
2018-03-02 09:51:43 -06:00
e06e90bdbc
HBASE-19437 Batch operation can't handle the null result for Append/Increment
...
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-02 23:42:41 +08:00
a2de29560f
HBASE-20113 Move branch-2 version from 2.0.0-beta-2-SNAPSHOT to 2.0.0-beta-2
2018-03-01 15:46:38 -08:00
4547edbfe0
HBASE-20055 Removed declaration of un-thrown exceptions and unused setRegionStateBackToOpen() from MergeTableRegionsProcedure
...
Plus some minor cleanup.
2018-03-01 08:45:54 -08:00
a2bf2ad71e
HBASE-20093 (addendum) remove unused import of ServerLoad
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-01 23:44:29 +08:00
e0ff595744
HBASE-20107 Add a test case for HBASE-14317 (Zephyr Guo)
2018-02-28 21:18:14 -08:00
313464f007
HBASE-19863 java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
Signed-off-by: ramkrish86 <ramkrishna.s.vasudevan@gmail.com>
2018-02-28 14:06:17 -05:00
f71c00c02e
HBASE-20097 Merge TableDescriptors#getAll and TableDescriptors#getAllDescriptors into one
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-28 15:21:31 +08:00
e50e6f7ce9
HBASE-20084 Refactor the RSRpcServices#doBatchOp
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-28 15:13:09 +08:00
c459282fe0
HBASE-20093 Replace ServerLoad by ServerMetrics for ServerManager
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-28 15:05:52 +08:00
5317ca92bc
HBASE-20069 fix existing findbugs errors in hbase-server; ADDENDUM Address review
...
Revert of the revert, i.e. reapply. Thought this had broken the build
but it was a bad nightly. Putting it back.
Revert "Revert "HBASE-20069 fix existing findbugs errors in hbase-server; ADDENDUM Address review""
This reverts commit 07eae00ec1
2018-02-27 21:07:38 -08:00
2ceafe108f
HBASE-20092 Fix TestRegionMetrics#testRegionMetrics
...
Reapplication of a patch temporarily removed...
I thought this was causing issue but now I don't think it the culprit.
Revert "Revert "HBASE-20092 Fix TestRegionMetrics#testRegionMetrics""
This reverts commit 367d316781
2018-02-27 15:42:17 -08:00
7d7ca33e6d
HBASE-20100 TestEnableTableProcedure flakey
...
Allow OPEN as a possible state when update region transition state.
Usually state is OPENING but if crash before finish step is completed,
on replay, master may have read that the state is OPEN from meta table
and so will think it open... When we replay the procedure finish, allow
that the region is already OPEN.
2018-02-27 12:10:51 -08:00
06cf04e458
HBASE-20074 [FindBugs] Same code on both branches in CompactingMemStore#initMemStoreCompactor
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-27 11:23:16 -06:00
e0f8076693
Revert "for creating patch HBASE-20074-V01.patch"
...
This reverts commit a312705dbc
2018-02-27 11:22:59 -06:00
07eae00ec1
Revert "HBASE-20069 fix existing findbugs errors in hbase-server; ADDENDUM Address review"
...
This reverts commit 43d661b008
2018-02-27 08:44:22 -08:00
367d316781
Revert "HBASE-20092 Fix TestRegionMetrics#testRegionMetrics"
...
This reverts commit ae288a7b41
2018-02-27 08:26:47 -08:00
ae288a7b41
HBASE-20092 Fix TestRegionMetrics#testRegionMetrics
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-27 16:48:48 +08:00
8e8e50683d
HBASE-20066 Region sequence id may go backward after split or merge
2018-02-27 15:37:32 +08:00
43d661b008
HBASE-20069 fix existing findbugs errors in hbase-server; ADDENDUM Address review
2018-02-26 23:15:43 -08:00
44544c7db0
HBASE-20069 fix existing findbugs errors in hbase-server
2018-02-26 10:55:53 -08:00
a312705dbc
for creating patch HBASE-20074-V01.patch
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-26 09:56:44 -08:00
372c68fca2
HBASE-20036 TestAvoidCellReferencesIntoShippedBlocks timed out (Ram)
2018-02-26 22:06:28 +05:30
ef02762dd8
HBASE-20083 Fix findbugs error for ReplicationSyncUp
2018-02-26 22:16:37 +08:00
8b3ae58e18
HBASE-20043 ITBLL fails against hadoop3
...
Fix MoveRandomRegionOfTableAction. It depended on old AM behavior.
Make it do explicit move as is required in AMv3; w/o it, it was just
closing region causing test to fail.
Fix pom so hadoop3 profile specifies a different netty3 version.
Bunch of logging format change that came of trying trying to read
the spew from this test.
2018-02-24 17:29:24 -08:00
2d5065bc7e
HBASE-20065 Revisit the timestamp usage in MetaTableAccessor
2018-02-25 09:14:09 +08:00
62b8da927e
HBASE-20061 HStore synchronized member variable filesCompacting should be private
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-24 10:58:14 -06:00
a66fa85a16
Revert "HBASE-19835 Use explicit casting to avoid side effects"
...
This reverts commit bc080e7500
2018-02-23 23:04:34 -06:00
d6d1e49a75
HBASE-20054 removes the indefinite retry introduced by HBASE-18282
2018-02-23 19:14:04 -08:00
4ddfdaffdc
HBASE-20048 Revert serial replication feature
2018-02-23 13:54:10 +08:00
8a22e4119f
HBASE-20049 Region replicas of SPLIT and MERGED regions are kept in in-memory states until restarting master (Toshihiro Suzuki)
2018-02-22 20:11:11 -08:00
991e163cc2
HBASE-19401 Add missing security checks in RSRpcServices
2018-02-22 16:23:47 -08:00
161f9de8e5
HBASE-19400 Add missing security checks in MasterRpcServices
...
- Added ADMIN permission check for following rpc calls:
normalize, setNormalizerRunning, runCatalogScan, enableCatalogJanitor, runCleanerChore,
setCleanerChoreRunning, execMasterService, execProcedure, execProcedureWithRet
- Moved authorizationEnabled check to start of AccessChecker's functions. Currently, and IDK why,
we call authManager.authorize() first and then discard its result if authorizationEnabled is false. Weird.
2018-02-22 16:23:43 -08:00
f3753e394a
HBASE-20052 TestRegionOpen#testNonExistentRegionReplica fails due to NPE
2018-02-22 12:35:50 -08:00
6c039383b9
HBASE-19767 Fix for Master web UI shows negative values for Remaining KVs
...
Negative Remaining KVs and progress percent greater than 100 is because CompactionProgress#totalCompactingKVs is sometimes less than CompactionProgress#currentCompactedKVs.
Changes add a getter to CompactionProgress#totalCompactingKVs and from inside getter warning is logged. currentCompactedKVs are return when totalCompactingKVs are less than current.
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-22 10:09:47 -08:00
1db1474591
HBASE-20044 TestClientClusterStatus is flakey"
...
Revert "Revert "HBASE-2004 TestClientClusterStatus is flakey""
This is a revert of a revert, i.e. a reapplication, just so I can fix
the JIRA number.
This reverts commit 6796b8e21f
2018-02-22 09:42:37 -08:00
6796b8e21f
Revert "HBASE-2004 TestClientClusterStatus is flakey"
...
Revert to fix JIRA number; should be 20044.
This reverts commit 92bb4db9ef
2018-02-22 09:41:27 -08:00
38a97b87e6
HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory; ADDEDNDUM to fix TestRegionOpen failure
2018-02-22 09:14:35 -08:00
9d3eadd924
HBASE-20035 Stabilize the flaky TestQuotaStatusRPCs
...
The test will fail if the quota moves to violation before
the second half of the test.
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-22 09:34:05 -05:00
0432d49d83
HBASE-20038 TestLockProcedure.testTimeout is flakey
2018-02-21 17:23:49 -08:00
c06c4f30d1
HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory
2018-02-21 16:34:53 -08:00
c677c30020
HBASE-20042 TestRegionServerAbort flakey; ADDENDUM, RETRY
2018-02-21 15:26:59 -08:00
92bb4db9ef
HBASE-2004 TestClientClusterStatus is flakey
2018-02-21 14:52:10 -08:00
baec532aa2
HBASE-20042 TestRegionServerAbort flakey
2018-02-21 13:24:39 -08:00
67b7ce38fb
HBASE-20027 Add test TestClusterPortAssignment
...
LocalHBaseCluster forces random port assignment for sake of concurrent unit test
execution friendliness, but we still need a positive test for RPC and info port
assignment.
2018-02-21 13:12:30 -08:00
5c988c9960
HBASE-19166 AsyncProtobufLogWriter persists ProtobufLogWriter as class name for backward compatibility
2018-02-21 13:05:54 -08:00
d8ec10e20f
HBASE-20039 MR tests out to hbase-mapreduce mobile
2018-02-21 14:53:30 -06:00
e10db9d29a
HBASE-19953 Ensure post DDL hooks are only called after successful operations
...
The 1.x functionality of Master DDL operations is that "post" observer hooks
are only invoked when the DDL action was successful. With the async-ness of
ProcV2, we find ourselves in a case where the post-hook may be invoked before
the Procedure runs and fails. We need to introduce some blocking to wait and
see if the Procedure is going to fail on a precondition before invoking the hook.
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-21 13:53:09 -05:00
30c2dcd883
HBASE-20037 Race when calling SequenceIdAccounting.resetHighest
2018-02-21 10:04:33 -08:00
2b4df5e36e
HBASE-19506: The CellChunkMap index chunks are usually small, so in order to prevent memory underutilization, HBASE-19506 presents small chunks preallocated in a small pool
2018-02-21 10:19:49 +02:00
bc1ac49de2
HBASE-20017 BufferedMutatorImpl submit the same mutation repeatedly
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-20 16:59:48 -08:00
cbd138d41a
HBASE-19954 Separate TestBlockReorder into individual tests to avoid ShutdownHook suppression error against hadoop3
2018-02-20 14:56:46 -08:00
7e2978787d
HBASE-20029 @Ignore TestQuotaThrottle and TestReplicasClient#testCancelOfMultiGet
2018-02-20 12:08:35 -08:00
9be0360c5d
HBASE-20024 TestMergeTableRegionsProcedure is STILL flakey
2018-02-20 11:07:36 -08:00
0152d5e2ae
HBASE-20021 TestFromClientSideWithCoprocessor is flakey
2018-02-20 06:24:22 -08:00
34d3e847cc
HBASE-20020 Make sure we throw DoNotRetryIOException when ConnectionImplementation is closed
2018-02-20 06:10:45 -08:00
c1fe9f441c
HBASE-19978 The keepalive logic is incomplete in ProcedureExecutor
2018-02-19 17:13:16 -08:00
4ef6319af0
HBASE-19950 Introduce a ColumnValueFilter
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-20 05:05:19 +08:00
a458d7c400
HBASE-18294 Reduce global heap pressure: flush based on heap occupancy
2018-02-18 12:58:52 +02:00
0082f55595
HBASE-20015 TestMergeTableRegionsProcedure and TestRegionMergeTransactionOnCluster flakey
2018-02-17 11:37:42 -08:00
07bdd492f4
Revert "HBASE-20015 TestMergeTableRegionsProcedure and TestRegionMergeTransactionOnCluster flakey"
...
This reverts commit 72193650dd
2018-02-16 23:21:56 -08:00
3ea2ffcd57
HBASE-20013 TestZKPermissionWatcher is flakey
2018-02-16 23:19:07 -08:00
72193650dd
HBASE-20015 TestMergeTableRegionsProcedure and TestRegionMergeTransactionOnCluster flakey
2018-02-16 22:59:32 -08:00
34d5f2b70e
HBASE-19680 BufferedMutatorImpl#mutate should wait the result from AP in order to throw the failed mutations
2018-02-17 07:27:23 +08:00
d59959d19e
HBASE-20011 Disable TestRestoreSnapshotFromClientWithRegionReplicas; it is flakey. Needs attention.
2018-02-16 14:45:55 -08:00
138f82c8c5
HBASE-19920 Lazy init for ProtobufUtil classloader
2018-02-16 09:11:45 -06:00
70d3413ee2
HBASE-19988 Fixed chatty log from HRegion#lockRowsAndBuildMiniBatch() when interrupted while waiting on a row lock
2018-02-16 07:01:23 -08:00
9f27fdaffc
HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey
...
Remove assert in splittableregionprocedure. It was in the prepare.
Was causing fail in legit case where a region split follows a
table split BEFORE the parent has been GC'd. The region split
finds the parent in SPLIT state which is right. The assert was
having us fail. No need.
Also disabled TestHTrace since not supported in 2.0.0 and flakey.
2018-02-15 20:44:39 -08:00
13bc4fe33c
HBASE-19998 Flakey TestVisibilityLabelsWithDefaultVisLabelService
...
Only call server.checkIfShouldMoveSystemRegionAsync if a node has been
added. Do not call it if only one regionserver in cluster. Make it
so ServerCrashProcedure runs before it. Add logging if
server.checkIfShouldMoveSystemRegionAsync was responsible for
MOVE (Previous was a mystery when it cut in).
Previous we'd call it when there was a nodeChildrenChanged. These
happen before nodeDeleted. If a server crashed,
checkIfShouldMoveSystemRegionAsync could run first, find the
server that had not yet registered as crashed, find system
tables on it and then try to move them. It would fail because
server would not respond to RPC. The region move would then
be waiting on the servercrashprocedure to wake it up when
done processing but this move had locked the region so
SCP couldn't run....
2018-02-15 19:41:17 -08:00
65ac99b675
HBASE-19996 Some nonce procs might not be cleaned up (follow up HBASE-19756)
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-15 04:48:24 -08:00
af804a586b
HBASE-19930: Fixing the bug, in the rare case when there is a merge into CCM and the one of the old segments has a big cell allocated on-heap and it needs to be copied to the MSLAB
2018-02-15 14:35:41 +02:00
ce5f5e6bcd
HBASE-20000 Remove the quantum logic in FairQueue, always put high priority queue in front
2018-02-15 16:40:21 +08:00
0743bda059
HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of a KeeperException
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-14 17:23:09 -08:00
9cbf936f90
HBASE-19980 NullPointerException when restoring a snapshot after splitting a region
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-14 09:38:16 -08:00
8a3b4cdc67
HBASE-19116 Currently the tail of hfiles with CellComparator* classname makes it so hbase1 can't open hbase2 written hfiles; fix
...
Serializing, if appropriate, write the hbase-1.x version of the
Comparator to the hfile trailer so hbase-1.x files can read hbase-2.x
hfiles (they are the same format).
2018-02-14 07:46:05 -08:00
02bba3b788
HBASE-19998 Flakey TestVisibilityLabelsWithDefaultVisLabelService
2018-02-13 22:31:57 -08:00
1f3c131371
HBASE-19965 Fix flaky TestAsyncRegionAdminApi; ADDENDUM do same to TestAsyncTableAdminAPI AGAIN (make TestAsyncTableAdminAPI3... already an TestAsyncTableAdminAPI2)
2018-02-13 21:05:34 -08:00
4594f7156d
HBASE-19979 ReplicationSyncUp tool may leak Zookeeper connection
2018-02-13 08:43:52 -08:00
562402ec23
Revert "HBASE-19970 Remove unused functions from TableAuthManager."
...
This reverts commit b19531f04b
2018-02-13 06:18:35 -08:00
6923472f75
HBASE-19876 The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
2018-02-13 21:20:21 +08:00
3623089cba
HBASE-19977 FileMmapEngine allocation of byte buffers should be
...
synchronized (Ram)
2018-02-13 15:51:21 +05:30
5b95ea01da
HBASE-19965 Fix flaky TestAsyncRegionAdminApi; ADDENDUM do same to TestAsyncTableAdminAPI
2018-02-12 22:53:29 -08:00
7104364a4d
HBASE-19965 Fix flaky TestAsyncRegionAdminApi
2018-02-12 22:24:50 -08:00
b19531f04b
HBASE-19970 Remove unused functions from TableAuthManager.
...
Functions deleted: setTableUserPermissions, setTableGroupPermissions, setNamespaceUserPermissions,
setNamespaceGroupPermissions, writeTableToZooKeeper, writeNamespaceToZooKeeper
2018-02-12 16:02:48 -08:00
e94f082776
HBASE-19986 If HBaseTestClassRule timesout a test, thread dump
2018-02-12 15:13:42 -08:00
1ae64ccee0
HBASE-19972 Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink; ADDENDUM to fix TestReplicationSink
2018-02-12 10:48:21 -08:00
c7473df2c3
HBASE-16060 1.x clients cannot access table state talking to 2.0 cluster
...
This patch adds mirroring of table state out to zookeeper. HBase-1.x
clients look for table state in zookeeper, not in hbase:meta where
hbase-2.x maintains table state.
The patch also moves and refactors the 'migration' code that was put in
place by HBASE-13032.
D hbase-client/src/main/java/org/apache/hadoop/hbase/CoordinatedStateException.java
Unused.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Move table state migration code from Master startup out to
TableStateManager where it belongs. Also start
MirroringTableStateManager dependent on config.
A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MirroringTableStateManager.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
Move migration from zookeeper of table state in here. Also plumb in
mechanism so subclass can get a chance to look at table state as we do
the startup fixup full-table scan of meta.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Bug-fix. Now we create regions in CLOSED state but we fail to check
table state; were presuming table always enabled. Meant on startup
there'd be an unassigned region that never got assigned.
A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMirroringTableStateManager.java
Test migration and mirroring.
2018-02-12 08:22:14 -08:00
43f507dfa8
HBASE-19554 Dump thread info before failure
2018-02-12 16:06:50 +08:00
d793db7c7d
HBASE-19972 Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink
2018-02-12 15:15:08 +08:00
3f127063ff
HBASE-19966 The WriteEntry for WALKey maybe null if we failed to call WAL.append
2018-02-09 19:11:08 -08:00
4461cb7d7a
HBASE-19942 (addendum) Fix flaky TestSimpleRpcScheduler
2018-02-09 13:38:05 +08:00
bc080e7500
HBASE-19835 Use explicit casting to avoid side effects
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-08 12:33:08 -08:00
c93ba37dfc
HBASE-19956 Remove category as a consideration timing out tests; set all test to timeout at 10minutes regardless
2018-02-07 20:10:36 -08:00
8f1e01b6e5
HBASE-19951 Cleanup the explicit timeout value for test method
2018-02-07 16:39:54 -08:00
7d449892af
HBASE-19947 ITU should overwrite HTU local FS
2018-02-07 16:56:11 -06:00
e71b0bf19d
HBASE-19929 Call RS.stop on a session expired RS may hang
2018-02-07 15:53:08 +08:00
e2636c81f7
HBASE-19900 Region-level exception destroy the result of batch
2018-02-07 08:38:04 +08:00
f481386e3e
HBASE-19942 Fix flaky TestSimpleRpcScheduler
2018-02-07 08:00:01 +08:00
4d24b19822
HBASE-19886 Display maintenance mode in shell, web UI
2018-02-06 14:53:48 -08:00
f155da3791
HBASE-19946 More test clean up for local-mode HTU
2018-02-06 15:39:17 -06:00
89baad5bbe
HBASE-19940 TestMetaShutdownHandler flakey; ADDENDUM: yet more debug
2018-02-06 09:48:26 -08:00
ca6b37d63c
HBASE-19940 TestMetaShutdownHandler flakey; ADDENDUM: more debug
2018-02-06 06:35:02 -08:00
fb480775d5
HBASE-19944 Fix timeout TestVisibilityLabelsWithCustomVisLabService
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-02-06 20:30:16 +08:00
e4aeb46177
HBASE-19941 Flaky TestCreateTableProcedure times out in nightly, needs to LargeTests
2018-02-05 22:11:53 -08:00
b9175680b6
HBASE-19934 HBaseSnapshotException when read replicas is enabled and online snapshot is taken after region splitting (Toshihiro Suzuki)
2018-02-05 21:06:20 -08:00
53e5707224
HBASE-19939 Fixed NPE in tests TestSplitTableRegionProcedure#testSplitWithoutPONR() and testRecoveryAndDoubleExecution()
...
Value of 'htd' is null as it is initialized in the constructor but when the object is deserialized its null. Got rid of member variable htd and made it local to method.
2018-02-05 20:48:20 -08:00
8b6b3326a6
HBASE-19927 Addendum join on RegionServerThread instead of HRegionServer
2018-02-06 10:01:41 +08:00
7723a3d60d
HBASE-19940 TestMetaShutdownHandler flakey
2018-02-05 12:38:51 -08:00
a1600c1490
HBASE-19840 Flakey TestMetaWithReplicas; ADDENDUM Adding debug
2018-02-05 11:00:46 -08:00
29016bc100
HBASE-19837 Flakey TestRegionLoad; ADDENDUM Report more often and wait less time on change (also add some debug on TestMetaShutdown test)
2018-02-05 08:39:52 -08:00
f0a5f12d97
HBASE-19703 Functionality added as part of HBASE-12583 is not working after moving the split code to master
...
Co-authored-by: Michael Stack <stack@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-05 23:41:32 +08:00
514eadbe95
HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress
2018-02-05 11:29:10 +02:00
30a7e4a35c
HBASE-19554 reenable TestDLSAsyncFSWAL/TestDLSFSHLog for debugging
2018-02-05 08:58:30 +08:00
054acca866
HBASE-19927 TestFullLogReconstruction flakey
2018-02-05 08:41:37 +08:00
32d1903377
HBASE-19931 TestMetaWithReplicas failing 100% of the time in testHBaseFsckWithMetaReplicas
2018-02-04 14:37:13 -08:00
2d5b36d194
HBASE-19905 ReplicationSyncUp tool will not exit if a peer replication is disabled
...
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-02-04 18:12:46 +05:30
3b603d2c08
HBASE-19926 Use a separated class to implement the WALActionListener for Replication
2018-02-04 20:32:14 +08:00
cb138c2da8
HBASE-19914 Refactor TestVisibilityLabelsOnNewVersionBehaviorTable
2018-02-04 13:37:09 +08:00
a3233572ac
HBASE-19928 TestVisibilityLabelsOnNewVersionBehaviorTable fails
2018-02-03 10:17:16 -08:00
00653a4d18
HBASE-19919 Tidying up logging; ADDENDUM Fix tests w/ mocked Servers
2018-02-03 09:25:45 -08:00
66a11f2ac0
HBASE-19904 Break dependency of WAL constructor on Replication
2018-02-03 17:11:30 +08:00
bac4687345
HBASE-19919 Tidying up logging
2018-02-02 22:42:30 -08:00
897afb7d18
HBASE-19726 Failed to start HMaster due to infinite retrying on meta assign
2018-02-02 18:30:24 -08:00
3398637bcd
HBASE-19918 Promote TestAsyncClusterAdminApi to LargeTests
2018-02-03 09:40:58 +08:00
b0bf6f504e
HBASE-19720 Rename WALKey#getTabnename to WALKey#getTableName
2018-02-02 20:02:04 +08:00
efed668f74
HBASE-19895 Add keepDeletedCells option in ScanOptions for customizing scanInfo in pre-hooks (Ankit Singhal)
2018-02-01 18:28:05 -08:00
a165bd766a
HBASE-19841 Every HTU should be local until DFS starts
2018-02-01 20:23:56 -06:00
79ccb0b38e
HBASE-19913 Split TestStochasticLoadBalancer2
2018-02-02 08:29:33 +08:00
ea8290f3dc
HBASE-19147 TestCacheOnWrite Times Out
2018-02-01 16:15:50 -08:00
6a1ed25581
HBASE-19897 RowMutations should follow the fluent pattern
2018-02-02 05:24:08 +08:00
7fe215db5f
HBASE-19884 BucketEntryGroup's equals, hashCode and compareTo methods are not consistent
...
Move back to default equals and hashCode.
Remove compareTo and Comparator to PriorityQueue.
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-01 13:21:23 -08:00
86ca65218d
HBASE-19912 Remove useless 'writeToWAL' flag of Region#checkAndRowMutate
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-01 12:34:46 -08:00
4cdc13b86e
HBASE-19839 Fixed flakey tests TestMergeTableRegionsProcedure#testRollbackAndDoubleExecution and TestSplitTableRegionProcedure#testRollbackAndDoubleExecution
...
* Added a comment in MergeTableRegionsProcedure and SplitTableRegionProcedure explaining specific rollbacks has side effect that AssignProcedure/s are submitted asynchronously and those procedures may continue to execute even after rollback() is done.
* Updated comments in tests with correct rollback state to abort
* Added overloaded method MasterProcedureTestingUtility#testRollbackAndDoubleExecution which takes additional argument for waiting on all procedures to finish before asserting conditions
* Updated TestMergeTableRegionsProcedure#testRollbackAndDoubleExecution and TestSplitTableRegionProcedure#testRollbackAndDoubleExecution to use newly added method
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-01 12:01:57 -08:00
23471deb75
HBASE-19906 TestZooKeeper Timeout Includes move of TestQoSFunction from smalltests to mediumtests; it timedout in a run.
2018-02-01 10:53:51 -08:00
4c210eb212
HBASE-19911 Convert some tests from small to medium because they are timing out: TestNettyRpcServer, TestClientClusterStatus; ADDENDUM changes TestCheckTestClasses to be medium sized test
2018-01-31 20:28:02 -08:00
ba265a7e51
HBASE-19911 Convert some tests from small to medium because they are timing out: TestNettyRpcServer, TestClientClusterStatus
2018-01-31 20:21:57 -08:00
113432c9b9
HBASE-19910 TestBucketCache TimesOut
2018-01-31 18:17:22 -08:00
faea89ea4f
HBASE-19909 TestRegionLocationFinder Timeout
2018-01-31 17:53:48 -08:00
13f92b7daa
HBASE-19908 TestCoprocessorShortCircuitRPC Timeout....
2018-01-31 17:34:05 -08:00
7a82126f8b
Revert "HBASE-19906 TestZooKeeper Timeout"
...
Revert. Pushed prematurely.
This reverts commit 80af9bdef0
2018-01-31 15:56:26 -08:00
e6d2682f0f
HBASE-19907 TestMetaWithReplicas still flakey
2018-01-31 15:18:45 -08:00
80af9bdef0
HBASE-19906 TestZooKeeper Timeout
2018-01-31 15:18:38 -08:00
4148d7c614
HBASE-19528 - Major Compaction Tool
2018-01-31 14:37:48 -08:00
28d13c54a9
HBASE-19887 Do not overwrite the surefire junit listener property in the pom of sub modules
2018-01-31 20:49:25 +08:00
b3824b8c92
HBASE-19728 Add lock to filesCompacting in all place.
2018-01-31 14:18:27 +08:00
0ee280db22
HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey
...
Recategorize TestCoprocessorWhitelistMasterObserver as Large test
2018-01-30 06:01:50 -08:00
364650bad5
HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey
2018-01-29 21:33:16 -08:00
90a75fb052
HBASE-19888 Move branch-2 version from 2.0.0-beta-1 to 2.0.0-beta-2-SNAPSHOT
2018-01-29 14:17:54 -08:00
27d73dec8a
HBASE-19811 Fix findbugs and error-prone warnings in hbase-server (branch-2) - addendum
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-29 14:02:42 -08:00
bbf3bae72a
HBASE-19873 Add a CategoryBasedTimeout ClassRule for all UTs
2018-01-29 12:41:14 -08:00
7cbb70945a
HBASE-19840 Flakey TestMetaWithReplicas; ADDENDUM to fix Checksyte
2018-01-29 09:35:52 -08:00
77607e4961
HBASE-19840 Flakey TestMetaWithReplicas
...
Fix two issues:
# Meta Replicas can all be assigned to the same server. This
will call the test to hang when we do our kill of the server
hosting meta because there'll be no replicas to read from
as test intends. Check is to look for this condition on
startup and adjust if we come across it. Replicas cross-cut
assignment. They need work.
# Other issue was shutdown. The master started toward the
end of the test may not have come up fully by the time
shutdown is called. We could be stuck assigning the
meta replicas. Have shutdown shutdown the procedure
executor engine.
There is other cleanup and notes in the below.
M HMaster
Remove the silly stops in startup now we have real
means of shutting down Master during init.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
This replica stuff was doing stuff it shouldn't be doing
like setting core Master state flags. It may have made
sense once but now meta is assigned by a Pv2 Procedure
so the flag setting in here is meddlesome. Clear out
methods no longer needed.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Remove unused methods.
Changes local variable names so they align w/ our naming elsewhere in
code base.
M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java
Check for all replicas on the one server.
2018-01-29 09:26:41 -08:00
1efa050ebf
HBASE-19885 Promote TestAssignmentManager to LargeTests
2018-01-30 00:25:32 +08:00
f8bde7b76c
HBASE-19866 TestRegionServerReportForDuty doesn't timeout
...
Synchronized on authManager instead of RpcServer
2018-01-29 21:23:57 +08:00
5b0d7dc45b
HBASE-19882 Promote TestProcedureManager to MediumTests
2018-01-29 17:50:29 +08:00
cdda0a79ea
HBASE-19881 Promote TestRegionReplicaReplicationEndpoint to LargeTests
2018-01-29 16:15:55 +08:00
b75c85a075
HBASE-19880 Promote TestFuzzyRowFilterEndToEnd to LargeTests
2018-01-29 16:15:36 +08:00
5ef5c5a247
HBASE-19879 Promote TestAcidGuaranteesXXX to LargeTests
2018-01-29 14:57:31 +08:00
380169eaf2
HBASE-19869 TestLockManager now uses the right class for logging
2018-01-28 14:04:46 +01:00
b355af0fad
HBASE-19867 Split TestStochasticLoadBalancer into several small tests
2018-01-27 18:26:01 +08:00
4c4352c727
HBASE-19861 - Avoid using RPCs when querying table infos for master status pages
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-26 08:02:43 -08:00
c88e570dc1
HBASE-19818 Scan time limit not work if the filter always filter row key
2018-01-26 17:06:00 +08:00
c16dae13f2
HBASE-19866 Add cateogry based timeout using Timeout Rule to TestRegionServerReportForDuty.
2018-01-26 00:44:04 -08:00
f8295a810d
HBASE-19862 addendum: Checkstyle fixes, remove unused imports and variables. Add @Rule for timeout.
2018-01-26 00:25:11 -08:00
dbe5cbf5ae
HBASE-19862 Fix TestTokenAuthentication - mock RegionCoprocessorEnvironment should be of type HasRegionServerServices also.
2018-01-25 23:55:12 -08:00
c9950b5a79
HBASE-19756 Master NPE during completed failed proc eviction
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-01-24 16:43:08 -08:00
739b9b4a8e
HBASE-19838 Can not shutdown backup master cleanly when it has already tried to become the active master
...
On Master@shutdown, close the shared Master connection to kill any
ongoing RPCs by hosted clients.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Call close ont the Master shared clusterconnection to kill any ongoing
rpcs.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
Remove guts of close; we were closing the Masters connection....not
our responsibility.
Added unit test written by Duo Zhang which demonstrates the case where
Master will not go down.
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-23 11:09:42 +08:00
f952779ba2
HBASE-4224 Need a flush by regionserver rather than by table option
2018-01-23 09:47:15 +08:00
b1269ec57f
HBASE-19811 Fix findbugs and error-prone warnings in hbase-server (branch-2)
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-22 17:14:15 -08:00
1e5fc1ed63
HBASE-19840 Flakey TestMetaWithReplicas
...
Adding debuging
2018-01-22 12:26:03 -08:00
0fa24dddb9
HBASE-15321 - Ability to open a HRegion from hdfs snapshot.
2018-01-22 12:13:13 -08:00
6f8c31242a
HBASE-19774 incorrect behavior of locateRegionInMeta
2018-01-22 11:49:19 -08:00
d4ffee2023
HBASE-19837 Flakey TestRegionLoad
...
Add debug, change name of the rows so readable.
2018-01-22 08:00:26 -08:00
5cce0f8f7d
HBASE-19836 Fix TestZooKeeper.testLogSplittingAfterMasterRecoveryDueToZKExpiry
2018-01-22 18:30:11 +08:00
b44be75bd1
HBASE-19837 Flakey TestRegionLoad
...
Add debug, change name of the rows so readable.
2018-01-21 23:24:58 -08:00
c1470714bb
HBASE-19794 TestZooKeeper hangs
...
Kill backup master first
Add some cleanup around NamespaceManager
Shorten the timeout waiting on namespace manager as workaround
until we have better soln for interrupting ongoing client rpcs.
Do it in general for all tests.
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-21 20:59:31 +08:00
86ecc963e4
HBASE-19828 Flakey TestRegionsOnMasterOptions.testRegionsOnAllServers
...
Rename the PE Worker threads.
Send an interrupt if worker taking a long time to go down
(it may be RPC'ing out to a dead server, retrying so
interrupt). Also join on the ProcedureExecutor shutting down.
This will make problems shutting down more obvious.
Disable TestRegionsOnMasterOptions. Master carrying Regions is broke.
2018-01-19 21:54:44 -08:00
8f713fcc19
HBASE-19829 hadoop-minicluster pulls zookeeper:test-jar:tests 3.4.6
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-19 15:22:15 -08:00
0f9a837bd1
HBASE-19827 Addendum for Flakey TestAssignmentManager Don't clear old location from RTP...
2018-01-19 13:47:11 -08:00
3131234ffd
HBASE-19822 Override cancel(boolean) in LogCleaner (Addendum)
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-19 11:48:16 -08:00
c01dc69123
HBASE-19799 Add web UI to rsgroup
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-18 19:30:06 -08:00
a57769a15c
HBASE-19822 HFileCleaner threads stick around after shutdown stuck on queue#take Interrupt wait on queue when cancel is called.
2018-01-18 13:53:11 -08:00
13547d53eb
HBASE-19815 Flakey TestAssignmentManager.testAssignWithRandExec (Part Two).
...
Part One cleaned up a ClassCastException.
Part Two adds the ServerCrashProcedure#handleRIT behavior to RecoverMetaProcedure.
Adds debug in the test.
2018-01-18 11:32:37 -08:00
7225899e01
HBASE-19527 Make ExecutorService threads daemon=true
...
Set the ProcedureExcecutor worker threads as daemon.
Ditto for the timeout thread.
Remove hack from TestRegionsOnMasterOptions that was
put in place because the test would not go down.
2018-01-18 11:30:46 -08:00
a2d9a17851
HBASE-19821 TestCleanerChore#testOnConfigurationChange() requires at least 4 processors to get passed
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-18 11:20:52 -08:00
25621c6ea2
HBASE-19820 Restore public constructor of MiniHBaseCluster (API compat).
2018-01-18 09:07:30 -08:00
251826e138
HBASE-19806 Lower max versions for table column family of hbase:meta
2018-01-18 07:58:44 -08:00
d7e2e0d02c
HBASE-19812 TestFlushSnapshotFromClient fails because of failing region.flush
2018-01-18 16:23:15 +08:00
0f2c1fa066
HBASE-19816 Refresh repliation sinks on UnknownHostException
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-17 16:25:17 -08:00
af2d890055
Revert "HBASE-19527 Make ExecutorService threads daemon=true"
...
Applied prematurely. Revert.
This reverts commit 5e4ed33fa2
2018-01-17 15:08:42 -08:00
0f5b0fb759
HBASE-19815 Flakey TestAssignmentManager.testAssignWithRandExec
2018-01-17 13:50:42 -08:00
5e4ed33fa2
HBASE-19527 Make ExecutorService threads daemon=true
...
Set the ProcedureExcecutor worker threads as daemon.
Ditto for the timeout thread.
Remove hack from TestRegionsOnMasterOptions that was
put in place because the test would not go down.
2018-01-17 13:41:38 -08:00
addd716549
HBASE-19808 Reenable TestMultiParallel
2018-01-17 08:59:05 -08:00
f22cf7226e
HBASE-19736 Remove BaseLogCleanerDelegate deprecated #isLogDeletable(FileStatus) and use #isFileDeletable(FileStatus) instead
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-17 21:25:20 +08:00
48bb19012c
HBASE-19792 TestReplicationSmallTests.testDisableEnable fails
2018-01-17 14:23:17 +08:00
f23fd7208b
HBASE-19732 Replica regions does not return back the MSLAB chunks to pool
...
(Ram)
2018-01-15 22:15:01 +05:30
e014e5f8ef
HBASE-19797 Operator priority leads to wrong logic in ReplicationSourceWALReader
2018-01-15 18:10:56 +08:00
204a4539c4
HBASE-19793 Minor improvements on Master/RS startup
2018-01-15 17:54:01 +08:00
8dd4bf8e54
HBASE-19795 Move the tests which only need zookeeper in TestZooKeeper to hbase-zookeeper module
2018-01-15 10:58:03 +08:00
ffa28502c4
HBASE-19768 RegionServer startup failing when DN is dead
2018-01-14 17:31:23 +08:00
026f535a77
HBASE-19787 Fix or disable tests broken in branch-2 so can cut beta-1
...
M dev-support/make_rc.sh
Disable checkstyle building site. Its an issue being fixed over in HBASE-19780
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
The clusterid was being set into the process only after the
regionserver registers with the Master. That can be too late for some
test clients in particular. e.g. TestZKAsyncRegistry needs it as soon
as it goes to run which could be before Master had called its run
method which is regionserver run method which then calls back to the
master to register itself... and only then do we set the clusterid.
HBASE-19694 changed start order which made it so this test failed.
Setting the clusterid right after we set it in zk makes the test pass.
Another change was that backup masters were not going down on stop.
Backup masters were sleeping for the default zk period which is 90
seconds. They were not being woken up to check for stop. On stop
master now tells active master manager.
M hbase-server/src/test/java/org/apache/hadoop/hbase/TestJMXConnectorServer.java
Prevent creation of acl table. Messes up our being able to go down
promptly.
M hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRegionsOnMasterOptions.java
M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReadRequestMetrics.java
Disabled for now because it wants to run with regions on the Master...
currently broke!
M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestZKAsyncRegistry.java
Add a bit of debugging.
M hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDLSAsyncFSWAL.java
Disabled. Fails 40% of the time.
M hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDLSFSHLog.java
Disabled. Fails 33% of the time.
Disabled stochastic load balancer for favored nodes because it fails on
occasion and we are not doing favored nodes in branch-2.
2018-01-12 14:09:23 -08:00
d8271b0361
HBASE-19694 The initialization order for a fresh cluster is incorrect Addendum. Adds in a missing line that helps debugging.
2018-01-12 13:08:27 -08:00
72702eeb70
HBASE-19783 Change replication peer cluster key/endpoint from a not-null value to null is not allowed
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-12 21:41:57 +08:00
3e6f80dcd5
HBASE-19746 Add default impl to Cell#getType
2018-01-11 15:04:25 -08:00
1a11fc92b1
HBASE-19694 The initialization order for a fresh cluster is incorrect
...
Become active Master before calling the super class's run method. Have
the wait-on-becoming-active-Master be in-line rather than off in a
background thread (i.e. undo running thread in startActiveMasterManager)
Purge the fragile HBASE-16367 hackery that attempted to fix this issue
previously by adding a latch to try and hold up superclass RegionServer
until cluster id set by subclass Master.
2018-01-11 14:25:25 -08:00
0dacdde0f9
HBASE-19758 Split TestHCM to several smaller tests
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-11 11:40:48 -08:00
3849db8f1d
HBASE-11409 - Add more flexibility for input directory structure to LoadIncrementalHFiles
2018-01-11 11:07:47 -08:00
482d413e0a
Revert due to missing jira ticket "Add more flexibility for input directory structure to LoadIncrementalHFiles"
...
This reverts commit d524768528
2018-01-11 11:07:10 -08:00
5d4140e09d
HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by increasing scanner timeout.
...
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-01-11 10:35:36 -08:00
31fe5dbf6b
Revert due to missing author. "HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by increasing scanner timeout."
...
This reverts commit 0b9dc14dbc
2018-01-11 10:35:32 -08:00
bf3c207785
HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by increasing scanner timeout.
2018-01-11 10:32:26 -08:00
814d08a2d6
HBASE-19751 Use RegionInfo directly instead of an identifier and a namespace when getting WAL
2018-01-11 15:48:33 +08:00
98e56ae4c6
HBASE-19749 Revisit logic of UserScanQueryMatcher#mergeFilterResponse method
2018-01-11 12:06:12 +08:00
47326729fe
HBASE-19740 Repeated error message for NamespaceExistException
2018-01-10 18:26:23 -08:00
d524768528
Add more flexibility for input directory structure to LoadIncrementalHFiles
2018-01-10 13:42:34 -08:00
345b8ba9a1
HBASE-19743 Disable TestMemstoreLABWithoutPool
...
This is a revert of a revert, i.e., a reapplication only this time
I'm pushing w/ the right JIRA number
2018-01-09 18:22:36 -08:00
950ecd2764
Revert "HBASE-19742 Disable TestMemstoreLABWithoutPool"
...
Revert because have wrong JIRA number.
This reverts commit 1454c44bcb
2018-01-09 18:20:37 -08:00
1454c44bcb
HBASE-19742 Disable TestMemstoreLABWithoutPool
2018-01-09 14:32:58 -08:00
3fa3dcd9f9
HBASE-19483 Add proper privilege check for rsgroup commands
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-09 08:11:07 -08:00
895267d09c
HBASE-19139 Create Async Admin methods for Clear Block Cache
2018-01-09 14:55:21 +08:00
d9ca212d55
HBASE-19729 UserScanQueryMatcher#mergeFilterResponse should return INCLUDE_AND_SEEK_NEXT_ROW when filterResponse is INCLUDE_AND_SEEK_NEXT_ROW
2018-01-09 07:22:09 +08:00
4c31374a90
HBASE-19731 TestFromClientSide#testCheckAndDeleteWithCompareOp and testNullQualifier are flakey
2018-01-08 10:34:57 -08:00
c8e8f7008d
HBASE-19684 BlockCacheKey toString Performance
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-08 17:06:34 +08:00
1e352f6247
HBASE-19358 Improve the stability of splitting log when do fail over
...
Signed-off-by: Yu Li <liyu@apache.org>
2018-01-08 10:57:46 +08:00
a66e923106
HBASE-19712 Fix TestSnapshotQuotaObserverChore#testSnapshotSize
2018-01-08 08:52:28 +08:00
0168c1e6cb
HBASE-19696 Filter returning INCLUDE_AND_NEXT_COL doesn't skip remaining versions when scan has explicit columns (Ankit Singhal)
2018-01-07 07:50:33 -08:00
443b27b707
HBASE-19709 Ensure that we don't set a poolSize of 0
...
In some situations, Runtime.getRuntime().getAvailableProcessors()
may return 0 which would result in calculatePoolSize returning 0
which will trigger an exception. Guard against this case.
Signed-off-by: Reid Chan <reidddchan@outlook.com>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-01-05 18:36:50 -05:00
ce80e8e38f
HBASE-19651 Remove LimitInputStream; AMENDMENT: fix compile
2018-01-05 17:28:40 -06:00
88c7493948
HBASE-19651 Remove LimitInputStream
2018-01-05 17:01:50 -06:00
e7da907c3c
Revert "HBASE-19651 Remove LimitInputStream"
...
Revert to fix author.
This reverts commit 4c19e1f4e3
2018-01-05 16:59:59 -06:00
4c19e1f4e3
HBASE-19651 Remove LimitInputStream
...
Signed-off-by: Beluga Behr <dam6923@gmail.com>
2018-01-05 08:38:12 -06:00
e35fec284d
HBASE-19688 TimeToLiveProcedureWALCleaner should extends BaseLogCleanerDelegate
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-04 10:26:51 -08:00
654edc5fa5
HBASE-19596 RegionMetrics/ServerMetrics/ClusterMetrics should apply to all public classes
2018-01-04 13:05:21 +08:00
f993425f01
HBASE-19490 Rare failure in TestRateLimiter
2018-01-04 10:43:26 +08:00
59558f020f
HBASE-19613 Miscellaneous changes to WALSplitter.
...
* Use ArrayList instead LinkedList
* Use Apache Commons where appropriate
* Parameterize and improve logging
2018-01-03 18:39:14 -08:00
16e8422855
HBASE-18806 VerifyRep by snapshot need not to restore snapshot for each mapper
2018-01-04 10:12:53 +08:00
f0011ebfe0
HBASE-19691 Removes Global(A) requirement for getClusterStatus
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-03 19:02:26 -05:00
97dc7d87cd
HBASE-19654 Remove misleading and chatty debug message in ReplicationLogCleaner
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-03 06:58:57 -08:00
47a264a6aa
HBASE-19620 Add UT to confirm the race in MasterRpcServices.getProcedureResult
2018-01-03 20:29:02 +08:00
37f846dc0c
HBASE-19666 TestDefaultCompactSelection test failed
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-03 19:12:17 +08:00
76580fa991
HBASE-19675 Miscellaneous HStore Class Improvements.
...
* Use HashSet instead of List for a variable which is only used for lookups
* Remove logging code guards in favor of slf4j parameters
* Use CollectionsUtils.isEmpty() consistently
* Small check-style fixes
2018-01-02 15:21:15 -08:00
97c770a1d3
HBASE-19667 Get rid of MasterEnvironment#supportGroupCPs
2018-01-03 05:03:57 +08:00
69e161cf64
HBASE-19676 CleanerChore logging improvements (BELUGA BEHR)
2018-01-02 06:27:41 -08:00
488797f4bb
HBASE-19658: Fixing small problem and enabling test
2018-01-02 12:54:05 +02:00
99399cdeef
HBASE-19671 Fix TestMultiParallel#testActiveThreadsCount
2018-01-02 17:05:47 +08:00
3f1cfc8f08
HBASE-19679 Superusers Logging and Data Structures (BELUGA BEHR)
2018-01-01 14:19:04 -08:00
c394f3919e
HBASE-19678 HBase Admin security capabilities should be represented as a Set - revert due to wrong issue
2018-01-01 14:16:46 -08:00
c2ca90f0fb
HBASE-19678 HBase Admin security capabilities should be represented as a Set (BELUGA BEHR)
2017-12-31 11:32:14 -08:00
2f5ca9010b
HBASE-19677 Miscellaneous HFileCleaner Improvements (BELUGA BEHR)
2017-12-31 09:47:53 -08:00
5b3513a5ee
HBASE-19672 Correct comments for default values of major compaction in SortedCompactionPolicy#getNextMajorCompactTime()
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-12-30 16:28:09 -08:00
24b7fc92f0
HBASE-19649 Use singleton feature for ImmutableSegment
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-30 06:42:24 +08:00
9e141d12a4
HBASE-8518 Get rid of hbase.hstore.compaction.complete setting
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-30 04:32:12 +08:00
5612f2f692
HBASE-19428 Deprecate the compareTo(Row)
2017-12-29 20:07:21 +08:00
Lars Hofhansl
huzheng
Josh Elser
zhangduo
Guanghao Zhang
Allan Yang
David Manning
Peter Somogyi
Artem Ervits
meiyi
lixiaobao
lixiaobao
Ted Yu
Reid Chan
jingyuntian
Ben Lau
Esteban Gutierrez
Michael Stack
Beluga Behr
xcang
mazhenlin
Toshihiro Suzuki
Mike Drob
haxiaolin
subrat.mishra
Andrew Purtell
Wellington Chevreuil
Nihal Jain
Kiran Kumar Maturi
Archana Katiyar
ramie-raufdeen
Zach York
Mingliang Liu
Guangxu Cheng
Umesh Agashe
Vasudevan
Francis Liu
Tyler Mi
Huaxiang Sun
krish.dey
liubangchen
Andrey Elenskiy
Sakthi
brandboat
Monani Mihir
TAK LON WU
Wei-Chiu Chuang
Alex Leblang
Saurabh Singh
Ankit Singhal
Ankit
Mohit Goel
Yu Li
eshcar
zhaoyuan
Balazs Meszaros
Abhishek Singh Chouhan
Ted Yu
huzheng
Pankaj
Sahil Aggarwal
Todd Lipcon
Sean Busbey
taiynlee
eric-maynard
maoling
Thiruvel Thirumoolan
Apekshit Sharma
Zach York
Guanghao Zhang
Chia-Ping Tsai
Rich Fecher
Huaxiang Sun
lujie
Ashish Singhi
Jan Hentschel
Rajeshbabu Chintaguntla
Reid Chan
anoopsamjohn
Peter Somogyi
Chance Li
Mike Grimes
haxiaolin
Sergey Soldatov
gsheffi
anastas
Aman Poonia
kewang
Rahul Gidwani
binlijin
Duo Zhang
Artem Ervits
lixiang
Scott Wilson
Rahul Gidwani
Xiang Li