7e911d9e21
HBASE-21631: list_quotas should print human readable values for LIMIT
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-12-22 22:10:26 +08:00
c5810f19a6
HBASE-21620 Problem in scan query when using more than one column prefix filter in some cases
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Allan Yang <allan163@apache.org>
2018-12-21 15:21:12 -08:00
8f39e7633e
HBASE-21401 Sanity check when constructing the KeyValue
2018-12-21 18:34:28 +08:00
512a6322d9
HBASE-21618 Scan with the same startRow(inclusive=true) and stopRow(inclusive=false) returns one result
2018-12-21 09:54:58 +08:00
a76a842a61
HBASE-21610, numOpenConnections metric is set to -1 when zero server channel exist
2018-12-20 16:36:02 -08:00
27a0f205c5
HBASE-21498 Master OOM when SplitTableRegionProcedure new CacheConfig and instantiate a new BlockCache
2018-12-19 14:16:05 +08:00
8339e44361
HBASE-21592 quota.addGetResult(r) throw NPE
...
Signed-off-by: huzheng <openinx@gmail.com>
2018-12-18 16:40:26 +08:00
7bed742931
HBASE-21589 TestCleanupMetaWAL fails
2018-12-17 09:31:13 -08:00
a700f1b02f
HBASE-21520 TestMultiColumnScanner cost long time when using ROWCOL bloom type
2018-12-14 17:40:10 +08:00
0bb6f9e870
HBASE-21554 Show replication endpoint classname for replication peer on master web UI
2018-12-13 13:57:41 +08:00
763e395d9e
HBASE-21582 If call HBaseAdmin#snapshotAsync but forget call isSnapshotFinished, then SnapshotHFileCleaner will skip to run every time
2018-12-12 22:07:46 +08:00
911b322e9f
HBASE-21568 Use CacheConfig.DISABLED where we don't expect to have blockcache running
...
This includes removing the "old way" of disabling blockcache in favor of the
new API.
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-12-11 11:58:40 -05:00
492ca34025
HBASE-21570 Add write buffer periodic flush support for AsyncBufferedMutator
2018-12-11 15:29:56 +08:00
c9e85773d9
HBASE-21413 Empty meta log doesn't get split when restart whole cluster
...
Signed-off-by: stack <stack@apache.org>
2018-12-06 21:15:30 -08:00
45324b6bb2
HBASE-21559 The RestoreSnapshotFromClientTestBase related UT are flaky
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-12-07 08:26:02 +08:00
ec39dc8c14
Revert "Backport "HBASE-21126 Add ability for HBase Canary to ignore a configurable number of ZooKeeper down nodes" to branch-2.0"
...
This reverts commit f60617ca0f
2018-12-05 22:24:09 -08:00
f60617ca0f
Backport "HBASE-21126 Add ability for HBase Canary to ignore a configurable number of ZooKeeper down nodes" to branch-2.0
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-12-05 22:19:20 -08:00
7c1f15bd2a
HBASE-21558 Set version to 2.1.2 on branch-2.1 so can cut an RC
2018-12-05 21:24:04 -08:00
0b181af4ec
HBASE-21551 Memory leak when use scan with STREAM at server side - (addendum)
2018-12-06 11:28:28 +08:00
e9b0d73793
HBASE-21551 Memory leak when use scan with STREAM at server side
2018-12-06 10:58:04 +08:00
58cfed60ec
HBASE-21479 Individual tests in TestHRegionReplayEvents class are failing
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-12-01 09:58:14 +01:00
4ebbfa3d60
HBASE-21518 TestMasterFailoverWithProcedures is flaky
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-11-30 09:14:20 +01:00
bba29961e0
HBASE-21504 If enable FIFOCompactionPolicy, a compaction may write a "empty" hfile whose maxTimeStamp is long max. This kind of hfile will never be archived.
2018-11-30 09:55:12 +08:00
349a961554
HBASE-18735 Provide an option to kill a MiniHBaseCluster without waiting on shutdown
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-11-29 17:18:42 -05:00
001aabd40a
HBASE-21300 Fix the wrong reference file path when restoring snapshots for tables with MOB columns
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-11-29 17:53:05 +08:00
7946f96035
HBASE-21517 Move the getTableRegionForRow method from HMaster to TestMaster
2018-11-28 09:45:18 +08:00
15ebfe42a3
HBASE-21492 CellCodec Written To WAL Before It's Verified
2018-11-27 08:57:49 -08:00
a53b85ec22
HBASE-21507 Compaction failed when execute AbstractMultiFileWriter.beforeShipped() method
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
2018-11-27 20:58:39 +08:00
b81cec47ec
Revert "HBASE-21507 Compaction failed when execute AbstractMultiFileWriter.beforeShipped() method"
...
This reverts commit 13955ba532
2018-11-27 20:58:34 +08:00
13955ba532
HBASE-21507 Compaction failed when execute AbstractMultiFileWriter.beforeShipped() method
...
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Anoop Sam John <anoop.hbase@gmail.com>
2018-11-27 20:31:54 +08:00
5ec30837ac
HBASE-21511 Remove in progress snapshot check in SnapshotFileCache#getUnreferencedFiles
2018-11-25 18:27:52 -08:00
795f677983
HBASE-21387 Addendum fix TestSnapshotFileCache
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-11-24 17:00:40 +08:00
a1d0199a16
HBASE-21387 Race condition surrounding in progress snapshot handling in snapshot cache leads to loss of snapshot files
2018-11-23 11:38:26 +08:00
0557afc461
HBASE-21503 Replication normal source can get stuck due potential race conditions between source wal reader and wal provider initialization threads.
...
Found and analysed by Wellington Chevreuil
2018-11-21 17:53:41 +08:00
030d06141c
HBASE-21490 WALProcedure may remove proc wal files still with active procedures
...
Signed-off-by: Allan Yang <allan163@apache.org>
2018-11-19 08:20:49 -08:00
297d49b680
HBASE-21480 Taking snapshot when RS crashes prevent we bring the regions online
2018-11-18 21:08:00 +08:00
e5758e86a8
HBASE-21485 Add more debug logs for remote procedure execution
2018-11-16 14:32:27 +08:00
d0c2e60e36
HBASE-21440 Assign procedure on the crashed server is not properly interrupted
2018-11-14 22:33:13 -08:00
f8213a719f
HBASE-21473 RowIndexSeekerV1 may return cell with extra two \x00\x00 bytes which has no tags
2018-11-14 23:17:52 +08:00
0f295de156
HBASE-21468 separate workers for meta table is not working
2018-11-14 11:43:41 +08:00
c95832159f
HBASE-21410 A helper page that help find all problematic regions and procedures
2018-11-13 11:11:36 +08:00
e9e9d0dcef
Revert "HBASE-21401 Sanity check in BaseDecoder#parseCell"
...
This reverts commit 0ec9f81bc8
2018-11-12 11:43:37 +08:00
ffefc49e85
HBASE-21376 Add some verbose log to MasterProcedureScheduler
2018-11-10 17:17:31 +08:00
0875fa0634
HBASE-21439 RegionLoads aren't being used in RegionLoad cost functions
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java
2018-11-08 14:09:55 -08:00
3a13088a2e
HBASE-20604 ProtobufLogReader#readNext can incorrectly loop to the same position in the stream until the the WAL is rolled
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-11-08 10:22:21 -08:00
0ec9f81bc8
HBASE-21401 Sanity check in BaseDecoder#parseCell
2018-11-08 20:28:48 +08:00
0250b4b536
HBASE-21247 Custom Meta WAL Provider doesn't default to custom WAL Provider whose configuration value is outside the enums in Providers
2018-11-06 12:36:29 -08:00
6f9084380b
HBASE-21442 Update branch-2.1 for next development cycle
...
* update pom versions to 2.1.2-SNAPSHOT
* update CHANGES.md to mark release date (as of arriving in dist/release svn repo)
2018-11-06 14:19:47 -06:00
5b8c767374
HBASE-21421 Do not kill RS if reportOnlineRegions fails
2018-11-06 14:51:54 +08:00
0b7c66642b
HBASE-21423 Procedures for meta table/region should be able to execute in separate workers
2018-11-05 20:37:15 +08:00
9e52e9eb7d
HBASE-21395 Abort split/merge procedure if there is a table procedure of the same table going on
2018-11-05 20:12:00 +08:00
8df5878932
HBASE-21425 2.1.1 fails to start over 1.x data; namespace not assigned
2018-11-03 09:45:36 -07:00
46eb8f1d0d
HBASE-21351 The force update thread may have race with PE worker when the procedure is rolling back
2018-11-03 08:25:43 +08:00
3836967e05
HBASE-21407 Resolve NPE in backup Master UI
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-11-02 11:46:15 -07:00
29e3eec703
HBASE-21388 No need to instantiate MemStoreLAB for master which not carry table
2018-11-01 16:27:23 +08:00
29d6eeb6e8
HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
...
ADDENDUM
2018-10-31 10:15:04 -07:00
bddd488c34
HBASE-21237 Use CompatRemoteProcedureResolver to dispatch open/close region requests to RS
2018-10-30 17:36:44 +08:00
422e98957b
HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-29 20:56:44 -07:00
2466032fdd
HBASE-21375 Revisit the lock and queue implementation in MasterProcedureScheduler
2018-10-29 20:18:10 +08:00
066082dff4
HBASE-21397 Set version to 2.1.1 on branch-2.1 in prep for first RC
2018-10-26 12:56:24 -07:00
24f5f7afa8
HBASE-21391 RefreshPeerProcedure should also wait master initialized before executing
2018-10-26 21:45:08 +08:00
940326d8f5
Revert "HBASE-21376 Add some verbose log to MasterProcedureScheduler"
...
This reverts commit 71224ee530
2018-10-26 06:10:47 -07:00
71224ee530
HBASE-21376 Add some verbose log to MasterProcedureScheduler
2018-10-26 05:54:50 -07:00
127de9e637
HBASE-21380 Filter finished SCP at start
2018-10-25 20:28:51 -07:00
7de5f1d60d
Revert "HBASE-21380 Completed SCPs shouldn't add to dead servers in processing"
...
This reverts commit 1add6e9ca4
2018-10-25 20:26:43 -07:00
1add6e9ca4
HBASE-21380 Completed SCPs shouldn't add to dead servers in processing
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-25 19:18:15 -07:00
7c3033d704
HBASE-21385 HTable.delete request use rpc call directly instead of AsyncProcess
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-25 16:21:58 -07:00
2e9381a839
HBASE-21372) Set hbase.assignment.maximum.attempts to Long.MAX
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Allan Yang <allan163@apache.org>
2018-10-24 09:07:01 -07:00
6c9e3d0670
HBASE-21364 Procedure holds the lock should put to front of the queue after restart
2018-10-24 10:52:52 +08:00
d35f65f396
HBASE-21342 FileSystem in use may get closed by other bulk load call in secure bulkLoad
...
Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Ted Yu <tyu@apache.org>
2018-10-23 16:46:28 -05:00
ae13a5c6ea
HBASE-21349 Do not run CatalogJanitor or Nomalizer when cluster is shutting down
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-23 14:44:22 -07:00
3979aebebf
HBASE-21338 Warn if balancer is an ill-fit for cluster size
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-23 13:22:41 -07:00
7c04a95f4a
HBASE-21321 Backport HBASE-21278 to branch-2.1 and branch-2.0 ("Do not rollback successful sub procedures when rolling back a procedure")
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-10-22 21:10:11 -07:00
2173770305
HBASE-21356 bulkLoadHFile API should ensure that rs has the source hfile's write permissionls
2018-10-23 10:22:30 +08:00
65d698439f
HBASE-21073 Redo concept of maintenance mode
...
Instead of being an ephemeral state set by hbck, maintenance mode is now
an explicit toggle set by either configuration property or environment
variable. In maintenance mode, master will host system tables and not
assign any user-space tables to RSs. This gives operators the ability to
affect repairs to meta table with fewer moving parts.
2018-10-22 20:12:38 -05:00
b232746d4c
HBASE-21281 Update bouncycastle dependency - addendum adds dependency for hbase-server module
2018-10-22 17:12:11 -04:00
fedaedd6a1
HBASE-21281 Upgrade bouncycastle to latest
...
BC 1.47 introduced some incompatible API changes which came in via
a new Maven artifact. We don't use any changed API in HBase. This
also removes some unnecessary dependencies on bcprov in other
modules (presumably, they are vestiges)
Signed-off-by: Mike Drob <mdrob@apache.org>
Signed-off-by: Ted Yu <tedyu@apache.org>
2018-10-22 17:12:11 -04:00
afa7d6ed43
HBASE-21336 Addendum remove unused code in HBTU
2018-10-22 20:27:24 +08:00
fc1ef790ac
HBASE-21355 (addendum) replace the expensive reload storefiles with reading the merge result of compacted storefiles and current storefiles
2018-10-22 19:31:02 +08:00
4ded75357b
HBASE-21336 Simplify the implementation of WALProcedureMap
2018-10-22 18:36:39 +08:00
6e5d1a4896
HBASE-21334 TestMergeTableRegionsProcedure is flakey
2018-10-22 14:19:08 +08:00
492172505a
HBASE-21355 HStore's storeSize is calculated repeatedly which causing the confusing region split
2018-10-22 10:12:52 +08:00
b3a11b78f7
HBASE-21348 Fix failing TestRegionBypass, broke by HBASE-21291
2018-10-19 21:27:54 -07:00
4ad63d77be
HBASE-21345 [hbck2] Allow version check to proceed even though master is 'initializing'.
...
Just remove the check state from the getClusterStatus call.
2018-10-19 17:40:03 -07:00
a08c2c269d
HBASE-21200 Memstore flush doesn't finish because of seekToPreviousRow() in memstore scanner.
2018-10-20 08:36:41 +09:00
b3c3393c19
HBASE-21288 HostingServer in UnassignProcedure is not accurate
...
Signed-off-by: Allan Yang <allan163@apache.org>
2018-10-18 21:10:53 +08:00
34a88fca76
HBASE-21055 NullPointerException when balanceOverall() but server balance info is null
...
Signed-off-by: huzheng <openinx@gmail.com>
2018-10-18 14:08:04 +08:00
46227c2275
HBASE-21310 & HBASE-21311 Addendum fix failed UTs, some UTs are not present on branch-2.1 and some are a bit different in the implementation
2018-10-17 10:53:13 -07:00
47364d4db6
HBASE-21327 Fix minor logging issue where we don't report servername if no associated SCP
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-10-17 09:34:58 -07:00
999a3c67d4
HBASE-21320 [canary] Cleanup of usage and add commentary
...
Signed-off-by: Peter Somogyi <psomogyi@cloudera.com>
2018-10-16 22:12:13 -07:00
b0846fb762
HBASE-21311 Split TestRestoreSnapshotFromClient
2018-10-17 11:19:10 +08:00
dd836aae12
HBASE-21263 Mention compression algorithm along with other storefile details
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Amending-Author: Andrew Purtell <apurtell@apache.org>
2018-10-16 12:47:18 -07:00
cfe875d3d2
HBASE-21310 Split TestCloneSnapshotFromClient
2018-10-16 15:34:50 +08:00
467323396a
HBASE-21266 Not running balancer because processing dead regionservers, but empty dead rs list
2018-10-15 22:27:52 -07:00
a81d9be876
HBASE-21290 No need to instantiate BlockCache for master which not carry table
2018-10-15 17:30:29 +08:00
31dec21538
HBASE-21260 The whole balancer plans might be aborted if there are more than one plans to move a same region
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-10-15 15:54:34 +08:00
3d0b253248
HBASE-21309 Increase the waiting timeout for TestProcedurePriority
2018-10-15 15:27:11 +08:00
ac31ebf53a
HBASE-21271 [amv2] Don't throw UnsupportedOperationException when rollback called on Assign/Unassign; spiral of death
2018-10-12 22:25:15 -07:00
72af27b8c9
HBASE-21259 [amv2] Revived deadservers; recreated serverstatenode
...
Remove a bunch of places where we create ServerStateNode. We were
creating a SSN even though the server was long dead and processed.
The revived SSN was messing up the little dance we do unassigning
procedures. In particular, in UnassignProcedure, the check for a
dead server inside in isLogSplittingDone returns true -- we can
proceed because server is dead -- fails if an SSN exists.
We were creating SSN when we didn't need it as well as inadvertently.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
Print serverstatenode when reporting expiration. Helps debugging.
Make moveFromOnlineToDeadServers return if server online or not.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Make do w/ serverName in place of serverNode in a few places.
In waitServerReportEvent, create a ServerStateNode if none though we
should not have to at this point; to figure out later: TODO.
addRegionToServer no longer automatically calls create SSN
so do explicit create processing load meta and the region
is OPEN so we can associate OPEN regions with the SSN.
Do not schedule an SCP if server is not online, not in fs, and not in
dead servers. No point (and there may be cases where server is long
gone but hbase:meta still refers to it though it has not carried
regions in a long time; running an assign/unassign against such a
server will fail because it is not there but SCP won't clean up
the outstanding hung RPC because our region is not on the long-gone
server).
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
Just cleanup. Make it so addRegionToServer and remove can deal if no SSN.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterWalManager.java
Add isWALDirectoryNameWithWALS utility.
2018-10-12 17:40:11 -07:00
4d50f6db5a
HBASE-21242 [amv2] Miscellaneous minor log and assign procedure create improvements; ADDENDUM Fix TestHRegionInfo AND TestRegionInfoDisplay
2018-10-12 16:22:10 -07:00
266544370d
Revert "HBASE-21242 [amv2] Miscellaneous minor log and assign procedure create improvements; ADDENDUM Fix TestHRegionInfo AND TestRegionInfoDisplay"
...
This reverts commit 7f3ca4643d
2018-10-12 16:15:57 -07:00
7f3ca4643d
HBASE-21242 [amv2] Miscellaneous minor log and assign procedure create improvements; ADDENDUM Fix TestHRegionInfo AND TestRegionInfoDisplay
2018-10-12 16:09:54 -07:00
5762f879d2
Revert "HBASE-21242 [amv2] Miscellaneous minor log and assign procedure create improvements; ADDENDUM Fix TestHRegionInfo"
...
This reverts commit a9d3ac23d84dcd728ee08f4262e3d9b31df26b7e.
Let me do a better fix, one that does TestHRegionInfo and
TestHRegionInfoDisplay
2018-10-12 16:09:53 -07:00
19cb105a7e
HBASE-21242 [amv2] Miscellaneous minor log and assign procedure create improvements; ADDENDUM Fix TestHRegionInfo
2018-10-12 12:42:05 -07:00
714127b4a5
HBASE-21299 List counts of actual region states in master UI tables section
2018-10-12 10:59:44 -07:00
9b38da685c
HBASE-21289 Remove the log "'hbase.regionserver.maxlogs' was deprecated." in AbstractFSWAL
2018-10-12 21:22:31 +08:00
c3401d4327
HBASE-21254 Need to find a way to limit the number of proc wal files
2018-10-12 11:47:48 +08:00
e726a89f5f
HBASE-21287 Allow configuring test master initialization wait time.
2018-10-11 09:50:57 -05:00
e283963533
HBASE-21251 Refactor RegionMover
2018-10-10 15:28:33 +08:00
976c7ea2ef
Revert "HBASE-21271 [amv2] Don't throw UnsupportedOperationException when rollback called on Assign/Unassign; spiral of death"
...
This reverts commit c96ecbde67
2018-10-09 22:46:26 -07:00
b51aae9432
HBASE-21280 Add anchors for each heading in UI
...
Signed-off-by: Ted Yu <tedyu@apache.org>
2018-10-09 22:44:57 -07:00
c96ecbde67
HBASE-21271 [amv2] Don't throw UnsupportedOperationException when rollback called on Assign/Unassign; spiral of death
2018-10-09 00:55:02 +09:00
9a3b7f16f9
HBASE-21250 Addendum remove unused modification in hbase-server module
2018-10-08 14:56:30 +08:00
5a300f3fc9
HBASE-21250 Refactor WALProcedureStore and add more comments for better understanding the implementation
2018-10-07 17:16:09 +08:00
9d34b4581c
HBASE-21242 [amv2] Miscellaneous minor log and assign procedure create improvements
...
For RIT Duration, do better than print ms/seconds. Remove redundant UI
column dedicated to duration when we log it in the status field too.
Make bypass log at INFO level.
Make it so on complete of subprocedure, we note count of outstanding
siblings so we have a clue how much further the parent has to go before
it is done (Helpful when hundreds of servers doing SCP).
Have the SCP run the AP preflight check before creating an AP; saves
creation of thousands of APs during fixup.
Don't log tablename three times when reporting remote call failed.
If lock is held already, note who has it. Also log after we get lock
or if we have to wait rather than log on entrance though we may
later have to wait (or we may have just picked up the lock).
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-10-04 17:18:13 -07:00
8fc90a23ae
HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign
...
Adds override to assigns and unassigns. Changes bypass 'force'
to align calling the param 'override' instead.
Adds recursive to 'bypass', a means of calling bypass on
parent and its subprocedures (usually bypass works on
leaf nodes rippling the bypass up to parent -- recursive
has us work in the opposite direction): EXPERIMENTAL.
bypass on an assign/unassign leaves region in RIT and the
RegionStateNode loaded with the bypassed procedure. First
implementation had assign/unassign cleanup leftover state.
Second implementation, on feedback, keeps the state in place
as a fence against other Procedures assuming the region entity,
and instead adds an 'override' function that hbck2 can set on
assigns/unassigns to override the fencing.
Note that the below also converts ProcedureExceptions that
come out of the Pv2 system into DoNotRetryIOEs. It is a
little awkward because DNRIOE is in client-module, not
in procedure module. Previous, we'd just keep retrying
the bypass, etc.
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
Have bypass take an environment like all other methods so subclasses.
Fix javadoc issues.
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
Javadoc issues. Pass environment when we invoke bypass.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Rename waitUntilNamespace... etc. to align with how these method types
are named elsehwere .. i.e. waitFor rather than waitUntil..
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
Cleanup message we emit when we find an exisitng procedure working
against this entity.
Add support for a force function which allows Assigns/Unassigns force
ownership of the Region entity.
A hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionBypass.java
Test bypass and force.
M hbase-shell/src/main/ruby/shell/commands/list_procedures.rb
Minor cleanup of the json output... do iso8601 timestamps.
2018-10-04 16:37:37 -07:00
b0ac1c6aba
HBASE-21185 - WALPrettyPrinter: Additional useful info to be printed by wal printer tool, for debugability purposes
...
Signed-off-by: Allan Yang <allan163@apache.org>
2018-10-04 03:28:21 -07:00
3df8b6f7bb
HBASE-18549 Add metrics for failed replication queue recovery
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-10-01 18:39:07 -07:00
76a487c062
HBASE-19275 TestSnapshotFileCache never worked properly
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-10-01 17:12:21 -07:00
259d12f739
Revert "Revert "Revert "HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign"""
...
This reverts commit 2174461cf7
2018-09-29 04:06:46 -07:00
2174461cf7
Revert "Revert "HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign""
...
This reverts commit b96905d1df
2018-09-29 03:34:36 -07:00
b96905d1df
Revert "HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign"
...
This reverts commit b42d7978cb
2018-09-29 03:34:10 -07:00
b42d7978cb
HBASE-21213 [hbck2] bypass leaves behind state in RegionStates when assign/unassign
...
bypass on an assign/unassign leaves region in RIT and the
RegionStateNode loaded with the bypassed procedure. First
implementation had assign/unassign cleanup leftover state.
Second implementation, on feedback, keeps the state in place
as a fence against other Procedures assuming the region entity,
and instead adds an 'override' function that hbck2 can set on
assigns/unassigns to override the fencing.
Note that the below also converts ProcedureExceptions that
come out of the Pv2 system into DoNotRetryIOEs. It is a
little awkward because DNRIOE is in client-module, not
in procedure module. Previous, we'd just keep retrying
the bypass, etc.
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
Have bypass take an environment like all other methods so subclasses.
Fix javadoc issues.
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureExecutor.java
Javadoc issues. Pass environment when we invoke bypass.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Rename waitUntilNamespace... etc. to align with how these method types
are named elsehwere .. i.e. waitFor rather than waitUntil..
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
Cleanup message we emit when we find an exisitng procedure working
against this entity.
Add support for a force function which allows Assigns/Unassigns force
ownership of the Region entity.
A hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestRegionBypass.java
Test bypass and force.
M hbase-shell/src/main/ruby/shell/commands/list_procedures.rb
Minor cleanup of the json output... do iso8601 timestamps.
2018-09-29 03:33:07 -07:00
1f90d00614
HBASE-21248 Implement exponential backoff when retrying for ModifyPeerProcedure
2018-09-29 13:26:28 +08:00
c41003f5e6
HBASE-21196 HTableMultiplexer clears the meta cache after every put operation
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-28 16:35:57 -07:00
b7c2b953bc
HBASE-20857 balancer status tag in jmx metrics
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-28 16:12:11 -07:00
209d0a8a16
HBASE-21207 Add client side sorting functionality in master web UI for table and region server details
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-28 15:40:43 -07:00
e44ed1b1ef
HBASE-19418 configurable range of delay in PeriodicMemstoreFlusher
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-28 14:39:52 -07:00
e26a6e0e10
HBASE-18451 PeriodicMemstoreFlusher should inspect the queue before adding a delayed flush request, fix logging
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-09-28 11:50:24 -07:00
f6c05faccf
Revert "HBASE-21237 Use CompatRemoteProcedureResolver to dispatch open/close region requests to RS"
2018-09-28 14:07:40 +08:00
0290f57c3a
HBASE-21237 Use CompatRemoteProcedureResolver to dispatch open/close region requests to RS
2018-09-28 09:41:31 +08:00
eb27251265
HBASE-21228 Memory leak since AbstractFSWAL caches Thread object and never clean later
2018-09-27 15:07:07 +08:00
5169cfc8c3
HBASE-21232 Show table state in Tables view on Master home page
2018-09-26 10:57:23 -07:00
504286d55c
HBASE-20734 Colocate recovered edits directory with hbase.wal.dir
...
Amending-Author: Reid Chan <reidchan@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-09-26 19:37:53 +08:00
ba8a252167
HBASE-21212 Wrong flush time when update flush metric
2018-09-26 19:11:23 +08:00
fea75742b4
HBASE-21164 reportForDuty should do backoff rather than retry
...
Remove unused methods from Sleeper (its ok, its @Private).
Remove notion of startTime from Sleeper handling (it is is unused).
Allow passing in how long to sleep so can maintain externally.
In HRS, use a RetryCounter to calculate backoff sleep time for when
reportForDuty is failing against a struggling Master.
2018-09-25 11:31:39 -07:00
101205345b
Amend HBASE-20704 Sometimes some compacted storefiles are not archived on region close
...
Forward port small logging improvements from branch-1 version of this change.
2018-09-21 16:12:51 -07:00
a22aec1dad
HBASE-21214 [hbck2] setTableState just sets hbase:meta state, not in-memory state
2018-09-21 16:03:58 -07:00
5a73a1ab25
HBASE-21206 Scan with batch size may return incomplete cells
2018-09-20 22:20:02 +08:00
c5af7b654b
HBASE-21204 NPE when scan raw DELETE_FAMILY_VERSION and codec is not set
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-09-20 06:59:43 -07:00
e6c7ed34e0
HBASE-21023 Added bypassProcedure() API to HbckService
2018-09-19 15:01:29 -07:00
37cc07a772
HBASE-21156 [hbck2] Queue an assign of hbase:meta and bulk assign/unassign
...
Adds 'raw' assigns and unassigns methods to Hbck Service.
Fixes HbckService so it works when cluster is Kerberized.
2018-09-19 09:02:43 -07:00
27b772ddc6
HBASE-21102 ServerCrashProcedure should select target server where no
...
other replicas exist for the current region (Ram)
2018-09-17 22:36:50 +05:30
39e0b8515f
HBASE-21191 Add a holding-pattern if no assign for meta or namespace (Can happen if masterprocwals have been cleared).
...
Add a check for hbase:meta being online before we go to read it.
If not online, move into a holding-pattern until rectified, probably
by external operator.
Incorporates bulk of patch made by Allan Yang over on HBASE-21035.
M hbase-common/src/main/java/org/apache/hadoop/hbase/util/RetryCounterFactory.java
Add a Constructor for case where retries are for ever.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Move stuff around so that the first hbase:meta read is the AM#loadMeta.
Previously, checking table state and/or favored nodes could end up
trying to read a meta that was not onlined holding up master startup.
Do similar for the namespace table. Adds new methods isMeta and
isNamespace which check that the regions/tables are online.. if not,
we wait logging with a back-off that assigns need to be run.
Signed-off-by: Allan Yang <allan163@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-09-16 21:12:59 -07:00
a925a4ce16
HBASE-20704 Sometimes some compacted storefiles are not archived on region close
2018-09-16 18:38:03 -07:00
842e0c974d
HBASE-21097 Flush pressure assertion may fail in testFlushThroughputTuning
...
Amending-Author: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-09-15 18:39:42 +08:00
589c1e4078
HBASE-20941 Created and implemented HbckService in master
...
Added API setTableStateInMeta() to update table state only in Meta. This will be used by hbck2 tool.
2018-09-12 21:31:13 -07:00
d81e806718
HBASE-21168 Insecure Randomness in BloomFilterUtil
...
Flagged by Fortify static analysis
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-09-12 09:52:41 -05:00
2da6dbe563
HBASE-21172 Reimplement the retry backoff logic for ReopenTableRegionsProcedure
2018-09-12 16:01:55 +08:00
75a7643b11
Backport "HBASE-21126 Add ability for HBase Canary to ignore a configurable number of ZooKeeper down nodes" to branch-2.1
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-09-12 10:01:28 +08:00
63ef89bff7
HBASE-21125 Backport 'HBASE-20942 Improve RpcServer TRACE logging' to branch-2.1
...
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2018-09-12 09:59:28 +08:00
b9d74f89ff
Revert "HBASE-20942 Fix ArrayIndexOutOfBoundsException for RpcServer TRACE logging"
...
This reverts commit 69756da503
2018-09-12 09:55:46 +08:00
69756da503
HBASE-20942 Fix ArrayIndexOutOfBoundsException for RpcServer TRACE logging
...
Also makes the trace log message length configurable.
Signed-off-by: Josh Elser <elserj@apache.org>
2018-09-12 09:44:22 +08:00
1c8c7e10f8
HBASE-21158 Empty qualifier cell is always returned when using QualifierFilter
2018-09-10 21:40:57 +08:00
6ab9997d1f
HBASE-21144 AssignmentManager.waitForAssignment is not stable
2018-09-10 17:28:57 +08:00
12ffa086c7
HBASE-21001 ReplicationObserver fails to load in HBase 2.0.0
2018-09-07 23:43:10 +08:00
5324911cd8
HBASE-21155 Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir; ADDENDUM
...
Address review comments.
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-09-06 17:24:03 -07:00
205783419c
HBASE-21155 Save on a few log strings and some churn in wal splitter by skipping out early if no logs in dir
2018-09-06 16:36:59 -07:00
2051b0982d
HBASE-20741 Split of a region with replicas creates all daughter regions
...
and its replica in same server (Ram)
2018-09-06 16:44:59 +05:30
c64814ec96
HBASE-20892 [UI] Start / End keys are empty on table.jsp
2018-09-05 09:37:25 +08:00
e33591515c
HBASE-21083 Introduce a mechanism to bypass the execution of a stuck procedure
2018-08-28 20:18:47 -07:00
4340930c71
HBASE-20649 Validate HFiles do not have PREFIX_TREE DataBlockEncoding; ADDEDNDUM ADD MISSING FILE
2018-08-28 07:45:27 -07:00
147694bb08
HBASE-20649 Validate HFiles do not have PREFIX_TREE DataBlockEncoding
2018-08-28 07:09:47 -07:00
c1cd6d5a89
HBASE-21088 HStoreFile should be closed in HStore#hasReferences
2018-08-27 20:31:50 -07:00
e826e3f2b8
HBASE-21120 MoveRegionProcedure makes no progress; goes to STUCK
2018-08-27 14:55:52 -07:00
625be5137e
HBASE-21072 Addendum do not write lock file when running TestHBaseFsckReplication
2018-08-27 21:05:16 +08:00
33fa32d711
HBASE-21113 Apply the branch-2 version of HBASE-21095, The timeout retry logic for several procedures are broken after master restarts(addendum)
2018-08-26 22:15:49 +08:00
d954031d50
HBASE-21078 [amv2] CODE-BUG NPE in RTP doing Unassign
2018-08-24 13:22:16 -07:00
e26ca63f88
Revert "Revert "HBASE-21095 The timeout retry logic for several procedures are broken after master restarts""
...
HBASE-21113 Apply the branch-2 version of HBASE-21095, The timeout retry
logic for several procedures are broken after master restarts
I applied the patch HBASE-21095 and then reverted it so could apply the
patch as HBASE-21113 (by reverting the HBASE-21095 revert but pushing
with this message!).
This reverts commit 4978db8102
2018-08-24 12:35:29 -07:00
4978db8102
Revert "HBASE-21095 The timeout retry logic for several procedures are broken after master restarts"
...
This reverts commit b82cd670c3
2018-08-24 12:24:32 -07:00
b82cd670c3
HBASE-21095 The timeout retry logic for several procedures are broken after master restarts
2018-08-24 12:20:43 -07:00
66add55234
HBASE-21072 Block out HBCK1 in hbase2
...
Write the hbase-1.x hbck1 lock file to block out hbck1 instances writing
state to an hbase-2.x cluster (could do damage).
Set hbase.write.hbck1.lock.file to false to disable this writing.
2018-08-24 09:22:53 -07:00
8a9acd4d2a
HBASE-21101 Remove the waitUntilAllRegionsAssigned call after split in TestTruncateTableProcedure
2018-08-24 10:35:10 +08:00
bf21a9dc33
HBASE-20193 Move TestCreateTableProcedure.testMRegions to a separated file
2018-08-24 10:09:31 +08:00
239d12dae8
HBASE-20194 Remove the explicit timeout config for TestTruncateTableProcedure
2018-08-23 06:27:41 +08:00
16ab716134
HBASE-21041 Memstore's heap size will be decreased to minus zero after flush
2018-08-22 22:54:14 +08:00
c07afa8875
HBASE-21031 Memory leak if replay edits failed during region opening
2018-08-22 22:13:26 +08:00
5f03be4675
HBASE-21032 ScanResponses contain only one cell each
...
Amending-Author: Duo Zhang <zhangduo@apache.org>
2018-08-21 13:31:18 -07:00
798cb1d793
HBASE-20940 HStore.cansplit should not allow split to happen if it has references (Vishal Khandelwal)
2018-08-17 15:02:26 -07:00
67ad0e6013
HBASE-21062 Correctly use the defaultProvider value on the Providers enum when constructing a WALProvider
2018-08-17 14:55:42 -04:00
48dee7e44d
HBASE-20705 Having RPC quota on a table now no longer prevents Space Quota to be recreate/removed
...
Just added 2 test cases as the subtasks of this jira solves the issue
Signed-off-by: Josh Elser <elserj@apache.org>
2018-08-17 14:09:26 -04:00
b49941012a
HBASE-21047 Object creation of StoreFileScanner thru constructor and close may leave refCount to -1 (Vishal Khandelwal)
2018-08-16 11:42:54 -07:00
145c92f3d6
HBASE-20469 Directory used for sidelining old recovered edits files should be made configurable
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-08-15 18:08:15 -07:00
2e5efa690a
HBASE-20772 Controlled shutdown fills Master log with the disturbing message 'No matching procedure found for rit=OPEN, location=ZZZZ, table=YYYYY, region=XXXX transition to CLOSED'
...
Look for the particular case where RS does the close of region w/o
involving Master and log special message in this case. Dodgy. But
until we have Master run shutdown of all regions, better than
the message we currently show.
2018-08-13 15:59:39 -07:00
161c018927
HBASE-21029 Miscount of memstore's heap/offheap size if same cell was put
2018-08-13 20:30:23 +08:00
95e3dec510
HBASE-20985 add two attributes when we do normalization
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-13 16:55:19 +08:00
846078f9b0
HBASE-21025 Addendum missed a 'succ = true'
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-13 10:55:18 +08:00
873d9f5082
HBASE-21012 Revert the change of serializing TimeRangeTracker
...
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-08-11 22:28:49 +08:00
5e12d6a98e
HBASE-21018 RS crashed because AsyncFS was unable to update HDFS data encryption key
2018-08-10 19:53:22 -07:00
ee164fcbc5
HBASE-21025 Add cache for TableStateManager
2018-08-10 21:11:53 +08:00
8a9ba0c65b
HBASE-18201 add UT and docs for DataBlockEncodingTool
...
Signed-off-by: Reid Chan <reidchan@apache.org>
2018-08-10 11:19:36 +08:00
e222686294
HBASE-20965 Separate region server report requests to new handlers
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-09 18:27:38 +08:00
b2fc0f48f6
HBASE-20813 Removed RPC quotas when the associated table/Namespace is dropped off
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-08-08 13:46:25 -04:00
9d594ac86a
HBASE-20986 Separate the config of block size when we do log splitting and write Hlog
...
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2018-08-07 14:03:03 +08:00
7e9f8c60e2
HBASE-20885 Removed entry for RPC quota from hbase:quota when RPC quota is removed
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-08-03 11:07:01 -04:00
2e1c12ca1b
HBASE-20856 PITA having to set WAL provider in two places
...
With this change if hbase.wal.meta_provider is not explicitly set,
it uses whatever set with hbase.wal.provider. this change avoids a use
case of unexpectedly using two different providers when only
hbase.wal.provider is set to non-default but not hbase.wal.meta_provider.
This change also include document (architecture.adoc) update
Also, this is a port from master to branch-2
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <Apache9@apache.org>
2018-08-01 14:45:11 -07:00
88f3148810
HBASE-20989 Minor, miscellaneous logging fixes
...
Signed-off-by: Zach York <zyork@amazon.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2018-08-01 11:20:01 -07:00
9338eaee65
HBASE-20794 add INFO level log to createTable operation
2018-08-01 11:04:00 -07:00
0f4e857c7a
HBASE-20893 Data loss if splitting region while ServerCrashProcedure executing ADDENDUM: Rather than rollback, just do region reopens.
...
In split, reopen the parent if recovered.edits and in merge, reopen the
parent region or regions that happened to have recovered.edits on close.
2018-08-01 00:33:12 -07:00
daeec8657e
HBASE-20935 HStore.removeCompactedFiles should log in case it is unable to delete a file (Vishal Khandelwal)
...
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
2018-07-31 16:06:15 -07:00
Sakthi
openinx
Guanghao Zhang
Pankaj
xuqinya
stack
Josh Elser
zhangduo
Allan Yang
stack
David Manning
Peter Somogyi
Artem Ervits
meiyi
BELUGA BEHR
lixiaobao
lixiaobao
Ted Yu
Ankit Singhal
jingyuntian
Ben Lau
Esteban Gutierrez
Sean Busbey
Mike Drob
mazhenlin
xcang
Toshihiro Suzuki
haxiaolin
subrat.mishra
Andrew Purtell
Wellington Chevreuil
Nihal Jain
Kiran Kumar Maturi
Archana Katiyar
ramie-raufdeen
Zach York
Mingliang Liu
Umesh Agashe
Vasudevan
Francis Liu
krish.dey
Guangxu Cheng
Balazs Meszaros
Andrey Elenskiy
Sakthi
brandboat
Wei-Chiu Chuang
TAK LON WU