82e3011166
HBASE-20585 Need to clear peer map when clearing MasterProcedureScheduler
2018-05-16 08:46:34 +08:00
60b8344cf1
HBASE-20457 Return immediately for a scan rpc call when we want to switch from pread to stream
2018-05-15 21:09:04 +08:00
b7def9b690
HBASE-20447 Only fail cacheBlock if block collisions aren't related to next block metadata
...
When we pread, we don't force the read to read all of the next block header.
However, when we get into a race condition where two opener threads try to
cache the same block and one thread read all of the next block header and
the other one didn't, it will fail the open process. This is especially important
in a splitting case where it will potentially fail the split process.
Instead, in the caches, we should only fail if the required blocks are different.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-14 16:09:14 -07:00
4b0ac73f51
HBASE-20560 Revisit the TestReplicationDroppedTables ut
2018-05-14 19:33:51 +08:00
be3df29cef
HBASE-20128 Add new UTs which extends the old replication UTs but set replication scope to SERIAL
2018-05-14 19:32:39 +08:00
5ac7740896
HBASE-20411 Ameliorate MutableSegment synchronize
...
Change the MemStore size accounting so we don't synchronize across three
volatiles applying deltas. Instead:
+ Make MemStoreSize, a datastructure of our memstore size longs, immutable.
+ Undo MemStoreSizing being an instance of MemStoreSize; instead it has-a.
+ Make two MemStoreSizing implementations; one thread-safe, the other not.
+ Let all memory sizing longs run independent, untied by
synchronize (Huaxiang and Anoop suggestion) using atomiclongs.
+ Review all use of MemStoreSizing. Many are single-threaded and do
not need to be synchronized; use the non-thread safe counter.
TODO: Use this technique accounting at the global level too.
2018-05-12 02:16:19 +01:00
1f10ef553e
HBASE-20545 Improve performance of BaseLoadBalancer.retainAssignment
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-10 10:45:17 -07:00
61f96b6ffa
HBASE-20544 Make HBTU default to random ports.
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Josh Elser <elserj@apache.org>
Conflicts:
hbase-backup/src/test/resources/hbase-site.xml
hbase-spark-it/src/test/resources/hbase-site.xml
hbase-spark/src/test/resources/hbase-site.xml
2018-05-09 23:45:39 -07:00
c430016cf9
HBASE-20554 "WALs outstanding" message from CleanerChore is noisy
2018-05-09 19:11:50 -07:00
cba8d2fb8d
HBASE-20204 Add locking to RefreshFileConnections in BucketCache
...
This is a follow-up to HBASE-20141 where Anoop suggested adding locking
for refreshing channels.
2018-05-09 14:23:27 -07:00
de25f8b209
HBASE-20539 Disable IMC; part 2
...
Set default for in-memory compaction to NONE. PE was using
this value creating its table. Add test to ensure expected
default.
2018-05-09 06:29:56 -07:00
55badefa38
HBASE-20485 Copy constructor of Scan doesn't copy the readType and replicaId
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-05-09 16:45:39 +08:00
d6d209564c
HBASE-20536 Make TestRegionServerAccounting stable and it should not use absolute number
2018-05-08 08:21:30 +08:00
fd0840c3e4
HBASE-20538 TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
...
Disable test TestSaslFanOutOneBlockAsyncDFSOutput for the moment.
Depends on HDFS-13494
2018-05-07 09:54:07 -07:00
de71cb5915
HBASE-20481 Replicate entries from same region serially in ReplicationEndpoint for serial replication
2018-05-04 15:52:43 +08:00
5f260451d5
HBASE-20378 Provide a hbck option to cleanup replication barrier for a table
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-05-04 15:20:17 +08:00
eaafdff761
HBASE-20524 Need to clear metrics when ReplicationSourceManager refresh replication sources
2018-05-04 14:34:07 +08:00
e92580e201
HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer
2018-05-03 12:26:54 -07:00
f89c5b08df
HBASE-20414 TestLockProcedure#testMultipleLocks may fail on slow machine
2018-05-03 07:12:11 -07:00
984fb5bd05
HBASE-20169 NPE when calling HBTU.shutdownMiniCluster (TestAssignmentManagerMetrics is flakey); AMENDMENT
2018-05-02 16:14:38 -07:00
b0719ec11b
HBASE-20514 On Master restart if table is stuck in DISABLING state, CLOSED regions should not be considered stuck in-transition
...
On startup CLOSED regions of DISABLED and DISABLING tables are treated the same way as not in-transition.
2018-05-02 12:18:04 -07:00
b43cde2ce1
HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer
2018-05-02 10:27:00 +08:00
da3e06afab
HBASE-20492 UnassignProcedure is stuck in retry loop on region stuck in OPENING state
...
Add backoff when stuck in RegionTransitionProcedure, the subclass of
AssignProcedure and UnassignProcedure. Can happen when we go to
transition but the current Region state is not what we expect.
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
Add doc on being able to suspend and wait on a timeout.
M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
Add 'attempt' counter so we can do backoff when we get stuck.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
Add persistence of new 'attempt' counter
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
Doc data members that are persisted by subclasses given this is 'odd'.
Add a counter for 'attempts' used when 'stuck' to implement backoff.
Add suspend with timeout when 'stuck'. Add callback when timeout is
exhausted which does wakeup of this procedure.
A hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestUnexpectedStateException.java
Test of backoff.
2018-04-30 17:58:27 -07:00
d38a104c4d
HBASE-20497 The getRecoveredQueueStartPos always return 0 in RecoveredReplicationSourceShipper
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-28 20:50:47 +08:00
72093178fb
HBASE-20475 Fix the flaky TestReplicationDroppedTables unit test.
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-28 18:46:41 +08:00
e8603e1d7c
HBASE-20476 Open sequence number could go backwards in AssignProcedure
2018-04-28 16:55:06 +08:00
20e2ac132e
HBASE-20040 Fixed: Master UI should include "Cluster Key" needed to use the cluster as a replication sink
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-26 06:46:07 -05:00
75004408b0
HBASE-20466 Consistently use override mechanism for exempt classes in CoprocessorClassloader
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-23 07:48:13 -07:00
45c087d988
HBASE-20470 [2.0.0RC1] has broken unit tests...
...
Fix test that depended upon flush being slow and one family only.
Fix MemStoreSize compare to allow passing alternate implementation
(needed when IMC was no longer default everywhere).
2018-04-22 15:32:56 -07:00
298ce96246
HBASE-19924 hbase rpc throttling does not work for multi() with request count rater.
2018-04-20 16:55:23 -07:00
22e7ae0311
HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-20 11:22:49 -05:00
24eb141bac
HBASE-20442 clean up incorrect use of commons-collections 3
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
Conflicts:
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/master/BackupLogCleaner.java
2018-04-20 08:17:22 -05:00
9740168fa8
HBASE-20440 Clean up incorrect use of commons-lang 2.y
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierFactoryImpl.java
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierImpl.java
2018-04-20 08:17:22 -05:00
944ecc729c
HBASE-20439 Clean up incorrect use of commons-logging in hbase-server
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierImpl.java
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeReportingChore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeStoreImpl.java
Additions:
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCellBasedHFileOutputFormat2.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCellBasedImportExport2.java
2018-04-20 08:04:17 -05:00
af4dd3eb3c
HBASE-18059 remove scanner order related code
2018-04-18 19:37:12 -07:00
3ac0ffde4c
HBASE-18792 Disabled destructive (mostly write) operations in hbck for HBase 2.0+
...
HBase server versioned is checked after connecting to the server and then following operations are not allowed:
-fix, -fixAssignments, -fixMeta, -fixHdfsHoles, -fixHdfsOrphans, -fixTableOrphans, -fixHdfsOverlaps, -maxMerge
-sidelineBigOverlaps, -maxOverlapsToSideline, -fixSplitParents, -removeParents, -fixEmptyMetaCells
-repair, -repairHoles
2018-04-18 15:22:03 -07:00
36680da549
HBASE-19994 Create a new class for RPC throttling exception, make it retryable
2018-04-17 10:45:44 -07:00
1c8d9d788f
HBASE-20404 Fixes to CleanChore correctness and operability.
...
* Make CleanerChore less chatty: move WARN message to DEBUG when we expect non-empty dirs
* Make CleanerChore less chatty: move IOE we'll retry to INFO
* CleanerChore should treat IOE for FileStatus as a failure
* Add tests asserting assumptions in above
Signed-off-by: Reid Chan <reidddchan@outlook.com>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-17 11:53:19 -05:00
d951675df5
HBASE-20398 Redirect doesn't work on web UI
2018-04-17 09:19:13 -07:00
46c7808393
HBASE-20399 Fix merge layout
2018-04-17 09:16:58 -07:00
ecd9914dc9
HBASE-19963 TestFSHDFSUtils assumes wrong default port for Hadoop 3.0.1+
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-16 12:00:08 -05:00
f73986d110
HBASE-20417 Do not read wal entries when peer is disabled
2018-04-16 22:07:48 +08:00
1cb05a18bc
HBASE-20419 Fix potential NPE in ZKUtil#listChildrenAndWatchForNewChildren callers
...
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-16 16:07:59 +08:00
4fea862963
HBASE-20409 Set hbase.client.meta.operation.timeout in TestClientOperationTimeout
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-04-15 10:12:34 +02:00
f5d970eba1
HBASE-20294 Also cleanup last pushed sequence id in ReplicationBarrierCleaner
2018-04-14 14:56:51 +08:00
ae8a21204d
HBASE-20377 Deal with table in enabling and disabling state when modifying serial replication peer
2018-04-13 20:33:25 +08:00
b0701434e3
HBASE-20243 [Shell] Add shell command to create a new table by cloning the existent table
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-11 19:34:30 -07:00
0f6ff895e9
HBASE-20350 NullPointerException in Scanner during close()
2018-04-11 14:34:23 -07:00
0773209dd3
HBASE-20310 Fixed false inconsistency shown by hbck -metaonly option on HBase 2
...
When -metaonly option is specified then tablesInfo is not populated for all tables from hdfs causing false inconsistency of dangling state in meta.
2018-04-11 12:50:55 -07:00
f3089704d2
HBASE-20219 An error occurs when scanning with reversed=true and loadColumnFamiliesOnDemand=true
2018-04-11 12:27:28 -07:00
b52bf4052f
HBASE-20382 If RSGroups not enabled, rsgroup.jsp prints stack trace
2018-04-11 09:49:50 -07:00
c78908cff3
HBASE-20384 [AMv2] Logging format improvements; use encoded name rather than full region name marking transitions
2018-04-11 08:40:49 -07:00
3eee7d37f4
HBASE-20367 Write a replication barrier for regions when disabling a table
2018-04-11 20:36:59 +08:00
4bcb560e22
HBASE-15291 FileSystem not closed in secure bulkLoad
...
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-11 12:11:41 +05:30
2487b4603c
HBASE-20363 TestNamespaceAuditor.testRegionMerge is flaky
2018-04-09 21:02:42 +08:00
74ab10c353
HBASE-20296 Remove last pushed sequence ids when removing tables from a peer
2018-04-09 15:18:44 +08:00
ead569c951
HBASE-20285 Delete all last pushed sequence ids when removing a peer or removing the serial flag for a peer
2018-04-09 15:18:44 +08:00
8de9596355
HBASE-20127 Add UT for serial replication after failover
2018-04-09 15:18:44 +08:00
6f13b28c19
HBASE-20271 ReplicationSourceWALReader.switched should use the file name instead of the path object directly
2018-04-09 15:18:44 +08:00
9369cf26eb
HBASE-20147 Serial replication will be stuck if we create a table with serial replication but add it to a peer after there are region moves
2018-04-09 15:18:44 +08:00
aec43bb207
HBASE-20242 The open sequence number will grow if we fail to open a region after writing the max sequence id file
2018-04-09 15:18:44 +08:00
16a4dd6b8f
HBASE-20206 WALEntryStream should not switch WAL file silently
2018-04-09 15:18:44 +08:00
644bfe36b2
HBASE-20117 Cleanup the unused replication barriers in meta table
2018-04-09 15:18:44 +08:00
2d0d6a3ba1
HBASE-20167 Optimize the implementation of ReplicationSourceWALReader
2018-04-09 15:18:44 +08:00
cea5199ea1
HBASE-20148 Make serial replication as a option for a peer instead of a table
2018-04-09 15:18:44 +08:00
6b5412759f
HBASE-20125 Add UT for serial replication after region split and merge
2018-04-09 15:18:44 +08:00
8b61a061d3
HBASE-20129 Add UT for serial replication checker
2018-04-09 15:18:44 +08:00
f29bf1d778
HBASE-20115 Reimplement serial replication based on the new replication storage layer
2018-04-09 15:18:44 +08:00
1d11cdb26c
HBASE-20050 Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-09 15:18:44 +08:00
39c1ddc6e3
HBASE-20362 TestMasterShutdown.testMasterShutdownBeforeStartingAnyRegionServer is flaky
2018-04-09 14:55:42 +08:00
dcc1d9e36a
HBASE-20182 Can not locate region after split and merge
2018-04-07 13:37:59 +08:00
e78a8e08f0
HBASE-20095 Redesign single instance pool in CleanerChore - addendum simplifies onConfigurationChange
2018-04-05 10:25:09 -07:00
039bc73571
HBASE-19488 Move to using Apache commons CollectionUtils
...
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-04-04 14:16:33 -07:00
9a34880724
HBASE-16499 slow replication for small HBase clusters
...
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-04 09:59:50 +05:30
ed21f26171
HBASE-20298 Doc change in read/write/total accounting metrics
2018-04-03 17:02:27 -07:00
d4e115bf34
HBASE-20259 Doc configs for in-memory-compaction and add detail to
...
in-memory-compaction logging
Adds logging of CompactingMemStore configuration on construction.
Add logging of detail about Store on creation including memstore type.
Add chapter to refguide on new in-memory compaction feature.
2018-04-02 09:56:32 -07:00
4b909b890d
HBASE-17631 Changed Canary interval to 60 seconds
2018-04-01 13:22:37 +02:00
468d4fcd73
HBASE-20282 Clean up tooling docs/help
2018-03-30 09:00:46 -07:00
4c9167a25c
HBASE-20159 Support using separate ZK quorums for client - addendum fixes flaky TestSeparateClientZKCluster
2018-03-30 07:23:25 -07:00
1687dc5ddb
HBASE-20197 Review of ByteBufferWriterOutputStream.java (BELUGA BEHR)
...
Signed-off-by: Vasudevan <ramkrishna.s.vasudevan@gmail.com>
2018-03-30 12:20:09 +05:30
b72e19e3b9
HBASE-20159 Support using separate ZK quorums for client
2018-03-29 03:20:52 +08:00
5712fd0451
HBASE-20199 Add a unit test to verify flush and snapshot permission requirements aren't excessive
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-27 20:10:28 -04:00
6f1aa0edff
HBASE-20280 Fix possibility of deadlocking in refreshFileConnections
...
When prefetch on open is specified, there is a deadlocking case
where if the prefetch is cancelled, the PrefetchExecutor interrupts
the threads if necessary, when that happens in FileIOEngine, it
causes an ClosedByInterruptException which is a subclass of
ClosedChannelException. If we retry all ClosedChannelExceptions,
this will lock as this access is expected to be interrupted.
This change removes calling refreshFileConnections for
ClosedByInterruptExceptions.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-03-27 16:52:59 -07:00
a601c57f97
HBASE-20111 A region's splittable state now includes the configuration splitPolicy
...
The Master asks a RegionServer whether a Region can be split or not, primarily to
verify that the region is not closing, opening, etc. This change has the RegionServer
also consult the configured RegionSplitPolicy.
Signed-off-by: Josh Elser <elserj@apache.org>
2018-03-27 14:58:58 -04:00
e1afc5932a
HBASE-20229 ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on
2018-03-26 15:13:13 -07:00
c3d82a283d
HBASE-20223 Update to hbase-thirdparty 2.1.0
...
Remove commons-cli and commons-collections4 use. Account
for the newer internal protobuf version of 3.5.1.
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-03-26 16:07:39 -04:00
7c03ea0436
HBASE-20095 Redesign single instance pool in CleanerChore
2018-03-26 12:49:22 -05:00
b5881dbd3f
HBASE-17819 Reduce the heap overhead for BucketCache.
2018-03-25 16:39:21 +05:30
c8c670b561
HBASE-20272 TestAsyncTable#testCheckAndMutateWithTimeRange fails due to TableExistsException
2018-03-24 12:42:49 -07:00
a6debe4845
Revert "HBASE-19504 Add TimeRange support into checkAndMutate; ADDENDUM to fix failing unit test"
...
Reverting an addendum in favor of patch at HBASE-20272.
This reverts commit 5da92ce352
2018-03-24 12:41:26 -07:00
5da92ce352
HBASE-19504 Add TimeRange support into checkAndMutate; ADDENDUM to fix failing unit test
2018-03-24 12:29:45 -07:00
52014b9423
HBASE-13300 Fixed casing in set/getTimeStamp for Mutations
2018-03-23 17:23:55 -07:00
e962e1ac2f
HBASE-20135 Fixed NullPointerException during reading bloom filter when upgraded from hbase-1 to hbase-2
2018-03-23 15:20:43 -07:00
e6d9e04511
HBASE-20261 Table page (table.jsp) in Master UI does not show replicaIds for hbase meta table
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-03-23 13:34:08 -04:00
6aba045aae
HBASE-19504 Add TimeRange support into checkAndMutate
...
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-24 00:05:22 +08:00
96d63fee11
HBASE-20224 Web UI is broken in standalone mode
...
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
2018-03-22 20:28:08 -07:00
79e4c9d925
Revert "HBASE-20224 Web UI is broken in standalone mode"
...
Broke shell tests.
This reverts commit dd9fe813ec
2018-03-22 10:47:47 -07:00
dd9fe813ec
HBASE-20224 Web UI is broken in standalone mode
...
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
2018-03-22 06:52:51 -07:00
dd9e46bbf5
HBASE-20212 Make all Public classes have InterfaceAudience category
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-22 18:09:54 +08:00
0a94c9c250
HBASE-17449 Add explicit document on different timeout settings
2018-03-22 12:45:06 +05:30
zhangduo
Zach York
huzheng
Michael Stack
Thiruvel Thirumoolan
Sean Busbey
Andrew Purtell
Nihal Jain
Guanghao Zhang
jingyuntian
Guanghao Zhang
tedyu
Chia-Ping Tsai
Umesh Agashe
Sakthi
Rich Fecher
Huaxiang Sun
Toshihiro Suzuki
Balazs Meszaros
Wei-Chiu Chuang
lujie
Peter Somogyi
Guangxu Cheng
Apekshit Sharma
Ashish Singhi
BELUGA BEHR
Jan Hentschel
Mike Drob
Vasudevan
Yu Li
Josh Elser
Rajeshbabu Chintaguntla
Reid Chan
anoopsamjohn
Peter Somogyi