2dc51934f4
HBASE-20597 Serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
2018-05-29 11:29:12 -07:00
7f154dc20e
Revert "HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint"
...
This reverts commit 60dcef289b
2018-05-29 11:24:30 -07:00
d36cce1574
HBASE-20633 Dropping a table containing a disable violation policy fails to remove the quota upon table delete
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-05-29 11:50:40 -04:00
aa00391140
HBASE-20390 ADDENDUM 2: fix TestHRegionWithInMemoryFlush OOME
2018-05-29 16:24:27 +03:00
cf1928aaca
HBASE-20390-ADDENDUM: fix TestHRegionWithInMemoryFlush OOME
2018-05-29 13:01:07 +03:00
c8fd6e0fb6
HBASE-20533 Fix the flaky TestAssignmentManagerMetrics
2018-05-29 09:50:04 +08:00
0455e75edd
HBASE-20648 HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-28 08:27:41 -07:00
bc72fcd8c5
HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-27 18:43:08 -07:00
f40c10a211
HBASE-20518 Need to serialize the enabled field for UpdatePeerConfigProcedure
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-05-25 14:45:49 +08:00
d1cbd561df
HBASE-20548 Master fails to startup on large clusters, refreshing block distribution
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-24 15:47:22 -07:00
db8789ab22
HBASE-20616 TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-24 15:17:01 -07:00
55c4791a63
HBASE-20588 Space quota change after quota violation doesn't seem to take in effect
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-05-24 12:50:22 -04:00
079e08d7c5
HBASE-20390: IMC Default Parameters for 2.0.0
2018-05-24 18:10:10 +03:00
bfab1e2f92
HBASE-20589 Don't need to assign meta to a new RS when standby master become active
2018-05-24 11:45:59 +08:00
a94c6dbadb
HBASE-20624 Race in ReplicationSource which causes walEntryFilter being null when creating new shipper
2018-05-24 10:48:35 +08:00
60dcef289b
HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
2018-05-23 16:46:20 -07:00
afddf6b1c2
HBASE-20620 HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; part 2
...
Adds new stripped-down, faster ByteBufferKeyValue comparator
(BBKV is the base Cell-type in hbase2). Creates an instance
of new Comparator each time we create new memstore rather
than use the universal CellComparator.
Remove unused and unneeded Interfaces from Cell base type.
2018-05-23 13:20:29 -07:00
fbda502435
HBASE-20612 TestReplicationKillSlaveRSWithSeparateOldWALs sometimes fail because it uses an expired cluster conn
2018-05-23 12:07:54 +08:00
c3c9a4a595
HBASE-20579 Improve snapshot manifest copy in ExportSnapshot
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-18 06:43:53 -07:00
d06673cf3e
HBASE-20583 SplitLogWorker should handle FileNotFoundException when split a wal
2018-05-18 14:30:40 +08:00
39ea1efa88
HBASE-20571 JMXJsonServlet generates invalid JSON if it has NaN in metrics
...
- CacheStats won't generate NaN metrics.
- JSONBean class will serialize special floating point values as
"NaN", "Infinity" or "-Infinity"
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-16 12:20:37 -07:00
61f2b5f071
HBASE-20567 Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace.
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-05-16 14:03:36 -05:00
77eaff0e10
HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; ADDENDUM
...
Add method the CellComparator Interface. Add implementation to
meta comparator so we don't fall back to the default comparator.
Includes a nothing change to hbase-server/pom.xml just to provoke
build.
2018-05-16 09:42:51 -07:00
5c4685e56e
HBASE-20520 Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
2018-05-16 09:18:06 -07:00
82e3011166
HBASE-20585 Need to clear peer map when clearing MasterProcedureScheduler
2018-05-16 08:46:34 +08:00
60b8344cf1
HBASE-20457 Return immediately for a scan rpc call when we want to switch from pread to stream
2018-05-15 21:09:04 +08:00
b7def9b690
HBASE-20447 Only fail cacheBlock if block collisions aren't related to next block metadata
...
When we pread, we don't force the read to read all of the next block header.
However, when we get into a race condition where two opener threads try to
cache the same block and one thread read all of the next block header and
the other one didn't, it will fail the open process. This is especially important
in a splitting case where it will potentially fail the split process.
Instead, in the caches, we should only fail if the required blocks are different.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-14 16:09:14 -07:00
4b0ac73f51
HBASE-20560 Revisit the TestReplicationDroppedTables ut
2018-05-14 19:33:51 +08:00
be3df29cef
HBASE-20128 Add new UTs which extends the old replication UTs but set replication scope to SERIAL
2018-05-14 19:32:39 +08:00
5ac7740896
HBASE-20411 Ameliorate MutableSegment synchronize
...
Change the MemStore size accounting so we don't synchronize across three
volatiles applying deltas. Instead:
+ Make MemStoreSize, a datastructure of our memstore size longs, immutable.
+ Undo MemStoreSizing being an instance of MemStoreSize; instead it has-a.
+ Make two MemStoreSizing implementations; one thread-safe, the other not.
+ Let all memory sizing longs run independent, untied by
synchronize (Huaxiang and Anoop suggestion) using atomiclongs.
+ Review all use of MemStoreSizing. Many are single-threaded and do
not need to be synchronized; use the non-thread safe counter.
TODO: Use this technique accounting at the global level too.
2018-05-12 02:16:19 +01:00
1f10ef553e
HBASE-20545 Improve performance of BaseLoadBalancer.retainAssignment
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-10 10:45:17 -07:00
61f96b6ffa
HBASE-20544 Make HBTU default to random ports.
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Josh Elser <elserj@apache.org>
Conflicts:
hbase-backup/src/test/resources/hbase-site.xml
hbase-spark-it/src/test/resources/hbase-site.xml
hbase-spark/src/test/resources/hbase-site.xml
2018-05-09 23:45:39 -07:00
c430016cf9
HBASE-20554 "WALs outstanding" message from CleanerChore is noisy
2018-05-09 19:11:50 -07:00
cba8d2fb8d
HBASE-20204 Add locking to RefreshFileConnections in BucketCache
...
This is a follow-up to HBASE-20141 where Anoop suggested adding locking
for refreshing channels.
2018-05-09 14:23:27 -07:00
de25f8b209
HBASE-20539 Disable IMC; part 2
...
Set default for in-memory compaction to NONE. PE was using
this value creating its table. Add test to ensure expected
default.
2018-05-09 06:29:56 -07:00
55badefa38
HBASE-20485 Copy constructor of Scan doesn't copy the readType and replicaId
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-05-09 16:45:39 +08:00
d6d209564c
HBASE-20536 Make TestRegionServerAccounting stable and it should not use absolute number
2018-05-08 08:21:30 +08:00
fd0840c3e4
HBASE-20538 TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
...
Disable test TestSaslFanOutOneBlockAsyncDFSOutput for the moment.
Depends on HDFS-13494
2018-05-07 09:54:07 -07:00
de71cb5915
HBASE-20481 Replicate entries from same region serially in ReplicationEndpoint for serial replication
2018-05-04 15:52:43 +08:00
5f260451d5
HBASE-20378 Provide a hbck option to cleanup replication barrier for a table
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-05-04 15:20:17 +08:00
eaafdff761
HBASE-20524 Need to clear metrics when ReplicationSourceManager refresh replication sources
2018-05-04 14:34:07 +08:00
e92580e201
HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer
2018-05-03 12:26:54 -07:00
f89c5b08df
HBASE-20414 TestLockProcedure#testMultipleLocks may fail on slow machine
2018-05-03 07:12:11 -07:00
984fb5bd05
HBASE-20169 NPE when calling HBTU.shutdownMiniCluster (TestAssignmentManagerMetrics is flakey); AMENDMENT
2018-05-02 16:14:38 -07:00
b0719ec11b
HBASE-20514 On Master restart if table is stuck in DISABLING state, CLOSED regions should not be considered stuck in-transition
...
On startup CLOSED regions of DISABLED and DISABLING tables are treated the same way as not in-transition.
2018-05-02 12:18:04 -07:00
b43cde2ce1
HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer
2018-05-02 10:27:00 +08:00
da3e06afab
HBASE-20492 UnassignProcedure is stuck in retry loop on region stuck in OPENING state
...
Add backoff when stuck in RegionTransitionProcedure, the subclass of
AssignProcedure and UnassignProcedure. Can happen when we go to
transition but the current Region state is not what we expect.
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
Add doc on being able to suspend and wait on a timeout.
M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
Add 'attempt' counter so we can do backoff when we get stuck.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
Add persistence of new 'attempt' counter
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
Doc data members that are persisted by subclasses given this is 'odd'.
Add a counter for 'attempts' used when 'stuck' to implement backoff.
Add suspend with timeout when 'stuck'. Add callback when timeout is
exhausted which does wakeup of this procedure.
A hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestUnexpectedStateException.java
Test of backoff.
2018-04-30 17:58:27 -07:00
d38a104c4d
HBASE-20497 The getRecoveredQueueStartPos always return 0 in RecoveredReplicationSourceShipper
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-28 20:50:47 +08:00
72093178fb
HBASE-20475 Fix the flaky TestReplicationDroppedTables unit test.
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-28 18:46:41 +08:00
e8603e1d7c
HBASE-20476 Open sequence number could go backwards in AssignProcedure
2018-04-28 16:55:06 +08:00
20e2ac132e
HBASE-20040 Fixed: Master UI should include "Cluster Key" needed to use the cluster as a replication sink
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-26 06:46:07 -05:00
75004408b0
HBASE-20466 Consistently use override mechanism for exempt classes in CoprocessorClassloader
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-23 07:48:13 -07:00
45c087d988
HBASE-20470 [2.0.0RC1] has broken unit tests...
...
Fix test that depended upon flush being slow and one family only.
Fix MemStoreSize compare to allow passing alternate implementation
(needed when IMC was no longer default everywhere).
2018-04-22 15:32:56 -07:00
298ce96246
HBASE-19924 hbase rpc throttling does not work for multi() with request count rater.
2018-04-20 16:55:23 -07:00
22e7ae0311
HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-20 11:22:49 -05:00
24eb141bac
HBASE-20442 clean up incorrect use of commons-collections 3
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
Conflicts:
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/master/BackupLogCleaner.java
2018-04-20 08:17:22 -05:00
9740168fa8
HBASE-20440 Clean up incorrect use of commons-lang 2.y
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierFactoryImpl.java
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierImpl.java
2018-04-20 08:17:22 -05:00
944ecc729c
HBASE-20439 Clean up incorrect use of commons-logging in hbase-server
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierImpl.java
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeReportingChore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeStoreImpl.java
Additions:
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCellBasedHFileOutputFormat2.java
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCellBasedImportExport2.java
2018-04-20 08:04:17 -05:00
af4dd3eb3c
HBASE-18059 remove scanner order related code
2018-04-18 19:37:12 -07:00
3ac0ffde4c
HBASE-18792 Disabled destructive (mostly write) operations in hbck for HBase 2.0+
...
HBase server versioned is checked after connecting to the server and then following operations are not allowed:
-fix, -fixAssignments, -fixMeta, -fixHdfsHoles, -fixHdfsOrphans, -fixTableOrphans, -fixHdfsOverlaps, -maxMerge
-sidelineBigOverlaps, -maxOverlapsToSideline, -fixSplitParents, -removeParents, -fixEmptyMetaCells
-repair, -repairHoles
2018-04-18 15:22:03 -07:00
36680da549
HBASE-19994 Create a new class for RPC throttling exception, make it retryable
2018-04-17 10:45:44 -07:00
1c8d9d788f
HBASE-20404 Fixes to CleanChore correctness and operability.
...
* Make CleanerChore less chatty: move WARN message to DEBUG when we expect non-empty dirs
* Make CleanerChore less chatty: move IOE we'll retry to INFO
* CleanerChore should treat IOE for FileStatus as a failure
* Add tests asserting assumptions in above
Signed-off-by: Reid Chan <reidddchan@outlook.com>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-17 11:53:19 -05:00
d951675df5
HBASE-20398 Redirect doesn't work on web UI
2018-04-17 09:19:13 -07:00
46c7808393
HBASE-20399 Fix merge layout
2018-04-17 09:16:58 -07:00
ecd9914dc9
HBASE-19963 TestFSHDFSUtils assumes wrong default port for Hadoop 3.0.1+
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-16 12:00:08 -05:00
f73986d110
HBASE-20417 Do not read wal entries when peer is disabled
2018-04-16 22:07:48 +08:00
1cb05a18bc
HBASE-20419 Fix potential NPE in ZKUtil#listChildrenAndWatchForNewChildren callers
...
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-16 16:07:59 +08:00
4fea862963
HBASE-20409 Set hbase.client.meta.operation.timeout in TestClientOperationTimeout
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-04-15 10:12:34 +02:00
f5d970eba1
HBASE-20294 Also cleanup last pushed sequence id in ReplicationBarrierCleaner
2018-04-14 14:56:51 +08:00
ae8a21204d
HBASE-20377 Deal with table in enabling and disabling state when modifying serial replication peer
2018-04-13 20:33:25 +08:00
b0701434e3
HBASE-20243 [Shell] Add shell command to create a new table by cloning the existent table
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-11 19:34:30 -07:00
0f6ff895e9
HBASE-20350 NullPointerException in Scanner during close()
2018-04-11 14:34:23 -07:00
0773209dd3
HBASE-20310 Fixed false inconsistency shown by hbck -metaonly option on HBase 2
...
When -metaonly option is specified then tablesInfo is not populated for all tables from hdfs causing false inconsistency of dangling state in meta.
2018-04-11 12:50:55 -07:00
f3089704d2
HBASE-20219 An error occurs when scanning with reversed=true and loadColumnFamiliesOnDemand=true
2018-04-11 12:27:28 -07:00
b52bf4052f
HBASE-20382 If RSGroups not enabled, rsgroup.jsp prints stack trace
2018-04-11 09:49:50 -07:00
c78908cff3
HBASE-20384 [AMv2] Logging format improvements; use encoded name rather than full region name marking transitions
2018-04-11 08:40:49 -07:00
3eee7d37f4
HBASE-20367 Write a replication barrier for regions when disabling a table
2018-04-11 20:36:59 +08:00
4bcb560e22
HBASE-15291 FileSystem not closed in secure bulkLoad
...
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-11 12:11:41 +05:30
2487b4603c
HBASE-20363 TestNamespaceAuditor.testRegionMerge is flaky
2018-04-09 21:02:42 +08:00
74ab10c353
HBASE-20296 Remove last pushed sequence ids when removing tables from a peer
2018-04-09 15:18:44 +08:00
ead569c951
HBASE-20285 Delete all last pushed sequence ids when removing a peer or removing the serial flag for a peer
2018-04-09 15:18:44 +08:00
8de9596355
HBASE-20127 Add UT for serial replication after failover
2018-04-09 15:18:44 +08:00
6f13b28c19
HBASE-20271 ReplicationSourceWALReader.switched should use the file name instead of the path object directly
2018-04-09 15:18:44 +08:00
9369cf26eb
HBASE-20147 Serial replication will be stuck if we create a table with serial replication but add it to a peer after there are region moves
2018-04-09 15:18:44 +08:00
aec43bb207
HBASE-20242 The open sequence number will grow if we fail to open a region after writing the max sequence id file
2018-04-09 15:18:44 +08:00
16a4dd6b8f
HBASE-20206 WALEntryStream should not switch WAL file silently
2018-04-09 15:18:44 +08:00
644bfe36b2
HBASE-20117 Cleanup the unused replication barriers in meta table
2018-04-09 15:18:44 +08:00
2d0d6a3ba1
HBASE-20167 Optimize the implementation of ReplicationSourceWALReader
2018-04-09 15:18:44 +08:00
cea5199ea1
HBASE-20148 Make serial replication as a option for a peer instead of a table
2018-04-09 15:18:44 +08:00
6b5412759f
HBASE-20125 Add UT for serial replication after region split and merge
2018-04-09 15:18:44 +08:00
8b61a061d3
HBASE-20129 Add UT for serial replication checker
2018-04-09 15:18:44 +08:00
f29bf1d778
HBASE-20115 Reimplement serial replication based on the new replication storage layer
2018-04-09 15:18:44 +08:00
1d11cdb26c
HBASE-20050 Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-09 15:18:44 +08:00
39c1ddc6e3
HBASE-20362 TestMasterShutdown.testMasterShutdownBeforeStartingAnyRegionServer is flaky
2018-04-09 14:55:42 +08:00
dcc1d9e36a
HBASE-20182 Can not locate region after split and merge
2018-04-07 13:37:59 +08:00
e78a8e08f0
HBASE-20095 Redesign single instance pool in CleanerChore - addendum simplifies onConfigurationChange
2018-04-05 10:25:09 -07:00
039bc73571
HBASE-19488 Move to using Apache commons CollectionUtils
...
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-04-04 14:16:33 -07:00
9a34880724
HBASE-16499 slow replication for small HBase clusters
...
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-04 09:59:50 +05:30
ed21f26171
HBASE-20298 Doc change in read/write/total accounting metrics
2018-04-03 17:02:27 -07:00
d4e115bf34
HBASE-20259 Doc configs for in-memory-compaction and add detail to
...
in-memory-compaction logging
Adds logging of CompactingMemStore configuration on construction.
Add logging of detail about Store on creation including memstore type.
Add chapter to refguide on new in-memory compaction feature.
2018-04-02 09:56:32 -07:00
Andrew Purtell
Nihal Jain
eshcar
huzheng
Toshihiro Suzuki
meiyi
Thiruvel Thirumoolan
Guanghao Zhang
zhangduo
Michael Stack
jingyuntian
Balazs Meszaros
Apekshit Sharma
Zach York
Sean Busbey
Guanghao Zhang
tedyu
Chia-Ping Tsai
Umesh Agashe
Sakthi
Rich Fecher
Huaxiang Sun
Wei-Chiu Chuang
lujie
Peter Somogyi
Guangxu Cheng
Ashish Singhi
BELUGA BEHR