b889c8a221
HBASE-20646 TestWALProcedureStoreOnHDFS failing on branch-1
2018-05-30 14:44:30 -07:00
31ae8dc7f3
Revert "TestWALProcedureStoreOnHDFS failing on branch-1"
...
This reverts commit dcfa01448c
2018-05-30 14:44:22 -07:00
dcfa01448c
TestWALProcedureStoreOnHDFS failing on branch-1
2018-05-30 13:45:38 -07:00
997747076d
HBASE-20659 Implement a reopen table regions procedure
2018-05-30 20:03:25 +08:00
266b251dfa
HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint - revert due to pending discussion
2018-05-29 19:57:51 -07:00
fe73fe8def
HBASE-20653 Add missing observer hooks for region server group to MasterObserver - revert due to pending discussion
2018-05-29 19:42:28 -07:00
8d19bbd347
HBASE-20653 Add missing observer hooks for region server group to MasterObserver
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-29 16:37:19 -07:00
06611256ee
HBASE-20597 Serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
2018-05-29 11:29:05 -07:00
807c905f90
Revert "HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint"
...
This reverts commit 9fbce1668b
2018-05-29 11:24:11 -07:00
7ff29d8e00
HBASE-20633 Dropping a table containing a disable violation policy fails to remove the quota upon table delete
...
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-05-29 11:33:56 -04:00
a110e1eff5
HBASE-20478 Update checkstyle to v8.2
...
Cannot go to latest (8.9) yet due to
https://github.com/checkstyle/checkstyle/issues/5279
* move hbaseanti import checks to checkstyle
* implment a few missing equals checks, and ignore one
* fix lots of javadoc errors
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-05-29 10:12:31 -05:00
42be553433
HBASE-20390 ADDENDUM 2: fix TestHRegionWithInMemoryFlush OOME
2018-05-29 16:27:20 +03:00
05f57f4c03
HBASE-20652 Remove internal uses of some deprecated MasterObserver hooks
...
Remove internal uses of these hooks:
preModifyNamespace
postModifyNamespace
preModifyTable
postModifyTable
preModifyTableAction
postCompletedModifyTableAction
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-28 21:10:52 -07:00
81228f72d0
HBASE-20533 Fix the flaky TestAssignmentManagerMetrics
2018-05-29 09:38:47 +08:00
1cd2b56802
HBASE-20390-ADDENDUM: fix TestHRegionWithInMemoryFlush OOME
2018-05-28 16:10:53 +03:00
9bd4b04ca8
HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-27 11:29:26 -07:00
1eabbb4295
HBASE-20390: IMC Default Parameters for 2.0.0
2018-05-26 22:57:28 +03:00
b1089e8310
HBASE-20648 HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-25 07:52:40 -07:00
36f3d9432a
HBASE-20518 Need to serialize the enabled field for UpdatePeerConfigProcedure
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-05-25 14:36:16 +08:00
1fbce10ff4
HBASE-20548 Master fails to startup on large clusters, refreshing block distribution
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-24 15:47:24 -07:00
554d513f50
HBASE-20616 TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-24 15:16:30 -07:00
09dac89908
HBASE-20588 Space quota change after quota violation doesn't seem to take in effect
...
Signed-off-by: Josh Elser <elserj@apache.org>
2018-05-24 12:40:55 -04:00
320a3332e0
HBASE-20589 Don't need to assign meta to a new RS when standby master become active
2018-05-24 11:26:48 +08:00
ee540c9f9e
HBASE-20624 Race in ReplicationSource which causes walEntryFilter being null when creating new shipper
2018-05-24 10:48:29 +08:00
9fbce1668b
HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint
2018-05-23 16:46:22 -07:00
079f168c5c
HBASE-20620 HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; part 2
...
Adds new stripped-down, faster ByteBufferKeyValue comparator
(BBKV is the base Cell-type in hbase2). Creates an instance
of new Comparator each time we create new memstore rather
than use the universal CellComparator.
Remove unused and unneeded Interfaces from Cell base type.
2018-05-23 13:20:47 -07:00
5721150c6d
HBASE-20612 TestReplicationKillSlaveRSWithSeparateOldWALs sometimes fail because it uses an expired cluster conn
2018-05-23 12:07:01 +08:00
6c1097e92f
HBASE-20609 SnapshotHFileCleaner#init should check that params is not null
2018-05-21 18:36:38 -07:00
c9f8c3436f
HBASE-20579 Improve snapshot manifest copy in ExportSnapshot
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-18 06:42:12 -07:00
0836b0719a
HBASE-20583 SplitLogWorker should handle FileNotFoundException when split a wal
2018-05-18 14:29:41 +08:00
6148b4785d
HBASE-20571 JMXJsonServlet generates invalid JSON if it has NaN in metrics
...
- CacheStats won't generate NaN metrics.
- JSONBean class will serialize special floating point values as
"NaN", "Infinity" or "-Infinity"
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-16 12:20:39 -07:00
8c9825a030
HBASE-20567 Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace.
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-05-16 14:03:18 -05:00
438af9bf74
HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; ADDENDUM
...
Add method the CellComparator Interface. Add implementation to
meta comparator so we don't fall back to the default comparator.
Includes a nothing change to hbase-server/pom.xml just to provoke
build.
2018-05-16 09:43:16 -07:00
060b8aca86
HBASE-20520 Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
2018-05-16 09:19:24 -07:00
ab53329cb3
HBASE-20585 Need to clear peer map when clearing MasterProcedureScheduler
2018-05-16 08:46:29 +08:00
26babcf013
HBASE-20457 Return immediately for a scan rpc call when we want to switch from pread to stream
2018-05-15 20:56:20 +08:00
d2daada970
HBASE-20447 Only fail cacheBlock if block collisions aren't related to next block metadata
...
When we pread, we don't force the read to read all of the next block header.
However, when we get into a race condition where two opener threads try to
cache the same block and one thread read all of the next block header and
the other one didn't, it will fail the open process. This is especially important
in a splitting case where it will potentially fail the split process.
Instead, in the caches, we should only fail if the required blocks are different.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-14 17:16:54 -07:00
eabe672ebd
HBASE-20560 Revisit the TestReplicationDroppedTables ut
2018-05-14 19:12:43 +08:00
021f66d11d
HBASE-20411 Ameliorate MutableSegment synchronize
...
Change the MemStore size accounting so we don't synchronize across three
volatiles applying deltas. Instead:
+ Make MemStoreSize, a datastructure of our memstore size longs, immutable.
+ Undo MemStoreSizing being an instance of MemStoreSize; instead it has-a.
+ Make two MemStoreSizing implementations; one thread-safe, the other not.
+ Let all memory sizing longs run independent, untied by
synchronize (Huaxiang and Anoop suggestion) using atomiclongs.
+ Review all use of MemStoreSizing. Many are single-threaded and do
not need to be synchronized; use the non-thread safe counter.
TODO: Use this technique accounting at the global level too.
2018-05-12 02:17:50 +01:00
8ba2a7eeb9
HBASE-20544 Make HBTU default to random ports.
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Josh Elser <elserj@apache.org>
2018-05-09 23:35:20 -07:00
a67909d3d6
HBASE-20545 Improve performance of BaseLoadBalancer.retainAssignment
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-09 19:48:27 -07:00
32dd633b98
HBASE-20554 "WALs outstanding" message from CleanerChore is noisy
2018-05-09 19:08:35 -07:00
9c046c091c
HBASE-20204 Add locking to RefreshFileConnections in BucketCache
...
This is a follow-up to HBASE-20141 where Anoop suggested adding locking
for refreshing channels.
2018-05-09 14:01:51 -07:00
4f2dfd3877
HBASE-20539 Disable IMC; part 2
...
Set default for in-memory compaction to NONE. PE was using
this value creating its table. Add test to ensure expected
default.
2018-05-09 06:30:28 -07:00
f8281aa208
HBASE-20485 Copy constructor of Scan doesn't copy the readType and replicaId
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-05-09 16:39:57 +08:00
3a2a76f6f8
HBASE-20536 Make TestRegionServerAccounting stable and it should not use absolute number
2018-05-08 08:20:48 +08:00
bb1a935636
HBASE-20538 TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
...
Disable test TestSaslFanOutOneBlockAsyncDFSOutput for the moment.
Depends on HDFS-13494
2018-05-07 09:54:32 -07:00
971f5350e8
HBASE-20531 RS may throw NPE when close meta regions in shutdown procedure.
2018-05-06 11:18:23 +08:00
87f5b5f341
HBASE-20378 Provide a hbck option to cleanup replication barrier for a table
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-05-04 15:27:33 +08:00
6225b4a492
HBASE-20481 Replicate entries from same region serially in ReplicationEndpoint for serial replication
2018-05-04 15:22:02 +08:00
9b9f851470
HBASE-20524 Need to clear metrics when ReplicationSourceManager refresh replication sources
2018-05-04 14:31:41 +08:00
09ca619389
HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer
2018-05-03 12:30:13 -07:00
4cb444e77b
HBASE-20169 NPE when calling HBTU.shutdownMiniCluster (TestAssignmentManagerMetrics is flakey); AMENDMENT
2018-05-02 16:14:58 -07:00
c4b4023b60
HBASE-20514 On Master restart if table is stuck in DISABLING state, CLOSED regions should not be considered stuck in-transition
...
On startup CLOSED regions of DISABLED and DISABLING tables are treated the same way as not in-transition.
2018-05-02 12:18:24 -07:00
2e9b96e4f1
HBASE-20414 TestLockProcedure#testMultipleLocks may fail on slow machine
2018-05-02 06:39:51 -07:00
5bdb52af2f
HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer
2018-05-02 10:26:55 +08:00
5a071dbe2b
HBASE-20492 UnassignProcedure is stuck in retry loop on region stuck in OPENING state
...
Add backoff when stuck in RegionTransitionProcedure, the subclass of
AssignProcedure and UnassignProcedure. Can happen when we go to
transition but the current Region state is not what we expect.
M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
Add doc on being able to suspend and wait on a timeout.
M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
Add 'attempt' counter so we can do backoff when we get stuck.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
Add persistence of new 'attempt' counter
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
Doc data members that are persisted by subclasses given this is 'odd'.
Add a counter for 'attempts' used when 'stuck' to implement backoff.
Add suspend with timeout when 'stuck'. Add callback when timeout is
exhausted which does wakeup of this procedure.
A hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestUnexpectedStateException.java
Test of backoff.
2018-04-30 20:40:22 -07:00
6d080762ef
HBASE-20499 Replication/Priority executors can use specific max queue length as default value instead of general maxQueueLength
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-30 07:42:32 -07:00
a136303833
HBASE-20497 The getRecoveredQueueStartPos always return 0 in RecoveredReplicationSourceShipper
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-28 20:50:30 +08:00
4be96dd8a6
HBASE-20476 Open sequence number could go backwards in AssignProcedure
2018-04-28 06:49:50 +08:00
650340abb5
HBASE-20487 Sorting table regions by region name does not work on web UI
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-27 11:13:00 -07:00
e9a278adc6
HBASE-20475 Fix the flaky TestReplicationDroppedTables unit test - addendum
2018-04-27 21:38:15 +08:00
39cf42be9a
HBASE-20476 Add more logs for debugging
2018-04-27 16:02:05 +08:00
96ed407c69
HBASE-20476 Fix the flaky TestReplicationSmallTests unit test
2018-04-27 10:25:52 +08:00
f761689136
HBASE-20040 Fixed: Master UI should include "Cluster Key" needed to use the cluster as a replication sink
...
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-26 06:43:55 -05:00
12c45cb2e8
HBASE-20475 Fix the flaky TestReplicationDroppedTables unit test.
2018-04-25 11:21:59 +08:00
a8be3bb814
HBASE-20466 Consistently use override mechanism for exempt classes in CoprocessorClassloader
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-23 07:47:30 -07:00
1633d9d61d
HBASE-20470 [2.0.0RC1] has broken unit tests...
...
Fix test that depended upon flush being slow and one family only.
Fix MemStoreSize compare to allow passing alternate implementation
(needed when IMC was no longer default everywhere).
2018-04-22 15:33:25 -07:00
e22f9e2d97
HBASE-19924 hbase rpc throttling does not work for multi() with request count rater.
2018-04-20 16:54:03 -07:00
758f4296a4
HBASE-20450 Provide metrics for number of total active, priority and replication rpc handlers
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-20 16:24:32 -07:00
4e183748c7
HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-20 10:51:11 -05:00
914de11416
HBASE-20442 clean up incorrect use of commons-collections 3
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-20 07:30:34 -05:00
09749f1574
HBASE-20440 Clean up incorrect use of commons-lang 2.y
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-20 07:29:23 -05:00
b1fc00e16f
HBASE-20439 Clean up incorrect use of commons-logging in hbase-server
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-20 07:27:18 -05:00
556b223744
HBASE-18059 remove scanner order related code
2018-04-18 19:38:08 -07:00
6ca8261fc0
HBASE-18792 Disabled destructive (mostly write) operations in hbck for HBase 2.0+
...
HBase server versioned is checked after connecting to the server and then following operations are not allowed:
-fix, -fixAssignments, -fixMeta, -fixHdfsHoles, -fixHdfsOrphans, -fixTableOrphans, -fixHdfsOverlaps, -maxMerge
-sidelineBigOverlaps, -maxOverlapsToSideline, -fixSplitParents, -removeParents, -fixEmptyMetaCells
-repair, -repairHoles
2018-04-18 15:22:39 -07:00
6ce1136eba
HBASE-20452 Master UI: Table merge button should validate required fields before submit
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-18 14:55:12 -07:00
80cbc0d1fe
HBASE-20449 The minimun number of region should be configurable in Normalizer
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-18 08:45:52 -07:00
824ee525df
HBASE-20404 Fixes to CleanChore correctness and operability.
...
* Make CleanerChore less chatty: move WARN message to DEBUG when we expect non-empty dirs
* Make CleanerChore less chatty: move IOE we'll retry to INFO
* CleanerChore should treat IOE for FileStatus as a failure
* Add tests asserting assumptions in above
Signed-off-by: Reid Chan <reidddchan@outlook.com>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-17 11:32:44 -05:00
b926d359b2
HBASE-20398 Redirect doesn't work on web UI
2018-04-17 09:18:52 -07:00
32764bfb2c
HBASE-20399 Fix merge layout
2018-04-17 09:17:25 -07:00
42d5447cfb
HBASE-20128 Add new UTs which extends the old replication UTs but set replication scope to SERIAL
2018-04-17 10:56:21 +08:00
1fe3d6a0e9
HBASE-19994 Create a new class for RPC throttling exception, make it retryable
2018-04-16 17:27:14 -07:00
44ebd28093
HBASE-19963 TestFSHDFSUtils assumes wrong default port for Hadoop 3.0.1+
...
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-16 12:00:17 -05:00
773aff90fd
HBASE-20417 Do not read wal entries when peer is disabled
2018-04-16 22:07:33 +08:00
1339ff9666
HBASE-20419 Fix potential NPE in ZKUtil#listChildrenAndWatchForNewChildren callers
...
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-16 16:06:57 +08:00
125767b44e
HBASE-20409 Set hbase.client.meta.operation.timeout in TestClientOperationTimeout
...
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-04-15 10:12:00 +02:00
edf5049502
HBASE-20294 Also cleanup last pushed sequence id in ReplicationBarrierCleaner
2018-04-14 14:56:18 +08:00
5a633adffe
HBASE-20377 Deal with table in enabling and disabling state when modifying serial replication peer
2018-04-13 20:33:29 +08:00
5a69465ea0
HBASE-20243 [Shell] Add shell command to create a new table by cloning the existent table
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-11 19:33:56 -07:00
281c29ff60
HBASE-20350 NullPointerException in Scanner during close()
2018-04-11 14:34:57 -07:00
199b392ec8
HBASE-20310 Fixed false inconsistency shown by hbck -metaonly option on HBase 2
...
When -metaonly option is specified then tablesInfo is not populated for all tables from hdfs causing false inconsistency of dangling state in meta.
2018-04-11 12:51:12 -07:00
072c503ec7
HBASE-20219 An error occurs when scanning with reversed=true and loadColumnFamiliesOnDemand=true
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-11 09:59:45 -07:00
35b4adf3ed
HBASE-20382 If RSGroups not enabled, rsgroup.jsp prints stack trace
2018-04-11 09:50:07 -07:00
ba02a8664b
HBASE-20384 [AMv2] Logging format improvements; use encoded name rather than full region name marking transitions
2018-04-11 08:35:54 -07:00
37e5b0b1b7
HBASE-20367 Write a replication barrier for regions when disabling a table
2018-04-11 20:36:51 +08:00
828a1c76c7
HBASE-15291 FileSystem not closed in secure bulkLoad
...
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-11 12:01:28 +05:30
bc35de597d
HBASE-20363 Addendum wait until region is splittable
2018-04-09 09:07:06 +08:00
f66343050f
HBASE-20363 TestNamespaceAuditor.testRegionMerge is flaky
2018-04-08 16:48:25 +08:00
852025dd00
HBASE-20362 TestMasterShutdown.testMasterShutdownBeforeStartingAnyRegionServer is flaky
2018-04-08 16:09:50 +08:00
Andrew Purtell
zhangduo
tedyu
Nihal Jain
Mike Drob
eshcar
Apekshit Sharma
huzheng
Toshihiro Suzuki
meiyi
Thiruvel Thirumoolan
Guanghao Zhang
Michael Stack
jingyuntian
Balazs Meszaros
Zach York
Sean Busbey
Guanghao Zhang
Chia-Ping Tsai
Umesh Agashe
Sakthi
Rich Fecher
Huaxiang Sun
wangyu
Wei-Chiu Chuang
lujie
Peter Somogyi
Guangxu Cheng
Ashish Singhi