Commit Graph

6924 Commits

Author SHA1 Message Date
Peter Somogyi 0968668283 HBASE-20592 Create a tool to verify tables do not have prefix tree encoding
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-06-01 19:17:49 +02:00
Andrew Purtell da3ecf1f13 Revert "HBASE-18116 fix replication source in-memory calculation by excluding bulk load file"
This reverts commit 6f3f34227e.
2018-05-31 15:28:28 -07:00
Xu Cang 6f3f34227e HBASE-18116 fix replication source in-memory calculation by excluding bulk load file
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-31 14:22:12 -07:00
Sean Busbey d909ec55aa HBASE-20444 Addendum keep folks from looking at raw version component array.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-31 13:29:52 -05:00
Nihal Jain 40a73a5ca7 HBASE-20653 Add missing observer hooks for region server group to MasterObserver
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-30 21:29:07 -07:00
Andrew Purtell b889c8a221 HBASE-20646 TestWALProcedureStoreOnHDFS failing on branch-1 2018-05-30 14:44:30 -07:00
Andrew Purtell 31ae8dc7f3 Revert "TestWALProcedureStoreOnHDFS failing on branch-1"
This reverts commit dcfa01448c.
2018-05-30 14:44:22 -07:00
Andrew Purtell dcfa01448c TestWALProcedureStoreOnHDFS failing on branch-1 2018-05-30 13:45:38 -07:00
zhangduo 997747076d HBASE-20659 Implement a reopen table regions procedure 2018-05-30 20:03:25 +08:00
tedyu 266b251dfa HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint - revert due to pending discussion 2018-05-29 19:57:51 -07:00
tedyu fe73fe8def HBASE-20653 Add missing observer hooks for region server group to MasterObserver - revert due to pending discussion 2018-05-29 19:42:28 -07:00
Nihal Jain 8d19bbd347 HBASE-20653 Add missing observer hooks for region server group to MasterObserver
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-29 16:37:19 -07:00
Andrew Purtell 06611256ee HBASE-20597 Serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint 2018-05-29 11:29:05 -07:00
Andrew Purtell 807c905f90 Revert "HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint"
This reverts commit 9fbce1668b.
2018-05-29 11:24:11 -07:00
Nihal Jain 7ff29d8e00 HBASE-20633 Dropping a table containing a disable violation policy fails to remove the quota upon table delete
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-05-29 11:33:56 -04:00
Mike Drob a110e1eff5 HBASE-20478 Update checkstyle to v8.2
Cannot go to latest (8.9) yet due to
  https://github.com/checkstyle/checkstyle/issues/5279

* move hbaseanti import checks to checkstyle
* implment a few missing equals checks, and ignore one
* fix lots of javadoc errors

Signed-off-by: Sean Busbey <busbey@apache.org>
2018-05-29 10:12:31 -05:00
eshcar 42be553433 HBASE-20390 ADDENDUM 2: fix TestHRegionWithInMemoryFlush OOME 2018-05-29 16:27:20 +03:00
Apekshit Sharma 05f57f4c03 HBASE-20652 Remove internal uses of some deprecated MasterObserver hooks
Remove internal uses of these hooks:
preModifyNamespace
postModifyNamespace
preModifyTable
postModifyTable
preModifyTableAction
postCompletedModifyTableAction

Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-28 21:10:52 -07:00
huzheng 81228f72d0 HBASE-20533 Fix the flaky TestAssignmentManagerMetrics 2018-05-29 09:38:47 +08:00
eshcar 1cd2b56802 HBASE-20390-ADDENDUM: fix TestHRegionWithInMemoryFlush OOME 2018-05-28 16:10:53 +03:00
Nihal Jain 9bd4b04ca8 HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-27 11:29:26 -07:00
eshcar 1eabbb4295 HBASE-20390: IMC Default Parameters for 2.0.0 2018-05-26 22:57:28 +03:00
Toshihiro Suzuki b1089e8310 HBASE-20648 HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-25 07:52:40 -07:00
meiyi 36f3d9432a HBASE-20518 Need to serialize the enabled field for UpdatePeerConfigProcedure
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-05-25 14:36:16 +08:00
Thiruvel Thirumoolan 1fbce10ff4 HBASE-20548 Master fails to startup on large clusters, refreshing block distribution
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-24 15:47:24 -07:00
Toshihiro Suzuki 554d513f50 HBASE-20616 TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-24 15:16:30 -07:00
Nihal Jain 09dac89908 HBASE-20588 Space quota change after quota violation doesn't seem to take in effect
Signed-off-by: Josh Elser <elserj@apache.org>
2018-05-24 12:40:55 -04:00
Guanghao Zhang 320a3332e0 HBASE-20589 Don't need to assign meta to a new RS when standby master become active 2018-05-24 11:26:48 +08:00
zhangduo ee540c9f9e HBASE-20624 Race in ReplicationSource which causes walEntryFilter being null when creating new shipper 2018-05-24 10:48:29 +08:00
Andrew Purtell 9fbce1668b HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint 2018-05-23 16:46:22 -07:00
Michael Stack 079f168c5c HBASE-20620 HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; part 2
Adds new stripped-down, faster ByteBufferKeyValue comparator
(BBKV is the base Cell-type in hbase2). Creates an instance
of new Comparator each time we create new memstore rather
than use the universal CellComparator.

Remove unused and unneeded Interfaces from Cell base type.
2018-05-23 13:20:47 -07:00
huzheng 5721150c6d HBASE-20612 TestReplicationKillSlaveRSWithSeparateOldWALs sometimes fail because it uses an expired cluster conn 2018-05-23 12:07:01 +08:00
tedyu 6c1097e92f HBASE-20609 SnapshotHFileCleaner#init should check that params is not null 2018-05-21 18:36:38 -07:00
jingyuntian c9f8c3436f HBASE-20579 Improve snapshot manifest copy in ExportSnapshot
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-18 06:42:12 -07:00
Guanghao Zhang 0836b0719a HBASE-20583 SplitLogWorker should handle FileNotFoundException when split a wal 2018-05-18 14:29:41 +08:00
Balazs Meszaros 6148b4785d HBASE-20571 JMXJsonServlet generates invalid JSON if it has NaN in metrics
- CacheStats won't generate NaN metrics.
- JSONBean class will serialize special floating point values as
  "NaN", "Infinity" or "-Infinity"

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-16 12:20:39 -07:00
Apekshit Sharma 8c9825a030 HBASE-20567 Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace.
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-05-16 14:03:18 -05:00
Michael Stack 438af9bf74
HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; ADDENDUM
Add method the CellComparator Interface. Add implementation to
meta comparator so we don't fall back to the default comparator.

Includes a nothing change to hbase-server/pom.xml just to provoke
build.
2018-05-16 09:43:16 -07:00
Michael Stack 060b8aca86
HBASE-20520 Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize 2018-05-16 09:19:24 -07:00
zhangduo ab53329cb3 HBASE-20585 Need to clear peer map when clearing MasterProcedureScheduler 2018-05-16 08:46:29 +08:00
zhangduo 26babcf013 HBASE-20457 Return immediately for a scan rpc call when we want to switch from pread to stream 2018-05-15 20:56:20 +08:00
Zach York d2daada970 HBASE-20447 Only fail cacheBlock if block collisions aren't related to next block metadata
When we pread, we don't force the read to read all of the next block header.
However, when we get into a race condition where two opener threads try to
cache the same block and one thread read all of the next block header and
the other one didn't, it will fail the open process. This is especially important
in a splitting case where it will potentially fail the split process.
Instead, in the caches, we should only fail if the required blocks are different.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-14 17:16:54 -07:00
huzheng eabe672ebd HBASE-20560 Revisit the TestReplicationDroppedTables ut 2018-05-14 19:12:43 +08:00
Michael Stack 021f66d11d
HBASE-20411 Ameliorate MutableSegment synchronize
Change the MemStore size accounting so we don't synchronize across three
volatiles applying deltas. Instead:

 + Make MemStoreSize, a datastructure of our memstore size longs, immutable.
 + Undo MemStoreSizing being an instance of MemStoreSize; instead it has-a.
 + Make two MemStoreSizing implementations; one thread-safe, the other not.
 + Let all memory sizing longs run independent, untied by
   synchronize (Huaxiang and Anoop suggestion) using atomiclongs.
 + Review all use of MemStoreSizing. Many are single-threaded and do
   not need to be synchronized; use the non-thread safe counter.

TODO: Use this technique accounting at the global level too.
2018-05-12 02:17:50 +01:00
Sean Busbey 8ba2a7eeb9 HBASE-20544 Make HBTU default to random ports.
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Josh Elser <elserj@apache.org>
2018-05-09 23:35:20 -07:00
Thiruvel Thirumoolan a67909d3d6 HBASE-20545 Improve performance of BaseLoadBalancer.retainAssignment
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-09 19:48:27 -07:00
Andrew Purtell 32dd633b98 HBASE-20554 "WALs outstanding" message from CleanerChore is noisy 2018-05-09 19:08:35 -07:00
Zach York 9c046c091c HBASE-20204 Add locking to RefreshFileConnections in BucketCache
This is a follow-up to HBASE-20141 where Anoop suggested adding locking
for refreshing channels.
2018-05-09 14:01:51 -07:00
Michael Stack 4f2dfd3877 HBASE-20539 Disable IMC; part 2
Set default for in-memory compaction to NONE. PE was using
this value creating its table. Add test to ensure expected
default.
2018-05-09 06:30:28 -07:00
Nihal Jain f8281aa208 HBASE-20485 Copy constructor of Scan doesn't copy the readType and replicaId
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-05-09 16:39:57 +08:00
Guanghao Zhang 3a2a76f6f8 HBASE-20536 Make TestRegionServerAccounting stable and it should not use absolute number 2018-05-08 08:20:48 +08:00
Michael Stack bb1a935636 HBASE-20538 TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
Disable test TestSaslFanOutOneBlockAsyncDFSOutput for the moment.
Depends on HDFS-13494
2018-05-07 09:54:32 -07:00
huzheng 971f5350e8 HBASE-20531 RS may throw NPE when close meta regions in shutdown procedure. 2018-05-06 11:18:23 +08:00
jingyuntian 87f5b5f341 HBASE-20378 Provide a hbck option to cleanup replication barrier for a table
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-05-04 15:27:33 +08:00
huzheng 6225b4a492 HBASE-20481 Replicate entries from same region serially in ReplicationEndpoint for serial replication 2018-05-04 15:22:02 +08:00
Guanghao Zhang 9b9f851470 HBASE-20524 Need to clear metrics when ReplicationSourceManager refresh replication sources 2018-05-04 14:31:41 +08:00
Michael Stack 09ca619389
HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer 2018-05-03 12:30:13 -07:00
Chia-Ping Tsai 4cb444e77b
HBASE-20169 NPE when calling HBTU.shutdownMiniCluster (TestAssignmentManagerMetrics is flakey); AMENDMENT 2018-05-02 16:14:58 -07:00
Umesh Agashe c4b4023b60 HBASE-20514 On Master restart if table is stuck in DISABLING state, CLOSED regions should not be considered stuck in-transition
On startup CLOSED regions of DISABLED and DISABLING tables are treated the same way as not in-transition.
2018-05-02 12:18:24 -07:00
tedyu 2e9b96e4f1 HBASE-20414 TestLockProcedure#testMultipleLocks may fail on slow machine 2018-05-02 06:39:51 -07:00
zhangduo 5bdb52af2f HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer 2018-05-02 10:26:55 +08:00
Michael Stack 5a071dbe2b HBASE-20492 UnassignProcedure is stuck in retry loop on region stuck in OPENING state
Add backoff when stuck in RegionTransitionProcedure, the subclass of
AssignProcedure and UnassignProcedure. Can happen when we go to
transition but the current Region state is not what we expect.

M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
 Add doc on being able to suspend and wait on a timeout.

M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
 Add 'attempt' counter so we can do backoff when we get stuck.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
 Add persistence of new 'attempt' counter

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
 Doc data members that are persisted by subclasses given this is 'odd'.
 Add a counter for 'attempts' used when 'stuck' to implement backoff.
 Add suspend with timeout when 'stuck'. Add callback when timeout is
 exhausted which does wakeup of this procedure.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestUnexpectedStateException.java
 Test of backoff.
2018-04-30 20:40:22 -07:00
Nihal Jain 6d080762ef HBASE-20499 Replication/Priority executors can use specific max queue length as default value instead of general maxQueueLength
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-30 07:42:32 -07:00
huzheng a136303833 HBASE-20497 The getRecoveredQueueStartPos always return 0 in RecoveredReplicationSourceShipper
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-28 20:50:30 +08:00
zhangduo 4be96dd8a6 HBASE-20476 Open sequence number could go backwards in AssignProcedure 2018-04-28 06:49:50 +08:00
Balazs Meszaros 650340abb5 HBASE-20487 Sorting table regions by region name does not work on web UI
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-27 11:13:00 -07:00
huzheng e9a278adc6 HBASE-20475 Fix the flaky TestReplicationDroppedTables unit test - addendum 2018-04-27 21:38:15 +08:00
zhangduo 39cf42be9a HBASE-20476 Add more logs for debugging 2018-04-27 16:02:05 +08:00
zhangduo 96ed407c69 HBASE-20476 Fix the flaky TestReplicationSmallTests unit test 2018-04-27 10:25:52 +08:00
Sakthi f761689136 HBASE-20040 Fixed: Master UI should include "Cluster Key" needed to use the cluster as a replication sink
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-26 06:43:55 -05:00
huzheng 12c45cb2e8 HBASE-20475 Fix the flaky TestReplicationDroppedTables unit test. 2018-04-25 11:21:59 +08:00
Rich Fecher a8be3bb814 HBASE-20466 Consistently use override mechanism for exempt classes in CoprocessorClassloader
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-23 07:47:30 -07:00
Michael Stack 1633d9d61d HBASE-20470 [2.0.0RC1] has broken unit tests...
Fix test that depended upon flush being slow and one family only.
Fix MemStoreSize compare to allow passing alternate implementation
(needed when IMC was no longer default everywhere).
2018-04-22 15:33:25 -07:00
Huaxiang Sun e22f9e2d97 HBASE-19924 hbase rpc throttling does not work for multi() with request count rater. 2018-04-20 16:54:03 -07:00
Nihal Jain 758f4296a4 HBASE-20450 Provide metrics for number of total active, priority and replication rpc handlers
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-20 16:24:32 -07:00
Toshihiro Suzuki 4e183748c7 HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-20 10:51:11 -05:00
Sean Busbey 914de11416 HBASE-20442 clean up incorrect use of commons-collections 3
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-20 07:30:34 -05:00
Sean Busbey 09749f1574 HBASE-20440 Clean up incorrect use of commons-lang 2.y
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-20 07:29:23 -05:00
Sean Busbey b1fc00e16f HBASE-20439 Clean up incorrect use of commons-logging in hbase-server
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-20 07:27:18 -05:00
jingyuntian 556b223744 HBASE-18059 remove scanner order related code 2018-04-18 19:38:08 -07:00
Umesh Agashe 6ca8261fc0 HBASE-18792 Disabled destructive (mostly write) operations in hbck for HBase 2.0+
HBase server versioned is checked after connecting to the server and then following operations are not allowed:
-fix, -fixAssignments, -fixMeta, -fixHdfsHoles, -fixHdfsOrphans, -fixTableOrphans, -fixHdfsOverlaps, -maxMerge
-sidelineBigOverlaps, -maxOverlapsToSideline, -fixSplitParents, -removeParents, -fixEmptyMetaCells
-repair, -repairHoles
2018-04-18 15:22:39 -07:00
Nihal Jain 6ce1136eba HBASE-20452 Master UI: Table merge button should validate required fields before submit
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-18 14:55:12 -07:00
wangyu 80cbc0d1fe HBASE-20449 The minimun number of region should be configurable in Normalizer
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-18 08:45:52 -07:00
Sean Busbey 824ee525df HBASE-20404 Fixes to CleanChore correctness and operability.
* Make CleanerChore less chatty: move WARN message to DEBUG when we expect non-empty dirs
* Make CleanerChore less chatty: move IOE we'll retry to INFO
* CleanerChore should treat IOE for FileStatus as a failure
* Add tests asserting assumptions in above

Signed-off-by: Reid Chan <reidddchan@outlook.com>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-17 11:32:44 -05:00
Balazs Meszaros b926d359b2 HBASE-20398 Redirect doesn't work on web UI 2018-04-17 09:18:52 -07:00
Balazs Meszaros 32764bfb2c HBASE-20399 Fix merge layout 2018-04-17 09:17:25 -07:00
huzheng 42d5447cfb HBASE-20128 Add new UTs which extends the old replication UTs but set replication scope to SERIAL 2018-04-17 10:56:21 +08:00
Huaxiang Sun 1fe3d6a0e9 HBASE-19994 Create a new class for RPC throttling exception, make it retryable 2018-04-16 17:27:14 -07:00
Wei-Chiu Chuang 44ebd28093 HBASE-19963 TestFSHDFSUtils assumes wrong default port for Hadoop 3.0.1+
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-16 12:00:17 -05:00
zhangduo 773aff90fd HBASE-20417 Do not read wal entries when peer is disabled 2018-04-16 22:07:33 +08:00
lujie 1339ff9666 HBASE-20419 Fix potential NPE in ZKUtil#listChildrenAndWatchForNewChildren callers
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-16 16:06:57 +08:00
Peter Somogyi 125767b44e HBASE-20409 Set hbase.client.meta.operation.timeout in TestClientOperationTimeout
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-04-15 10:12:00 +02:00
zhangduo edf5049502 HBASE-20294 Also cleanup last pushed sequence id in ReplicationBarrierCleaner 2018-04-14 14:56:18 +08:00
zhangduo 5a633adffe HBASE-20377 Deal with table in enabling and disabling state when modifying serial replication peer 2018-04-13 20:33:29 +08:00
Guangxu Cheng 5a69465ea0 HBASE-20243 [Shell] Add shell command to create a new table by cloning the existent table
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-11 19:33:56 -07:00
Apekshit Sharma 281c29ff60 HBASE-20350 NullPointerException in Scanner during close() 2018-04-11 14:34:57 -07:00
Umesh Agashe 199b392ec8 HBASE-20310 Fixed false inconsistency shown by hbck -metaonly option on HBase 2
When -metaonly option is specified then tablesInfo is not populated for all tables from hdfs causing false inconsistency of dangling state in meta.
2018-04-11 12:51:12 -07:00
Toshihiro Suzuki 072c503ec7 HBASE-20219 An error occurs when scanning with reversed=true and loadColumnFamiliesOnDemand=true
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-11 09:59:45 -07:00
Balazs Meszaros 35b4adf3ed HBASE-20382 If RSGroups not enabled, rsgroup.jsp prints stack trace 2018-04-11 09:50:07 -07:00
Michael Stack ba02a8664b HBASE-20384 [AMv2] Logging format improvements; use encoded name rather than full region name marking transitions 2018-04-11 08:35:54 -07:00
zhangduo 37e5b0b1b7 HBASE-20367 Write a replication barrier for regions when disabling a table 2018-04-11 20:36:51 +08:00
Ashish Singhi 828a1c76c7 HBASE-15291 FileSystem not closed in secure bulkLoad
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-11 12:01:28 +05:30
zhangduo bc35de597d HBASE-20363 Addendum wait until region is splittable 2018-04-09 09:07:06 +08:00
zhangduo f66343050f HBASE-20363 TestNamespaceAuditor.testRegionMerge is flaky 2018-04-08 16:48:25 +08:00
zhangduo 852025dd00 HBASE-20362 TestMasterShutdown.testMasterShutdownBeforeStartingAnyRegionServer is flaky 2018-04-08 16:09:50 +08:00
zhangduo adc0e85e85 HBASE-20182 Can not locate region after split and merge 2018-04-07 13:37:53 +08:00
tedyu 8f6849ff02 HBASE-20095 Redesign single instance pool in CleanerChore - addendum simplifies onConfigurationChange 2018-04-05 10:24:37 -07:00
BELUGA BEHR d866e7c658 HBASE-19488 Move to using Apache commons CollectionUtils
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-04-04 14:12:19 -07:00
Ashish Singhi 5937202fd5 HBASE-16499 slow replication for small HBase clusters
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-04 09:54:41 +05:30
Michael Stack b1b0db3195 HBASE-20298 Doc change in read/write/total accounting metrics 2018-04-03 17:00:43 -07:00
Michael Stack c88ca54c16 HBASE-20259 Doc configs for in-memory-compaction and add detail to
in-memory-compaction logging

Adds logging of CompactingMemStore configuration on construction.

Add logging of detail about Store on creation including memstore type.

Add chapter to refguide on new in-memory compaction feature.
2018-04-02 09:53:12 -07:00
zhangduo 1e56938757 HBASE-20296 Remove last pushed sequence ids when removing tables from a peer 2018-04-01 21:14:31 +08:00
Jan Hentschel 97b3a04019 HBASE-17631 Changed Canary interval to 60 seconds 2018-04-01 13:03:52 +02:00
Mike Drob d57001ee2d HBASE-20282 Clean up tooling docs/help 2018-03-30 08:59:47 -07:00
tedyu e60f34fdb5 HBASE-20159 Support using separate ZK quorums for client - addendum fixes flaky TestSeparateClientZKCluster 2018-03-30 07:19:47 -07:00
Vasudevan cbd8b15e6b HBASE-20197 Review of ByteBufferWriterOutputStream.java (BELUGA BEHR)
Signed-off-by: Vasudevan <ramkrishna.s.vasudevan@gmail.com>
2018-03-30 12:13:41 +05:30
Mike Drob 621e2b5f5e HBASE-20273 Additional removed/changed configs 2018-03-29 11:47:14 -07:00
Yu Li 061a31fad1 HBASE-20159 Support using separate ZK quorums for client 2018-03-29 02:37:26 +08:00
Josh Elser 69f5d707b6 HBASE-20199 Add a unit test to verify flush and snapshot permission requirements aren't excessive
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-27 20:17:08 -04:00
Zach York 09ed7c7a10 HBASE-20280 Fix possibility of deadlocking in refreshFileConnections
When prefetch on open is specified, there is a deadlocking case
where if the prefetch is cancelled, the PrefetchExecutor interrupts
the threads if necessary, when that happens in FileIOEngine, it
causes an ClosedByInterruptException which is a subclass of
ClosedChannelException. If we retry all ClosedChannelExceptions,
this will lock as this access is expected to be interrupted.
This change removes calling refreshFileConnections for
ClosedByInterruptExceptions.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-03-27 16:53:01 -07:00
Rajeshbabu Chintaguntla d87139989f HBASE-20111 A region's splittable state now includes the configuration splitPolicy
The Master asks a RegionServer whether a Region can be split or not, primarily to
verify that the region is not closing, opening, etc. This change has the RegionServer
also consult the configured RegionSplitPolicy.

Signed-off-by: Josh Elser <elserj@apache.org>
2018-03-27 18:42:49 -04:00
zhangduo 056c3395d9 HBASE-20285 Delete all last pushed sequence ids when removing a peer or removing the serial flag for a peer 2018-03-27 12:20:51 +08:00
Josh Elser 15c398f7d2 HBASE-20223 Update to hbase-thirdparty 2.1.0
Remove commons-cli and commons-collections4 use. Account
for the newer internal protobuf version of 3.5.1.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-03-26 22:05:19 -04:00
Toshihiro Suzuki ff5250ca2b HBASE-20229 ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on 2018-03-26 15:09:20 -07:00
Yuki Tawara 4cedd9914b HBASE-20289 Fix comparator for NormalizationPlan
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-26 14:30:25 -07:00
Reid Chan 83fa0ad9ed HBASE-20095 Redesign single instance pool in CleanerChore 2018-03-26 12:48:31 -05:00
zhangduo e9701a0595 HBASE-20127 Add UT for serial replication after failover 2018-03-26 20:16:02 +08:00
anoopsamjohn 3f7222df36 HBASE-17819 Reduce the heap overhead for BucketCache. 2018-03-25 16:36:30 +05:30
tedyu b50b2e51bf HBASE-20272 TestAsyncTable#testCheckAndMutateWithTimeRange fails due to TableExistsException 2018-03-24 06:27:20 -07:00
zhangduo c44e886860 HBASE-20271 ReplicationSourceWALReader.switched should use the file name instead of the path object directly 2018-03-24 21:12:40 +08:00
Jan Hentschel 64ccd2b295 HBASE-13300 Fixed casing in set/getTimeStamp for Mutations 2018-03-23 17:23:22 -07:00
Sakthi b8f999bf33 HBASE-20135 Fixed NullPointerException during reading bloom filter when upgraded from hbase-1 to hbase-2 2018-03-23 15:21:51 -07:00
Toshihiro Suzuki 88eac3ca18 HBASE-20261 Table page (table.jsp) in Master UI does not show replicaIds for hbase meta table
Signed-off-by: Josh Elser <elserj@apache.org>
2018-03-23 15:31:30 -04:00
Chia-Ping Tsai ad47c2daf4 HBASE-19504 Add TimeRange support into checkAndMutate
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-24 00:12:38 +08:00
zhangduo 64061f896f HBASE-20147 Serial replication will be stuck if we create a table with serial replication but add it to a peer after there are region moves 2018-03-23 14:31:20 +08:00
Umesh Agashe c614b9f3e8 HBASE-20224 Web UI is broken in standalone mode
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
2018-03-22 20:27:39 -07:00
Michael Stack 5d1b2110d1
Revert "HBASE-20224 Web UI is broken in standalone mode"
Broke shell tests.

This reverts commit dd9fe813ec.
2018-03-22 10:57:42 -07:00
Umesh Agashe 4cb40e6d84 HBASE-20224 Web UI is broken in standalone mode
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
2018-03-22 06:52:20 -07:00
Chia-Ping Tsai a6eeb26cc0 HBASE-20212 Make all Public classes have InterfaceAudience category
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-22 18:10:23 +08:00
Peter Somogyi 68b2f5502a HBASE-17449 Add explicit document on different timeout settings 2018-03-22 12:06:22 +05:30
Michael Stack 9601ab2272 HBASE-20237 Put back getClosestRowBefore and throw UnsupportedOperation instead... for asynchbase client Throw exception if an old client connects. 2018-03-21 21:51:25 -07:00
zhangduo d7475b92e8 HBASE-20242 The open sequence number will grow if we fail to open a region after writing the max sequence id file 2018-03-22 08:42:36 +08:00
Andrew Purtell 17ac2fe9c1 HBASE-19024 Provide a configurable option to hsync WAL edits to the disk for better durability (Harshal Jain) 2018-03-20 21:07:59 -07:00
tedyu e7cfdac7b5 HBASE-20090 Properly handle Preconditions check failure in MemStoreFlusher.run 2018-03-20 21:02:09 -07:00
Michael Stack acbdb86bb4 HBASE-20169 NPE when calling HBTU.shutdownMiniCluster
Adds a prepare step to RecoverMetaProcedure in which we test for
cluster up and master being up. If not up, we fail the run.

Modified hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java
Modified hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java
 Minor log cleanup.

Modified hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RecoverMetaProcedure.java
 Add pepare step.

Modified hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerMetrics.java
 Debug for the failing test....

Added hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRecoverMetaProcedure.java
 Test the prepare step goes down if master or cluster are down.
2018-03-20 13:11:57 -07:00
Michael Stack 74c28bdf44 HBASE-20232 [LOGGING] Formatting around close and flush 2018-03-20 10:34:49 -07:00
tedyu 2a3f4a0a4e HBASE-20214 Review of RegionLocationFinder Class - revert due to the pending removal of commons-collections4 dependency 2018-03-19 16:52:27 -07:00
tedyu df5de33a02 HBASE-20196 Maintain all regions with same size in memstore flusher 2018-03-19 08:10:39 -07:00
zhangduo 67f013430c HBASE-20206 WALEntryStream should not switch WAL file silently 2018-03-19 11:40:11 +08:00
zhangduo 00095a2ef9 Revert "HBASE-19665 Add table based replication peers/queues storage back"
This reverts commit 31978c31bb.

 Conflicts:
	hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/TableReplicationStorageBase.java
2018-03-17 20:25:27 +08:00