Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,426 Commits 95 Branches 866 Tags 550 MiB
Commit Graph

6853 Commits

Author SHA1 Message Date
eshcar 1eabbb4295 HBASE-20390: IMC Default Parameters for 2.0.0 2018-05-26 22:57:28 +03:00
Toshihiro Suzuki b1089e8310 HBASE-20648 HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-05-25 07:52:40 -07:00
meiyi 36f3d9432a HBASE-20518 Need to serialize the enabled field for UpdatePeerConfigProcedure 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-05-25 14:36:16 +08:00
Thiruvel Thirumoolan 1fbce10ff4 HBASE-20548 Master fails to startup on large clusters, refreshing block distribution 
Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-05-24 15:47:24 -07:00
Toshihiro Suzuki 554d513f50 HBASE-20616 TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-05-24 15:16:30 -07:00
Nihal Jain 09dac89908 HBASE-20588 Space quota change after quota violation doesn't seem to take in effect 
Signed-off-by: Josh Elser <elserj@apache.org>
 2018-05-24 12:40:55 -04:00
Guanghao Zhang 320a3332e0 HBASE-20589 Don't need to assign meta to a new RS when standby master become active 2018-05-24 11:26:48 +08:00
zhangduo ee540c9f9e HBASE-20624 Race in ReplicationSource which causes walEntryFilter being null when creating new shipper 2018-05-24 10:48:29 +08:00
Andrew Purtell 9fbce1668b HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint 2018-05-23 16:46:22 -07:00
Michael Stack 079f168c5c HBASE-20620 HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; part 2 
Adds new stripped-down, faster ByteBufferKeyValue comparator
(BBKV is the base Cell-type in hbase2). Creates an instance
of new Comparator each time we create new memstore rather
than use the universal CellComparator.

Remove unused and unneeded Interfaces from Cell base type.
 2018-05-23 13:20:47 -07:00
huzheng 5721150c6d HBASE-20612 TestReplicationKillSlaveRSWithSeparateOldWALs sometimes fail because it uses an expired cluster conn 2018-05-23 12:07:01 +08:00
tedyu 6c1097e92f HBASE-20609 SnapshotHFileCleaner#init should check that params is not null 2018-05-21 18:36:38 -07:00
jingyuntian c9f8c3436f HBASE-20579 Improve snapshot manifest copy in ExportSnapshot 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-05-18 06:42:12 -07:00
Guanghao Zhang 0836b0719a HBASE-20583 SplitLogWorker should handle FileNotFoundException when split a wal 2018-05-18 14:29:41 +08:00
Balazs Meszaros 6148b4785d HBASE-20571 JMXJsonServlet generates invalid JSON if it has NaN in metrics 
- CacheStats won't generate NaN metrics.
- JSONBean class will serialize special floating point values as
  "NaN", "Infinity" or "-Infinity"

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-05-16 12:20:39 -07:00
Apekshit Sharma 8c9825a030 HBASE-20567 Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace. 
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-05-16 14:03:18 -05:00
Michael Stack 438af9bf74
HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; ADDENDUM 
Add method the CellComparator Interface. Add implementation to
meta comparator so we don't fall back to the default comparator.

Includes a nothing change to hbase-server/pom.xml just to provoke
build.
 2018-05-16 09:43:16 -07:00
Michael Stack 060b8aca86
HBASE-20520 Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize 2018-05-16 09:19:24 -07:00
zhangduo ab53329cb3 HBASE-20585 Need to clear peer map when clearing MasterProcedureScheduler 2018-05-16 08:46:29 +08:00
zhangduo 26babcf013 HBASE-20457 Return immediately for a scan rpc call when we want to switch from pread to stream 2018-05-15 20:56:20 +08:00
Zach York d2daada970 HBASE-20447 Only fail cacheBlock if block collisions aren't related to next block metadata 
When we pread, we don't force the read to read all of the next block header.
However, when we get into a race condition where two opener threads try to
cache the same block and one thread read all of the next block header and
the other one didn't, it will fail the open process. This is especially important
in a splitting case where it will potentially fail the split process.
Instead, in the caches, we should only fail if the required blocks are different.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-05-14 17:16:54 -07:00
huzheng eabe672ebd HBASE-20560 Revisit the TestReplicationDroppedTables ut 2018-05-14 19:12:43 +08:00
Michael Stack 021f66d11d
HBASE-20411 Ameliorate MutableSegment synchronize 
Change the MemStore size accounting so we don't synchronize across three
volatiles applying deltas. Instead:

 + Make MemStoreSize, a datastructure of our memstore size longs, immutable.
 + Undo MemStoreSizing being an instance of MemStoreSize; instead it has-a.
 + Make two MemStoreSizing implementations; one thread-safe, the other not.
 + Let all memory sizing longs run independent, untied by
   synchronize (Huaxiang and Anoop suggestion) using atomiclongs.
 + Review all use of MemStoreSizing. Many are single-threaded and do
   not need to be synchronized; use the non-thread safe counter.

TODO: Use this technique accounting at the global level too.
 2018-05-12 02:17:50 +01:00
Sean Busbey 8ba2a7eeb9 HBASE-20544 Make HBTU default to random ports. 
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Josh Elser <elserj@apache.org>
 2018-05-09 23:35:20 -07:00
Thiruvel Thirumoolan a67909d3d6 HBASE-20545 Improve performance of BaseLoadBalancer.retainAssignment 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-05-09 19:48:27 -07:00
Andrew Purtell 32dd633b98 HBASE-20554 "WALs outstanding" message from CleanerChore is noisy 2018-05-09 19:08:35 -07:00
Zach York 9c046c091c HBASE-20204 Add locking to RefreshFileConnections in BucketCache 
This is a follow-up to HBASE-20141 where Anoop suggested adding locking
for refreshing channels.
 2018-05-09 14:01:51 -07:00
Michael Stack 4f2dfd3877 HBASE-20539 Disable IMC; part 2 
Set default for in-memory compaction to NONE. PE was using
this value creating its table. Add test to ensure expected
default.
 2018-05-09 06:30:28 -07:00
Nihal Jain f8281aa208 HBASE-20485 Copy constructor of Scan doesn't copy the readType and replicaId 
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
 2018-05-09 16:39:57 +08:00
Guanghao Zhang 3a2a76f6f8 HBASE-20536 Make TestRegionServerAccounting stable and it should not use absolute number 2018-05-08 08:20:48 +08:00
Michael Stack bb1a935636 HBASE-20538 TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property 
Disable test TestSaslFanOutOneBlockAsyncDFSOutput for the moment.
Depends on HDFS-13494
 2018-05-07 09:54:32 -07:00
huzheng 971f5350e8 HBASE-20531 RS may throw NPE when close meta regions in shutdown procedure. 2018-05-06 11:18:23 +08:00
jingyuntian 87f5b5f341 HBASE-20378 Provide a hbck option to cleanup replication barrier for a table 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-05-04 15:27:33 +08:00
huzheng 6225b4a492 HBASE-20481 Replicate entries from same region serially in ReplicationEndpoint for serial replication 2018-05-04 15:22:02 +08:00
Guanghao Zhang 9b9f851470 HBASE-20524 Need to clear metrics when ReplicationSourceManager refresh replication sources 2018-05-04 14:31:41 +08:00
Michael Stack 09ca619389
HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer 2018-05-03 12:30:13 -07:00
Chia-Ping Tsai 4cb444e77b
HBASE-20169 NPE when calling HBTU.shutdownMiniCluster (TestAssignmentManagerMetrics is flakey); AMENDMENT 2018-05-02 16:14:58 -07:00
Umesh Agashe c4b4023b60 HBASE-20514 On Master restart if table is stuck in DISABLING state, CLOSED regions should not be considered stuck in-transition 
On startup CLOSED regions of DISABLED and DISABLING tables are treated the same way as not in-transition.
 2018-05-02 12:18:24 -07:00
tedyu 2e9b96e4f1 HBASE-20414 TestLockProcedure#testMultipleLocks may fail on slow machine 2018-05-02 06:39:51 -07:00
zhangduo 5bdb52af2f HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer 2018-05-02 10:26:55 +08:00
Michael Stack 5a071dbe2b HBASE-20492 UnassignProcedure is stuck in retry loop on region stuck in OPENING state 
Add backoff when stuck in RegionTransitionProcedure, the subclass of
AssignProcedure and UnassignProcedure. Can happen when we go to
transition but the current Region state is not what we expect.

M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
 Add doc on being able to suspend and wait on a timeout.

M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
 Add 'attempt' counter so we can do backoff when we get stuck.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
 Add persistence of new 'attempt' counter

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
 Doc data members that are persisted by subclasses given this is 'odd'.
 Add a counter for 'attempts' used when 'stuck' to implement backoff.
 Add suspend with timeout when 'stuck'. Add callback when timeout is
 exhausted which does wakeup of this procedure.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestUnexpectedStateException.java
 Test of backoff.
 2018-04-30 20:40:22 -07:00
Nihal Jain 6d080762ef HBASE-20499 Replication/Priority executors can use specific max queue length as default value instead of general maxQueueLength 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-04-30 07:42:32 -07:00
huzheng a136303833 HBASE-20497 The getRecoveredQueueStartPos always return 0 in RecoveredReplicationSourceShipper 
Signed-off-by: zhangduo <zhangduo@apache.org>
 2018-04-28 20:50:30 +08:00
zhangduo 4be96dd8a6 HBASE-20476 Open sequence number could go backwards in AssignProcedure 2018-04-28 06:49:50 +08:00
Balazs Meszaros 650340abb5 HBASE-20487 Sorting table regions by region name does not work on web UI 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-04-27 11:13:00 -07:00
huzheng e9a278adc6 HBASE-20475 Fix the flaky TestReplicationDroppedTables unit test - addendum 2018-04-27 21:38:15 +08:00
zhangduo 39cf42be9a HBASE-20476 Add more logs for debugging 2018-04-27 16:02:05 +08:00
zhangduo 96ed407c69 HBASE-20476 Fix the flaky TestReplicationSmallTests unit test 2018-04-27 10:25:52 +08:00
Sakthi f761689136 HBASE-20040 Fixed: Master UI should include "Cluster Key" needed to use the cluster as a replication sink 
Signed-off-by: Sean Busbey <busbey@apache.org>
 2018-04-26 06:43:55 -05:00
huzheng 12c45cb2e8 HBASE-20475 Fix the flaky TestReplicationDroppedTables unit test. 2018-04-25 11:21:59 +08:00
Rich Fecher a8be3bb814 HBASE-20466 Consistently use override mechanism for exempt classes in CoprocessorClassloader 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-04-23 07:47:30 -07:00
Michael Stack 1633d9d61d HBASE-20470 [2.0.0RC1] has broken unit tests... 
Fix test that depended upon flush being slow and one family only.
Fix MemStoreSize compare to allow passing alternate implementation
(needed when IMC was no longer default everywhere).
 2018-04-22 15:33:25 -07:00
Huaxiang Sun e22f9e2d97 HBASE-19924 hbase rpc throttling does not work for multi() with request count rater. 2018-04-20 16:54:03 -07:00
Nihal Jain 758f4296a4 HBASE-20450 Provide metrics for number of total active, priority and replication rpc handlers 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-04-20 16:24:32 -07:00
Toshihiro Suzuki 4e183748c7 HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey 
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
 2018-04-20 10:51:11 -05:00
Sean Busbey 914de11416 HBASE-20442 clean up incorrect use of commons-collections 3 
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
 2018-04-20 07:30:34 -05:00
Sean Busbey 09749f1574 HBASE-20440 Clean up incorrect use of commons-lang 2.y 
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
 2018-04-20 07:29:23 -05:00
Sean Busbey b1fc00e16f HBASE-20439 Clean up incorrect use of commons-logging in hbase-server 
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
 2018-04-20 07:27:18 -05:00
jingyuntian 556b223744 HBASE-18059 remove scanner order related code 2018-04-18 19:38:08 -07:00
Umesh Agashe 6ca8261fc0 HBASE-18792 Disabled destructive (mostly write) operations in hbck for HBase 2.0+ 
HBase server versioned is checked after connecting to the server and then following operations are not allowed:
-fix, -fixAssignments, -fixMeta, -fixHdfsHoles, -fixHdfsOrphans, -fixTableOrphans, -fixHdfsOverlaps, -maxMerge
-sidelineBigOverlaps, -maxOverlapsToSideline, -fixSplitParents, -removeParents, -fixEmptyMetaCells
-repair, -repairHoles
 2018-04-18 15:22:39 -07:00
Nihal Jain 6ce1136eba HBASE-20452 Master UI: Table merge button should validate required fields before submit 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-04-18 14:55:12 -07:00
wangyu 80cbc0d1fe HBASE-20449 The minimun number of region should be configurable in Normalizer 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-04-18 08:45:52 -07:00
Sean Busbey 824ee525df HBASE-20404 Fixes to CleanChore correctness and operability. 
* Make CleanerChore less chatty: move WARN message to DEBUG when we expect non-empty dirs
* Make CleanerChore less chatty: move IOE we'll retry to INFO
* CleanerChore should treat IOE for FileStatus as a failure
* Add tests asserting assumptions in above

Signed-off-by: Reid Chan <reidddchan@outlook.com>
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-04-17 11:32:44 -05:00
Balazs Meszaros b926d359b2 HBASE-20398 Redirect doesn't work on web UI 2018-04-17 09:18:52 -07:00
Balazs Meszaros 32764bfb2c HBASE-20399 Fix merge layout 2018-04-17 09:17:25 -07:00
huzheng 42d5447cfb HBASE-20128 Add new UTs which extends the old replication UTs but set replication scope to SERIAL 2018-04-17 10:56:21 +08:00
Huaxiang Sun 1fe3d6a0e9 HBASE-19994 Create a new class for RPC throttling exception, make it retryable 2018-04-16 17:27:14 -07:00
Wei-Chiu Chuang 44ebd28093 HBASE-19963 TestFSHDFSUtils assumes wrong default port for Hadoop 3.0.1+ 
Signed-off-by: Mike Drob <mdrob@apache.org>
 2018-04-16 12:00:17 -05:00
zhangduo 773aff90fd HBASE-20417 Do not read wal entries when peer is disabled 2018-04-16 22:07:33 +08:00
lujie 1339ff9666 HBASE-20419 Fix potential NPE in ZKUtil#listChildrenAndWatchForNewChildren callers 
Signed-off-by: Yu Li <liyu@apache.org>
 2018-04-16 16:06:57 +08:00
Peter Somogyi 125767b44e HBASE-20409 Set hbase.client.meta.operation.timeout in TestClientOperationTimeout 
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
 2018-04-15 10:12:00 +02:00
zhangduo edf5049502 HBASE-20294 Also cleanup last pushed sequence id in ReplicationBarrierCleaner 2018-04-14 14:56:18 +08:00
zhangduo 5a633adffe HBASE-20377 Deal with table in enabling and disabling state when modifying serial replication peer 2018-04-13 20:33:29 +08:00
Guangxu Cheng 5a69465ea0 HBASE-20243 [Shell] Add shell command to create a new table by cloning the existent table 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-04-11 19:33:56 -07:00
Apekshit Sharma 281c29ff60 HBASE-20350 NullPointerException in Scanner during close() 2018-04-11 14:34:57 -07:00
Umesh Agashe 199b392ec8 HBASE-20310 Fixed false inconsistency shown by hbck -metaonly option on HBase 2 
When -metaonly option is specified then tablesInfo is not populated for all tables from hdfs causing false inconsistency of dangling state in meta.
 2018-04-11 12:51:12 -07:00
Toshihiro Suzuki 072c503ec7 HBASE-20219 An error occurs when scanning with reversed=true and loadColumnFamiliesOnDemand=true 
Signed-off-by: tedyu <yuzhihong@gmail.com>
 2018-04-11 09:59:45 -07:00
Balazs Meszaros 35b4adf3ed HBASE-20382 If RSGroups not enabled, rsgroup.jsp prints stack trace 2018-04-11 09:50:07 -07:00
Michael Stack ba02a8664b HBASE-20384 [AMv2] Logging format improvements; use encoded name rather than full region name marking transitions 2018-04-11 08:35:54 -07:00
zhangduo 37e5b0b1b7 HBASE-20367 Write a replication barrier for regions when disabling a table 2018-04-11 20:36:51 +08:00
Ashish Singhi 828a1c76c7 HBASE-15291 FileSystem not closed in secure bulkLoad 
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
 2018-04-11 12:01:28 +05:30
zhangduo bc35de597d HBASE-20363 Addendum wait until region is splittable 2018-04-09 09:07:06 +08:00
zhangduo f66343050f HBASE-20363 TestNamespaceAuditor.testRegionMerge is flaky 2018-04-08 16:48:25 +08:00
zhangduo 852025dd00 HBASE-20362 TestMasterShutdown.testMasterShutdownBeforeStartingAnyRegionServer is flaky 2018-04-08 16:09:50 +08:00
zhangduo adc0e85e85 HBASE-20182 Can not locate region after split and merge 2018-04-07 13:37:53 +08:00
tedyu 8f6849ff02 HBASE-20095 Redesign single instance pool in CleanerChore - addendum simplifies onConfigurationChange 2018-04-05 10:24:37 -07:00
BELUGA BEHR d866e7c658 HBASE-19488 Move to using Apache commons CollectionUtils 
Signed-off-by: Apekshit Sharma <appy@apache.org>
 2018-04-04 14:12:19 -07:00
Ashish Singhi 5937202fd5 HBASE-16499 slow replication for small HBase clusters 
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
 2018-04-04 09:54:41 +05:30
Michael Stack b1b0db3195 HBASE-20298 Doc change in read/write/total accounting metrics 2018-04-03 17:00:43 -07:00
Michael Stack c88ca54c16 HBASE-20259 Doc configs for in-memory-compaction and add detail to 
in-memory-compaction logging

Adds logging of CompactingMemStore configuration on construction.

Add logging of detail about Store on creation including memstore type.

Add chapter to refguide on new in-memory compaction feature.
 2018-04-02 09:53:12 -07:00
zhangduo 1e56938757 HBASE-20296 Remove last pushed sequence ids when removing tables from a peer 2018-04-01 21:14:31 +08:00
Jan Hentschel 97b3a04019 HBASE-17631 Changed Canary interval to 60 seconds 2018-04-01 13:03:52 +02:00
Mike Drob d57001ee2d HBASE-20282 Clean up tooling docs/help 2018-03-30 08:59:47 -07:00
tedyu e60f34fdb5 HBASE-20159 Support using separate ZK quorums for client - addendum fixes flaky TestSeparateClientZKCluster 2018-03-30 07:19:47 -07:00
Vasudevan cbd8b15e6b HBASE-20197 Review of ByteBufferWriterOutputStream.java (BELUGA BEHR) 
Signed-off-by: Vasudevan <ramkrishna.s.vasudevan@gmail.com>
 2018-03-30 12:13:41 +05:30
Mike Drob 621e2b5f5e HBASE-20273 Additional removed/changed configs 2018-03-29 11:47:14 -07:00
Yu Li 061a31fad1 HBASE-20159 Support using separate ZK quorums for client 2018-03-29 02:37:26 +08:00
Josh Elser 69f5d707b6 HBASE-20199 Add a unit test to verify flush and snapshot permission requirements aren't excessive 
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
 2018-03-27 20:17:08 -04:00
Zach York 09ed7c7a10 HBASE-20280 Fix possibility of deadlocking in refreshFileConnections 
When prefetch on open is specified, there is a deadlocking case
where if the prefetch is cancelled, the PrefetchExecutor interrupts
the threads if necessary, when that happens in FileIOEngine, it
causes an ClosedByInterruptException which is a subclass of
ClosedChannelException. If we retry all ClosedChannelExceptions,
this will lock as this access is expected to be interrupted.
This change removes calling refreshFileConnections for
ClosedByInterruptExceptions.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
 2018-03-27 16:53:01 -07:00
Rajeshbabu Chintaguntla d87139989f HBASE-20111 A region's splittable state now includes the configuration splitPolicy 
The Master asks a RegionServer whether a Region can be split or not, primarily to
verify that the region is not closing, opening, etc. This change has the RegionServer
also consult the configured RegionSplitPolicy.

Signed-off-by: Josh Elser <elserj@apache.org>
 2018-03-27 18:42:49 -04:00