Commit Graph

7074 Commits

Author SHA1 Message Date
tedyu 856a3ac154 HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint - revert due to pending discussion 2018-05-29 19:58:32 -07:00
Andrew Purtell 2dc51934f4 HBASE-20597 Serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint 2018-05-29 11:29:12 -07:00
Andrew Purtell 7f154dc20e Revert "HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint"
This reverts commit 60dcef289b.
2018-05-29 11:24:30 -07:00
Nihal Jain d36cce1574 HBASE-20633 Dropping a table containing a disable violation policy fails to remove the quota upon table delete
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2018-05-29 11:50:40 -04:00
eshcar aa00391140 HBASE-20390 ADDENDUM 2: fix TestHRegionWithInMemoryFlush OOME 2018-05-29 16:24:27 +03:00
eshcar cf1928aaca HBASE-20390-ADDENDUM: fix TestHRegionWithInMemoryFlush OOME 2018-05-29 13:01:07 +03:00
huzheng c8fd6e0fb6 HBASE-20533 Fix the flaky TestAssignmentManagerMetrics 2018-05-29 09:50:04 +08:00
Toshihiro Suzuki 0455e75edd HBASE-20648 HBASE-19364 "Truncate_preserve fails with table when replica region > 1" for master branch
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-28 08:27:41 -07:00
Nihal Jain bc72fcd8c5 HBASE-20639 Implement permission checking through AccessController instead of RSGroupAdminEndpoint
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-27 18:43:08 -07:00
meiyi f40c10a211 HBASE-20518 Need to serialize the enabled field for UpdatePeerConfigProcedure
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-05-25 14:45:49 +08:00
Thiruvel Thirumoolan d1cbd561df HBASE-20548 Master fails to startup on large clusters, refreshing block distribution
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-24 15:47:22 -07:00
Toshihiro Suzuki db8789ab22 HBASE-20616 TruncateTableProcedure is stuck in retry loop in TRUNCATE_TABLE_CREATE_FS_LAYOUT state
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-24 15:17:01 -07:00
Nihal Jain 55c4791a63 HBASE-20588 Space quota change after quota violation doesn't seem to take in effect
Signed-off-by: Josh Elser <elserj@apache.org>
2018-05-24 12:50:22 -04:00
eshcar 079e08d7c5 HBASE-20390: IMC Default Parameters for 2.0.0 2018-05-24 18:10:10 +03:00
Guanghao Zhang bfab1e2f92 HBASE-20589 Don't need to assign meta to a new RS when standby master become active 2018-05-24 11:45:59 +08:00
zhangduo a94c6dbadb HBASE-20624 Race in ReplicationSource which causes walEntryFilter being null when creating new shipper 2018-05-24 10:48:35 +08:00
Andrew Purtell 60dcef289b HBASE-20597 Use a lock to serialize access to a shared reference to ZooKeeperWatcher in HBaseReplicationEndpoint 2018-05-23 16:46:20 -07:00
Michael Stack afddf6b1c2 HBASE-20620 HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; part 2
Adds new stripped-down, faster ByteBufferKeyValue comparator
(BBKV is the base Cell-type in hbase2). Creates an instance
of new Comparator each time we create new memstore rather
than use the universal CellComparator.

Remove unused and unneeded Interfaces from Cell base type.
2018-05-23 13:20:29 -07:00
huzheng fbda502435 HBASE-20612 TestReplicationKillSlaveRSWithSeparateOldWALs sometimes fail because it uses an expired cluster conn 2018-05-23 12:07:54 +08:00
jingyuntian c3c9a4a595 HBASE-20579 Improve snapshot manifest copy in ExportSnapshot
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-18 06:43:53 -07:00
Guanghao Zhang d06673cf3e HBASE-20583 SplitLogWorker should handle FileNotFoundException when split a wal 2018-05-18 14:30:40 +08:00
Balazs Meszaros 39ea1efa88 HBASE-20571 JMXJsonServlet generates invalid JSON if it has NaN in metrics
- CacheStats won't generate NaN metrics.
- JSONBean class will serialize special floating point values as
  "NaN", "Infinity" or "-Infinity"

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-16 12:20:37 -07:00
Apekshit Sharma 61f2b5f071 HBASE-20567 Pass both old and new descriptors to pre/post hooks of modify operations for table and namespace.
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-05-16 14:03:36 -05:00
Michael Stack 77eaff0e10
HBASE-20564 Tighter ByteBufferKeyValue Cell Comparator; ADDENDUM
Add method the CellComparator Interface. Add implementation to
meta comparator so we don't fall back to the default comparator.

Includes a nothing change to hbase-server/pom.xml just to provoke
build.
2018-05-16 09:42:51 -07:00
Michael Stack 5c4685e56e
HBASE-20520 Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize 2018-05-16 09:18:06 -07:00
zhangduo 82e3011166 HBASE-20585 Need to clear peer map when clearing MasterProcedureScheduler 2018-05-16 08:46:34 +08:00
zhangduo 60b8344cf1 HBASE-20457 Return immediately for a scan rpc call when we want to switch from pread to stream 2018-05-15 21:09:04 +08:00
Zach York b7def9b690 HBASE-20447 Only fail cacheBlock if block collisions aren't related to next block metadata
When we pread, we don't force the read to read all of the next block header.
However, when we get into a race condition where two opener threads try to
cache the same block and one thread read all of the next block header and
the other one didn't, it will fail the open process. This is especially important
in a splitting case where it will potentially fail the split process.
Instead, in the caches, we should only fail if the required blocks are different.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-05-14 16:09:14 -07:00
huzheng 4b0ac73f51 HBASE-20560 Revisit the TestReplicationDroppedTables ut 2018-05-14 19:33:51 +08:00
huzheng be3df29cef HBASE-20128 Add new UTs which extends the old replication UTs but set replication scope to SERIAL 2018-05-14 19:32:39 +08:00
Michael Stack 5ac7740896
HBASE-20411 Ameliorate MutableSegment synchronize
Change the MemStore size accounting so we don't synchronize across three
volatiles applying deltas. Instead:

 + Make MemStoreSize, a datastructure of our memstore size longs, immutable.
 + Undo MemStoreSizing being an instance of MemStoreSize; instead it has-a.
 + Make two MemStoreSizing implementations; one thread-safe, the other not.
 + Let all memory sizing longs run independent, untied by
   synchronize (Huaxiang and Anoop suggestion) using atomiclongs.
 + Review all use of MemStoreSizing. Many are single-threaded and do
   not need to be synchronized; use the non-thread safe counter.

TODO: Use this technique accounting at the global level too.
2018-05-12 02:16:19 +01:00
Thiruvel Thirumoolan 1f10ef553e HBASE-20545 Improve performance of BaseLoadBalancer.retainAssignment
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-05-10 10:45:17 -07:00
Sean Busbey 61f96b6ffa HBASE-20544 Make HBTU default to random ports.
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Josh Elser <elserj@apache.org>

 Conflicts:
	hbase-backup/src/test/resources/hbase-site.xml
	hbase-spark-it/src/test/resources/hbase-site.xml
	hbase-spark/src/test/resources/hbase-site.xml
2018-05-09 23:45:39 -07:00
Andrew Purtell c430016cf9 HBASE-20554 "WALs outstanding" message from CleanerChore is noisy 2018-05-09 19:11:50 -07:00
Zach York cba8d2fb8d HBASE-20204 Add locking to RefreshFileConnections in BucketCache
This is a follow-up to HBASE-20141 where Anoop suggested adding locking
for refreshing channels.
2018-05-09 14:23:27 -07:00
Michael Stack de25f8b209 HBASE-20539 Disable IMC; part 2
Set default for in-memory compaction to NONE. PE was using
this value creating its table. Add test to ensure expected
default.
2018-05-09 06:29:56 -07:00
Nihal Jain 55badefa38 HBASE-20485 Copy constructor of Scan doesn't copy the readType and replicaId
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-05-09 16:45:39 +08:00
Guanghao Zhang d6d209564c HBASE-20536 Make TestRegionServerAccounting stable and it should not use absolute number 2018-05-08 08:21:30 +08:00
Michael Stack fd0840c3e4 HBASE-20538 TestSaslFanOutOneBlockAsyncDFSOutput failing: UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
Disable test TestSaslFanOutOneBlockAsyncDFSOutput for the moment.
Depends on HDFS-13494
2018-05-07 09:54:07 -07:00
huzheng de71cb5915 HBASE-20481 Replicate entries from same region serially in ReplicationEndpoint for serial replication 2018-05-04 15:52:43 +08:00
jingyuntian 5f260451d5 HBASE-20378 Provide a hbck option to cleanup replication barrier for a table
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-05-04 15:20:17 +08:00
Guanghao Zhang eaafdff761 HBASE-20524 Need to clear metrics when ReplicationSourceManager refresh replication sources 2018-05-04 14:34:07 +08:00
Michael Stack e92580e201 HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer 2018-05-03 12:26:54 -07:00
tedyu f89c5b08df HBASE-20414 TestLockProcedure#testMultipleLocks may fail on slow machine 2018-05-03 07:12:11 -07:00
Chia-Ping Tsai 984fb5bd05
HBASE-20169 NPE when calling HBTU.shutdownMiniCluster (TestAssignmentManagerMetrics is flakey); AMENDMENT 2018-05-02 16:14:38 -07:00
Umesh Agashe b0719ec11b HBASE-20514 On Master restart if table is stuck in DISABLING state, CLOSED regions should not be considered stuck in-transition
On startup CLOSED regions of DISABLED and DISABLING tables are treated the same way as not in-transition.
2018-05-02 12:18:04 -07:00
zhangduo b43cde2ce1 HBASE-20507 Do not need to call recoverLease on the broken file when we fail to create a wal writer 2018-05-02 10:27:00 +08:00
Michael Stack da3e06afab HBASE-20492 UnassignProcedure is stuck in retry loop on region stuck in OPENING state
Add backoff when stuck in RegionTransitionProcedure, the subclass of
AssignProcedure and UnassignProcedure. Can happen when we go to
transition but the current Region state is not what we expect.

M hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
 Add doc on being able to suspend and wait on a timeout.

M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
 Add 'attempt' counter so we can do backoff when we get stuck.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
 Add persistence of new 'attempt' counter

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
 Doc data members that are persisted by subclasses given this is 'odd'.
 Add a counter for 'attempts' used when 'stuck' to implement backoff.
 Add suspend with timeout when 'stuck'. Add callback when timeout is
 exhausted which does wakeup of this procedure.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestUnexpectedStateException.java
 Test of backoff.
2018-04-30 17:58:27 -07:00
huzheng d38a104c4d HBASE-20497 The getRecoveredQueueStartPos always return 0 in RecoveredReplicationSourceShipper
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-28 20:50:47 +08:00
huzheng 72093178fb HBASE-20475 Fix the flaky TestReplicationDroppedTables unit test.
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-28 18:46:41 +08:00
zhangduo e8603e1d7c HBASE-20476 Open sequence number could go backwards in AssignProcedure 2018-04-28 16:55:06 +08:00
Sakthi 20e2ac132e HBASE-20040 Fixed: Master UI should include "Cluster Key" needed to use the cluster as a replication sink
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-26 06:46:07 -05:00
Rich Fecher 75004408b0 HBASE-20466 Consistently use override mechanism for exempt classes in CoprocessorClassloader
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-23 07:48:13 -07:00
Michael Stack 45c087d988 HBASE-20470 [2.0.0RC1] has broken unit tests...
Fix test that depended upon flush being slow and one family only.
Fix MemStoreSize compare to allow passing alternate implementation
(needed when IMC was no longer default everywhere).
2018-04-22 15:32:56 -07:00
Huaxiang Sun 298ce96246 HBASE-19924 hbase rpc throttling does not work for multi() with request count rater. 2018-04-20 16:55:23 -07:00
Toshihiro Suzuki 22e7ae0311 HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2018-04-20 11:22:49 -05:00
Sean Busbey 24eb141bac HBASE-20442 clean up incorrect use of commons-collections 3
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>

 Conflicts:
	hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/master/BackupLogCleaner.java
2018-04-20 08:17:22 -05:00
Sean Busbey 9740168fa8 HBASE-20440 Clean up incorrect use of commons-lang 2.y
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>

 Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierFactoryImpl.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierImpl.java
2018-04-20 08:17:22 -05:00
Sean Busbey 944ecc729c HBASE-20439 Clean up incorrect use of commons-logging in hbase-server
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>

 Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierImpl.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeReportingChore.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionSizeStoreImpl.java
 Additions:
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCellBasedHFileOutputFormat2.java
	hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCellBasedImportExport2.java
2018-04-20 08:04:17 -05:00
jingyuntian af4dd3eb3c HBASE-18059 remove scanner order related code 2018-04-18 19:37:12 -07:00
Umesh Agashe 3ac0ffde4c HBASE-18792 Disabled destructive (mostly write) operations in hbck for HBase 2.0+
HBase server versioned is checked after connecting to the server and then following operations are not allowed:
-fix, -fixAssignments, -fixMeta, -fixHdfsHoles, -fixHdfsOrphans, -fixTableOrphans, -fixHdfsOverlaps, -maxMerge
-sidelineBigOverlaps, -maxOverlapsToSideline, -fixSplitParents, -removeParents, -fixEmptyMetaCells
-repair, -repairHoles
2018-04-18 15:22:03 -07:00
Huaxiang Sun 36680da549 HBASE-19994 Create a new class for RPC throttling exception, make it retryable 2018-04-17 10:45:44 -07:00
Sean Busbey 1c8d9d788f HBASE-20404 Fixes to CleanChore correctness and operability.
* Make CleanerChore less chatty: move WARN message to DEBUG when we expect non-empty dirs
* Make CleanerChore less chatty: move IOE we'll retry to INFO
* CleanerChore should treat IOE for FileStatus as a failure
* Add tests asserting assumptions in above

Signed-off-by: Reid Chan <reidddchan@outlook.com>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-17 11:53:19 -05:00
Balazs Meszaros d951675df5 HBASE-20398 Redirect doesn't work on web UI 2018-04-17 09:19:13 -07:00
Balazs Meszaros 46c7808393 HBASE-20399 Fix merge layout 2018-04-17 09:16:58 -07:00
Wei-Chiu Chuang ecd9914dc9 HBASE-19963 TestFSHDFSUtils assumes wrong default port for Hadoop 3.0.1+
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-04-16 12:00:08 -05:00
zhangduo f73986d110 HBASE-20417 Do not read wal entries when peer is disabled 2018-04-16 22:07:48 +08:00
lujie 1cb05a18bc HBASE-20419 Fix potential NPE in ZKUtil#listChildrenAndWatchForNewChildren callers
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-16 16:07:59 +08:00
Peter Somogyi 4fea862963 HBASE-20409 Set hbase.client.meta.operation.timeout in TestClientOperationTimeout
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-04-15 10:12:34 +02:00
zhangduo f5d970eba1 HBASE-20294 Also cleanup last pushed sequence id in ReplicationBarrierCleaner 2018-04-14 14:56:51 +08:00
zhangduo ae8a21204d HBASE-20377 Deal with table in enabling and disabling state when modifying serial replication peer 2018-04-13 20:33:25 +08:00
Guangxu Cheng b0701434e3 HBASE-20243 [Shell] Add shell command to create a new table by cloning the existent table
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-04-11 19:34:30 -07:00
Apekshit Sharma 0f6ff895e9 HBASE-20350 NullPointerException in Scanner during close() 2018-04-11 14:34:23 -07:00
Umesh Agashe 0773209dd3 HBASE-20310 Fixed false inconsistency shown by hbck -metaonly option on HBase 2
When -metaonly option is specified then tablesInfo is not populated for all tables from hdfs causing false inconsistency of dangling state in meta.
2018-04-11 12:50:55 -07:00
Toshihiro Suzuki f3089704d2 HBASE-20219 An error occurs when scanning with reversed=true and loadColumnFamiliesOnDemand=true 2018-04-11 12:27:28 -07:00
Balazs Meszaros b52bf4052f HBASE-20382 If RSGroups not enabled, rsgroup.jsp prints stack trace 2018-04-11 09:49:50 -07:00
Michael Stack c78908cff3 HBASE-20384 [AMv2] Logging format improvements; use encoded name rather than full region name marking transitions 2018-04-11 08:40:49 -07:00
zhangduo 3eee7d37f4 HBASE-20367 Write a replication barrier for regions when disabling a table 2018-04-11 20:36:59 +08:00
Ashish Singhi 4bcb560e22 HBASE-15291 FileSystem not closed in secure bulkLoad
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-11 12:11:41 +05:30
zhangduo 2487b4603c HBASE-20363 TestNamespaceAuditor.testRegionMerge is flaky 2018-04-09 21:02:42 +08:00
zhangduo 74ab10c353 HBASE-20296 Remove last pushed sequence ids when removing tables from a peer 2018-04-09 15:18:44 +08:00
zhangduo ead569c951 HBASE-20285 Delete all last pushed sequence ids when removing a peer or removing the serial flag for a peer 2018-04-09 15:18:44 +08:00
zhangduo 8de9596355 HBASE-20127 Add UT for serial replication after failover 2018-04-09 15:18:44 +08:00
zhangduo 6f13b28c19 HBASE-20271 ReplicationSourceWALReader.switched should use the file name instead of the path object directly 2018-04-09 15:18:44 +08:00
zhangduo 9369cf26eb HBASE-20147 Serial replication will be stuck if we create a table with serial replication but add it to a peer after there are region moves 2018-04-09 15:18:44 +08:00
zhangduo aec43bb207 HBASE-20242 The open sequence number will grow if we fail to open a region after writing the max sequence id file 2018-04-09 15:18:44 +08:00
zhangduo 16a4dd6b8f HBASE-20206 WALEntryStream should not switch WAL file silently 2018-04-09 15:18:44 +08:00
zhangduo 644bfe36b2 HBASE-20117 Cleanup the unused replication barriers in meta table 2018-04-09 15:18:44 +08:00
zhangduo 2d0d6a3ba1 HBASE-20167 Optimize the implementation of ReplicationSourceWALReader 2018-04-09 15:18:44 +08:00
zhangduo cea5199ea1 HBASE-20148 Make serial replication as a option for a peer instead of a table 2018-04-09 15:18:44 +08:00
zhangduo 6b5412759f HBASE-20125 Add UT for serial replication after region split and merge 2018-04-09 15:18:44 +08:00
zhangduo 8b61a061d3 HBASE-20129 Add UT for serial replication checker 2018-04-09 15:18:44 +08:00
zhangduo f29bf1d778 HBASE-20115 Reimplement serial replication based on the new replication storage layer 2018-04-09 15:18:44 +08:00
huzheng 1d11cdb26c HBASE-20050 Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-09 15:18:44 +08:00
zhangduo 39c1ddc6e3 HBASE-20362 TestMasterShutdown.testMasterShutdownBeforeStartingAnyRegionServer is flaky 2018-04-09 14:55:42 +08:00
zhangduo dcc1d9e36a HBASE-20182 Can not locate region after split and merge 2018-04-07 13:37:59 +08:00
tedyu e78a8e08f0 HBASE-20095 Redesign single instance pool in CleanerChore - addendum simplifies onConfigurationChange 2018-04-05 10:25:09 -07:00
BELUGA BEHR 039bc73571 HBASE-19488 Move to using Apache commons CollectionUtils
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-04-04 14:16:33 -07:00
Ashish Singhi 9a34880724 HBASE-16499 slow replication for small HBase clusters
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-04 09:59:50 +05:30
Michael Stack ed21f26171 HBASE-20298 Doc change in read/write/total accounting metrics 2018-04-03 17:02:27 -07:00
Michael Stack d4e115bf34 HBASE-20259 Doc configs for in-memory-compaction and add detail to
in-memory-compaction logging

Adds logging of CompactingMemStore configuration on construction.

Add logging of detail about Store on creation including memstore type.

Add chapter to refguide on new in-memory compaction feature.
2018-04-02 09:56:32 -07:00
Jan Hentschel 4b909b890d HBASE-17631 Changed Canary interval to 60 seconds 2018-04-01 13:22:37 +02:00
Mike Drob 468d4fcd73 HBASE-20282 Clean up tooling docs/help 2018-03-30 09:00:46 -07:00
tedyu 4c9167a25c HBASE-20159 Support using separate ZK quorums for client - addendum fixes flaky TestSeparateClientZKCluster 2018-03-30 07:23:25 -07:00
Vasudevan 1687dc5ddb HBASE-20197 Review of ByteBufferWriterOutputStream.java (BELUGA BEHR)
Signed-off-by: Vasudevan <ramkrishna.s.vasudevan@gmail.com>
2018-03-30 12:20:09 +05:30
Yu Li b72e19e3b9 HBASE-20159 Support using separate ZK quorums for client 2018-03-29 03:20:52 +08:00
Josh Elser 5712fd0451 HBASE-20199 Add a unit test to verify flush and snapshot permission requirements aren't excessive
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-27 20:10:28 -04:00
Zach York 6f1aa0edff HBASE-20280 Fix possibility of deadlocking in refreshFileConnections
When prefetch on open is specified, there is a deadlocking case
where if the prefetch is cancelled, the PrefetchExecutor interrupts
the threads if necessary, when that happens in FileIOEngine, it
causes an ClosedByInterruptException which is a subclass of
ClosedChannelException. If we retry all ClosedChannelExceptions,
this will lock as this access is expected to be interrupted.
This change removes calling refreshFileConnections for
ClosedByInterruptExceptions.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-03-27 16:52:59 -07:00
Rajeshbabu Chintaguntla a601c57f97 HBASE-20111 A region's splittable state now includes the configuration splitPolicy
The Master asks a RegionServer whether a Region can be split or not, primarily to
verify that the region is not closing, opening, etc. This change has the RegionServer
also consult the configured RegionSplitPolicy.

Signed-off-by: Josh Elser <elserj@apache.org>
2018-03-27 14:58:58 -04:00
Toshihiro Suzuki e1afc5932a HBASE-20229 ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on 2018-03-26 15:13:13 -07:00
Josh Elser c3d82a283d HBASE-20223 Update to hbase-thirdparty 2.1.0
Remove commons-cli and commons-collections4 use. Account
for the newer internal protobuf version of 3.5.1.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-03-26 16:07:39 -04:00
Reid Chan 7c03ea0436 HBASE-20095 Redesign single instance pool in CleanerChore 2018-03-26 12:49:22 -05:00
anoopsamjohn b5881dbd3f HBASE-17819 Reduce the heap overhead for BucketCache. 2018-03-25 16:39:21 +05:30
tedyu c8c670b561 HBASE-20272 TestAsyncTable#testCheckAndMutateWithTimeRange fails due to TableExistsException 2018-03-24 12:42:49 -07:00
Michael Stack a6debe4845 Revert "HBASE-19504 Add TimeRange support into checkAndMutate; ADDENDUM to fix failing unit test"
Reverting an addendum in favor of patch at HBASE-20272.

This reverts commit 5da92ce352.
2018-03-24 12:41:26 -07:00
Michael Stack 5da92ce352 HBASE-19504 Add TimeRange support into checkAndMutate; ADDENDUM to fix failing unit test 2018-03-24 12:29:45 -07:00
Jan Hentschel 52014b9423 HBASE-13300 Fixed casing in set/getTimeStamp for Mutations 2018-03-23 17:23:55 -07:00
Sakthi e962e1ac2f HBASE-20135 Fixed NullPointerException during reading bloom filter when upgraded from hbase-1 to hbase-2 2018-03-23 15:20:43 -07:00
Toshihiro Suzuki e6d9e04511 HBASE-20261 Table page (table.jsp) in Master UI does not show replicaIds for hbase meta table
Signed-off-by: Josh Elser <elserj@apache.org>
2018-03-23 13:34:08 -04:00
Chia-Ping Tsai 6aba045aae HBASE-19504 Add TimeRange support into checkAndMutate
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-24 00:05:22 +08:00
Umesh Agashe 96d63fee11 HBASE-20224 Web UI is broken in standalone mode
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
2018-03-22 20:28:08 -07:00
Michael Stack 79e4c9d925
Revert "HBASE-20224 Web UI is broken in standalone mode"
Broke shell tests.

This reverts commit dd9fe813ec.
2018-03-22 10:47:47 -07:00
Umesh Agashe dd9fe813ec HBASE-20224 Web UI is broken in standalone mode
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
2018-03-22 06:52:51 -07:00
Chia-Ping Tsai dd9e46bbf5 HBASE-20212 Make all Public classes have InterfaceAudience category
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-22 18:09:54 +08:00
Peter Somogyi 0a94c9c250 HBASE-17449 Add explicit document on different timeout settings 2018-03-22 12:45:06 +05:30
Michael Stack 2bc99e4b5e HBASE-20237 Put back getClosestRowBefore and throw UnsupportedOperation instead... for asynchbase client Throw exception if an old client connects. 2018-03-21 21:48:15 -07:00
tedyu 7a130dcc33 HBASE-20090 Properly handle Preconditions check failure in MemStoreFlusher.run 2018-03-21 09:09:50 -07:00
Andrew Purtell 4fcc1ac7f7 HBASE-19024 Provide a configurable option to hsync WAL edits to the disk for better durability (Harshal Jain) 2018-03-20 17:30:39 -07:00
Michael Stack fabb1d97cc HBASE-20169 NPE when calling HBTU.shutdownMiniCluster
Adds a prepare step to RecoverMetaProcedure in which we test for
cluster up and master being up. If not up, we fail the run.

Modified hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java
Modified hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java
 Minor log cleanup.

Modified hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RecoverMetaProcedure.java
 Add pepare step.

Modified hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerMetrics.java
 Debug for the failing test....

Added hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRecoverMetaProcedure.java
 Test the prepare step goes down if master or cluster are down.
2018-03-20 13:09:43 -07:00
Michael Stack 75ff2504ec HBASE-20232 [LOGGING] Formatting around close and flush 2018-03-20 10:33:07 -07:00
Michael Stack 3f1c86786c HBASE-20213 [LOGGING] Aligning formatting and logging less (compactions,
in-memory compactions)

Log less. Log using same format as used elsewhere in log.

Align logs in HFileArchiver with how we format elsewhere. Removed
redundant 'region' qualifiers, tried to tighten up the emissions so
easier to read the long lines.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java
 Add a label for each of the chunkcreators we make (I was confused by
two chunk creater stats emissions in log file -- didn't know that one
was for data and the other index).

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java
 Formatting. Log less.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactionStrategy.java
 Make the emissions in here trace-level. When more than a few regions,
log is filled with this stuff.
2018-03-16 13:07:34 -07:00
Zach York 6bf967adfb [HBASE-20141] Fix TooManyFiles exception when RefreshingChannels
HBASE-19435 implements a fix for reopening file channels when they are unnexpected closed
to avoid disabling the BucketCache. However, it was missed that the the channels might not
actually be completely closed (the write or read channel might still be open
(see https://docs.oracle.com/javase/7/docs/api/java/nio/channels/ClosedChannelException.html)
This commit closes any open channels before creating a new channel.
2018-03-16 10:54:43 -07:00
Michael Stack 79d47dd57a HBASE-20202 [AMv2] Don't move region if its a split parent or offlined
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/DoNotRetryRegionException.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/exceptions/MergeRegionException.java
 Allow passing cause to Constructor.

M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
 Add prepare step to move procedure.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
 Add check that regions to merge are actually online to the Constructor
so we can fail fast if they are offline

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
 Add prepare step. Check regions and context and skip move if not right.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
 Add check parent region is online to constructor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java
 Add generic check region is online utility function for use by subclasses.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMove.java
 Add test that we fail if we try to move an offlined region.
2018-03-16 09:34:15 -07:00
Michael Stack e0bdc87b27
HBASE-20190 Fix default for MIGRATE_TABLE_STATE_FROM_ZK_KEY 2018-03-15 10:34:31 -07:00
Chia-Ping Tsai 95596e8ba7 HBASE-20119 Introduce a pojo class to carry coprocessor information in order to make TableDescriptorBuilder accept multiple cp at once
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-16 01:26:08 +08:00
Ashish Singhi 24ea9d1e48 HBASE-20146 Addendum Regions are stuck while opening when WAL is disabled
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-03-15 10:13:55 +08:00
Michael Stack 63ce0c0733 HBASE-20178 [AMv2] Throw exception if hostile environment
Add Fail-Fast to Procedures by throwing exception out of Procedure
constructor so if move but table is disabled or if master is going
down, etc., we can give notice before the procedure is scheduled.
Will help guard against scheduling Procedures that will have a hard
time succeeding; e.g. a move when table is offline.

Also fixed bug around table state where we presumed ENABLED though no
entry in hbase:meta (we were using this mechanism for whether a table
existed or not).

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMove.java
 Test stolen from HBASE-20131

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableState.java
 Add convenience isEnabled/isDisabled

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Promote assert state to throw exception.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
 Add isClusterUp

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Move constructor now throws exception
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RestoreSnapshotProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java
 Do environment check at construction and fail-fast if hostile.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java
 Add preflightCheck utility method.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Removed setting time setting table state; broke when using other than
 default environment edge masked by presumption that no state meant
 active.
2018-03-14 14:43:34 -07:00
Mike Drob 5e2f8e4023 HBASE-20180 Avoid Class::newInstance 2018-03-14 12:23:38 -05:00
Apekshit Sharma ad425e8603 HBASE-20185 Fix ACL check for MasterRpcServices#execProcedure 2018-03-14 19:08:17 +05:30
Chance Li 47b227b55b HBASE-19389 Limit concurrency of put with dense (hundreds) columns to prevent write handler exhausted
Signed-off-by: Yu Li <liyu@apache.org>
2018-03-14 18:36:58 +08:00
Yu Li c8ecfc5461 Revert "HBASE-19389 Limit concurrency of put with dense (hundreds) columns to prevent write handler exhausted"
This reverts commit c6d89b494c.
2018-03-14 18:35:22 +08:00
Yu Li c6d89b494c HBASE-19389 Limit concurrency of put with dense (hundreds) columns to prevent write handler exhausted
Signed-off-by: Yu Li <liyu@apache.org>
2018-03-14 18:33:39 +08:00
Mike Grimes a9d9fa35a2 HBASE-17165 Make use of retry setting in LoadIncrementalHFiles & fix test 2018-03-13 14:59:56 -07:00
Sahil Aggarwal 3757915dac HBASE-19075: Fix the 'tasks' table on master info page to not scroll up on clicking the tab 2018-03-13 14:24:56 -07:00
Michael Stack 260ee0da60 HBASE-20173 [AMv2] DisableTableProcedure concurrent to ServerCrashProcedure can deadlock
Allow that DisableTableProcedue can grab a region lock before
ServerCrashProcedure can. Cater to this cricumstance where SCP
was not unable to make progress by running the search for RIT
against the crashed server a second time, post creation of all
crashed-server assignemnts. The second run will uncover such as
the above DisableTableProcedure unassign and will interrupt its
suspend allowing both procedures to make progress.

M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
 Add new procedure step post-assigns that reruns the RIT finder method.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Make this important log more specific as to what is going on.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
 Better explanation as to what is going on.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
 Add extra step and run handleRIT a second time after we've queued up
 all SCP assigns. Also fix a but. SCP was adding an assign of a RIT
 that was actually trying to unassign (made the deadlock more likely).
2018-03-13 05:44:43 -07:00
BELUGA BEHR 4033405b36 HBASE-19449 Minor logging change in HFileArchiver
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-03-12 22:12:27 +05:30
Umesh Agashe 842678c991 HBASE-20120 Removed unused classes/ java files from hbase-server
deleted:    hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NoOpHeapMemoryTuner.java
deleted:    hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseWALEntryFilter.java
deleted:    hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSMapRUtils.java
deleted:    hbase-server/src/main/java/org/apache/hadoop/hbase/util/ProtoUtil.java
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-03-10 12:59:48 +08:00
Guangxu Cheng 1728b48c82 HBASE-20132 Change the "KV" to "Cell" for web UI
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-03-10 12:36:29 +08:00
Umesh Agashe 55e3dda25d HBASE-20024 Fixed flakyness of TestMergeTableRegionsProcedure
We assumed that we can run for loop from 0 to lastStep sequentially. MergeTableRegionProcedure skips step 2. So, when i is 0 the procedure is already at step 3.
Added a method StateMachineProcedure#getCurrentStateId that can be used from test code only.
2018-03-09 12:44:49 -08:00
Ashish Singhi cdf7be8929 HBASE-20146 Regions are stuck while opening when WAL is disabled
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 21:14:42 +08:00
zhangduo d4e0fc74d4 HBASE-20082 Fix findbugs errors only on master which are introduced by HBASE-19397 2018-03-09 20:55:48 +08:00
Guanghao Zhang 55d0d3026a HBASE-19923 Reset peer state and config when refresh replication source failed 2018-03-09 20:55:48 +08:00
zhangduo d1e775e35e HBASE-19936 Introduce a new base class for replication peer procedure 2018-03-09 20:55:48 +08:00
zhangduo abaa80f26e HBASE-19873 addendum add missing rule for new tests 2018-03-09 20:55:48 +08:00
Guanghao Zhang db4cb01a9e HBASE-19783 Change replication peer cluster key/endpoint from a not-null value to null is not allowed
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
huzheng a7dce36766 HBASE-19748 TestRegionReplicaFailover and TestRegionReplicaReplicationEndpoint UT hangs 2018-03-09 20:55:48 +08:00
zhangduo 1c32241a00 HBASE-19719 Fix checkstyle issues 2018-03-09 20:55:48 +08:00
Guanghao Zhang 1e69d7fa24 HBASE-19711 TestReplicationAdmin.testConcurrentPeerOperations hangs
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
zhangduo 0165455d34 HBASE-19707 Race in start and terminate of a replication source after we async start replicatione endpoint 2018-03-09 20:55:48 +08:00
Guanghao Zhang d36aacdf9e HBASE-19636 All rs should already start work with the new peer change when replication peer procedure is finished
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
zhangduo 53b18fe0ac HBASE-19634 Add permission check for executeProcedures in AccessController 2018-03-09 20:55:48 +08:00
zhangduo 029406c45b HBASE-19697 Remove TestReplicationAdminUsingProcedure 2018-03-09 20:55:48 +08:00
huzheng 5655b3c0ca HBASE-19661 Replace ReplicationStateZKBase with ZKReplicationStorageBase 2018-03-09 20:55:48 +08:00
zhangduo b84fbde175 HBASE-19687 Move the logic in ReplicationZKNodeCleaner to ReplicationChecker and remove ReplicationZKNodeCleanerChore 2018-03-09 20:55:48 +08:00
Guanghao Zhang 13318dd350 HBASE-19544 Add UTs for testing concurrent modifications on replication peer
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
zhangduo b0de0ec439 HBASE-19686 Use KeyLocker instead of ReentrantLock in PeerProcedureHandlerImpl 2018-03-09 20:55:48 +08:00
zhangduo 5d922a5e0c HBASE-19623 Create replication endpoint asynchronously when adding a replication source 2018-03-09 20:55:48 +08:00
zhangduo 525fef572e HBASE-19633 Clean up the replication queues in the postPeerModification stage when removing a peer 2018-03-09 20:55:48 +08:00
huzheng 1e36a84afc HBASE-19622 Reimplement ReplicationPeers with the new replication storage interface 2018-03-09 20:55:48 +08:00
zhangduo 4c6942df58 HBASE-19635 Introduce a thread at RS side to call reportProcedureDone 2018-03-09 20:55:48 +08:00
zhangduo d9b5eb3abb HBASE-19617 Remove ReplicationQueues, use ReplicationQueueStorage directly 2018-03-09 20:55:48 +08:00
zhangduo b40c426806 HBASE-19642 Fix locking for peer modification procedure 2018-03-09 20:55:48 +08:00
zhangduo ac07e07988 HBASE-19592 Add UTs to test retry on update zk failure 2018-03-09 20:55:48 +08:00
Guanghao Zhang 712b5a80dc HBASE-19630 Add peer cluster key check when add new replication peer
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
Guanghao Zhang eae251d203 HBASE-19573 Rewrite ReplicationPeer with the new replication storage interface 2018-03-09 20:55:48 +08:00
zhangduo 15055a421b HBASE-19599 Remove ReplicationQueuesClient, use ReplicationQueueStorage directly 2018-03-09 20:55:48 +08:00
zhangduo 1de08ded58 HBASE-19543 Abstract a replication storage interface to extract the zk specific code 2018-03-09 20:55:48 +08:00
huzheng 3fd417600e HBASE-19525 RS side changes for moving peer modification from zk watcher to procedure 2018-03-09 20:55:48 +08:00
zhangduo da07870995 HBASE-19580 Use slf4j instead of commons-logging in new, just-added Peer Procedure classes 2018-03-09 20:55:48 +08:00
Guanghao Zhang 9ead5934c0 HBASE-19520 Add UTs for the new lock type PEER
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
zhangduo 7c2a0d7e21 HBASE-19564 Procedure id is missing in the response of peer related operations 2018-03-09 20:55:48 +08:00
zhangduo 5e410d8140 HBASE-19524 Master side changes for moving peer modification from zk watcher to procedure 2018-03-09 20:55:48 +08:00
zhangduo 95af14fea6 HBASE-19216 Implement a general framework to execute remote procedure on RS 2018-03-09 20:55:48 +08:00
zhangduo 3576eb6bd8 HBASE-19598 Fix TestAssignmentManagerMetrics flaky test 2018-03-09 20:31:45 +08:00
zhangduo b4e297f3f6 HBASE-20160 TestRestartCluster.testRetainAssignmentOnRestart uses the wrong condition to decide whether the assignment is finished 2018-03-09 11:08:57 +08:00
haxiaolin 78c02c5081 HBASE-20114 Fix IllegalFormatConversionException in rsgroup.jsp
Repplication. Was reverted earlier.
2018-03-08 09:29:01 -08:00
Sean Busbey 71cc7869db HBASE-20155 update branch-2 version to 2.1.0-SNAPSHOT
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2018-03-08 08:44:30 -08:00
zhangduo 25efd37f32 HBASE-20144 The shutdown of master will hang if there are no live region server 2018-03-08 15:54:40 +08:00
Michael Stack 96a42b7359 Revert "HBASE-20137 TestRSGroups is flakey"
Revert. Fix is not right.

This reverts commit 6d1740d498.
2018-03-07 09:25:02 -08:00
Michael Stack 6d1740d498 HBASE-20137 TestRSGroups is flakey
On failed RPC we expire the server and suspend expecting the
resultant ServerCrashProcedure to wake us back up again. In tests,
TestRSGroup hung because it failed to schedule a server expiration
because the server was already expired undergoing processing (the
test was shutting down). Deal with this case by having expire
servers return false if unable to expire. Callers will then know
where a ServerCrashProcedure has been scheduled or not.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
  Have expireServer return true if successful.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
 The log that included an exception whose message was the current
procedure as a String totally baffled me. Make it more obvious what
exception is.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
 If failed expire of a server, wake our procedure -- do not suspend --
and presume ok to move region to CLOSED state (because going down or
concurrent crashed server processing ongoing).
2018-03-05 22:53:27 -08:00
tedyu 9e9b347d66 HBASE-20114 Fix IllegalFormatConversionException in rsgroup.jsp - revert premature commit 2018-03-02 12:07:16 -08:00
haxiaolin 4ec2eeef63 HBASE-20114 Fix IllegalFormatConversionException in rsgroup.jsp
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-02 09:01:05 -08:00
Sean Busbey 9927c2e14a HBASE-20070 refactor website generation
* rely on git plumbing commands when checking if we've built the site for a particular commit already
* switch to forcing '-e' for bash
* add command line switches for: path to hbase, working directory, and publishing
* only export JAVA/MAVEN HOME if they aren't already set.
* add some docs about assumptions
* Update javadoc plugin to consistently be version 3.0.0
* avoid duplicative site invocations on reactor modules
* update use of cp command so it works both on linux and mac
* manually skip enforcer plugin during build
* still doing install of all jars due to MJAVADOC-490, but then skip rebuilding during aggregate reports.
* avoid the pager on git-diff by teeing to a log file, which also helps later reviewing in the case of big changesets.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>

 Conflicts:
	hbase-backup/pom.xml
	hbase-spark-it/pom.xml
2018-03-02 09:51:43 -06:00
Chia-Ping Tsai e06e90bdbc HBASE-19437 Batch operation can't handle the null result for Append/Increment
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-02 23:42:41 +08:00
Michael Stack a2de29560f HBASE-20113 Move branch-2 version from 2.0.0-beta-2-SNAPSHOT to 2.0.0-beta-2 2018-03-01 15:46:38 -08:00
Umesh Agashe 4547edbfe0 HBASE-20055 Removed declaration of un-thrown exceptions and unused setRegionStateBackToOpen() from MergeTableRegionsProcedure
Plus some minor cleanup.
2018-03-01 08:45:54 -08:00
Chia-Ping Tsai a2bf2ad71e HBASE-20093 (addendum) remove unused import of ServerLoad
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-01 23:44:29 +08:00
tedyu e0ff595744 HBASE-20107 Add a test case for HBASE-14317 (Zephyr Guo) 2018-02-28 21:18:14 -08:00
Sergey Soldatov 313464f007 HBASE-19863 java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
Signed-off-by: ramkrish86 <ramkrishna.s.vasudevan@gmail.com>
2018-02-28 14:06:17 -05:00
Chia-Ping Tsai f71c00c02e HBASE-20097 Merge TableDescriptors#getAll and TableDescriptors#getAllDescriptors into one
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-28 15:21:31 +08:00
Chia-Ping Tsai e50e6f7ce9 HBASE-20084 Refactor the RSRpcServices#doBatchOp
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-28 15:13:09 +08:00
Chia-Ping Tsai c459282fe0 HBASE-20093 Replace ServerLoad by ServerMetrics for ServerManager
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-28 15:05:52 +08:00
Michael Stack 5317ca92bc HBASE-20069 fix existing findbugs errors in hbase-server; ADDENDUM Address review
Revert of the revert, i.e. reapply. Thought this had broken the build
but it was a bad nightly. Putting it back.

Revert "Revert "HBASE-20069 fix existing findbugs errors in hbase-server; ADDENDUM Address review""

This reverts commit 07eae00ec1.
2018-02-27 21:07:38 -08:00
Michael Stack 2ceafe108f HBASE-20092 Fix TestRegionMetrics#testRegionMetrics
Reapplication of a patch temporarily removed...
I thought this was causing issue but now I don't think it the culprit.

Revert "Revert "HBASE-20092 Fix TestRegionMetrics#testRegionMetrics""

This reverts commit 367d316781.
2018-02-27 15:42:17 -08:00
Michael Stack 7d7ca33e6d HBASE-20100 TestEnableTableProcedure flakey
Allow OPEN as a possible state when update region transition state.
Usually state is OPENING but if crash before finish step is completed,
on replay, master may have read that the state is OPEN from meta table
and so will think it open... When we replay the procedure finish, allow
that the region is already OPEN.
2018-02-27 12:10:51 -08:00
gsheffi 06cf04e458 HBASE-20074 [FindBugs] Same code on both branches in CompactingMemStore#initMemStoreCompactor
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-27 11:23:16 -06:00
Sean Busbey e0f8076693 Revert "for creating patch HBASE-20074-V01.patch"
This reverts commit a312705dbc.

bad commit message
2018-02-27 11:22:59 -06:00
Michael Stack 07eae00ec1 Revert "HBASE-20069 fix existing findbugs errors in hbase-server; ADDENDUM Address review"
This reverts commit 43d661b008.
2018-02-27 08:44:22 -08:00
Michael Stack 367d316781 Revert "HBASE-20092 Fix TestRegionMetrics#testRegionMetrics"
This reverts commit ae288a7b41.
2018-02-27 08:26:47 -08:00
Chia-Ping Tsai ae288a7b41 HBASE-20092 Fix TestRegionMetrics#testRegionMetrics
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-27 16:48:48 +08:00
zhangduo 8e8e50683d HBASE-20066 Region sequence id may go backward after split or merge 2018-02-27 15:37:32 +08:00
Michael Stack 43d661b008 HBASE-20069 fix existing findbugs errors in hbase-server; ADDENDUM Address review 2018-02-26 23:15:43 -08:00
Michael Stack 44544c7db0 HBASE-20069 fix existing findbugs errors in hbase-server 2018-02-26 10:55:53 -08:00
gsheffi a312705dbc for creating patch HBASE-20074-V01.patch
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-26 09:56:44 -08:00
Vasudevan 372c68fca2 HBASE-20036 TestAvoidCellReferencesIntoShippedBlocks timed out (Ram) 2018-02-26 22:06:28 +05:30
zhangduo ef02762dd8 HBASE-20083 Fix findbugs error for ReplicationSyncUp 2018-02-26 22:16:37 +08:00
Michael Stack 8b3ae58e18 HBASE-20043 ITBLL fails against hadoop3
Fix MoveRandomRegionOfTableAction. It depended on old AM behavior.
Make it do explicit move as is required in AMv3; w/o it, it was just
closing region causing test to fail.

Fix pom so hadoop3 profile specifies a different netty3 version.

Bunch of logging format change that came of trying trying to read
the spew from this test.
2018-02-24 17:29:24 -08:00
zhangduo 2d5065bc7e HBASE-20065 Revisit the timestamp usage in MetaTableAccessor 2018-02-25 09:14:09 +08:00
Sean Busbey 62b8da927e HBASE-20061 HStore synchronized member variable filesCompacting should be private
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-24 10:58:14 -06:00
Sean Busbey a66fa85a16 Revert "HBASE-19835 Use explicit casting to avoid side effects"
This reverts commit bc080e7500.

 Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Segment.java

patch reverted changes that happened in parallel without explanation. see jira.
2018-02-23 23:04:34 -06:00
tedyu d6d1e49a75 HBASE-20054 removes the indefinite retry introduced by HBASE-18282 2018-02-23 19:14:04 -08:00
zhangduo 4ddfdaffdc HBASE-20048 Revert serial replication feature 2018-02-23 13:54:10 +08:00
tedyu 8a22e4119f HBASE-20049 Region replicas of SPLIT and MERGED regions are kept in in-memory states until restarting master (Toshihiro Suzuki) 2018-02-22 20:11:11 -08:00
Apekshit Sharma 991e163cc2 HBASE-19401 Add missing security checks in RSRpcServices 2018-02-22 16:23:47 -08:00
Apekshit Sharma 161f9de8e5 HBASE-19400 Add missing security checks in MasterRpcServices
- Added ADMIN permission check for following rpc calls:
  normalize, setNormalizerRunning, runCatalogScan, enableCatalogJanitor, runCleanerChore,
  setCleanerChoreRunning, execMasterService, execProcedure, execProcedureWithRet
- Moved authorizationEnabled check to start of AccessChecker's functions. Currently, and IDK why,
  we call authManager.authorize() first and then discard its result if authorizationEnabled is false. Weird.
2018-02-22 16:23:43 -08:00
tedyu f3753e394a HBASE-20052 TestRegionOpen#testNonExistentRegionReplica fails due to NPE 2018-02-22 12:35:50 -08:00
Umesh Agashe 6c039383b9 HBASE-19767 Fix for Master web UI shows negative values for Remaining KVs
Negative Remaining KVs and progress percent greater than 100 is because CompactionProgress#totalCompactingKVs is sometimes less than CompactionProgress#currentCompactedKVs.
Changes add a getter to CompactionProgress#totalCompactingKVs and from inside getter warning is logged. currentCompactedKVs are return when totalCompactingKVs are less than current.

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-22 10:09:47 -08:00
Michael Stack 1db1474591 HBASE-20044 TestClientClusterStatus is flakey"
Revert "Revert "HBASE-2004 TestClientClusterStatus is flakey""
This is a revert of a revert, i.e. a reapplication, just so I can fix
the JIRA number.

This reverts commit 6796b8e21f.
2018-02-22 09:42:37 -08:00
Michael Stack 6796b8e21f Revert "HBASE-2004 TestClientClusterStatus is flakey"
Revert to fix JIRA number; should be 20044.

This reverts commit 92bb4db9ef.
2018-02-22 09:41:27 -08:00
Michael Stack 38a97b87e6 HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory; ADDEDNDUM to fix TestRegionOpen failure 2018-02-22 09:14:35 -08:00
Josh Elser 9d3eadd924 HBASE-20035 Stabilize the flaky TestQuotaStatusRPCs
The test will fail if the quota moves to violation before
the second half of the test.

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-22 09:34:05 -05:00
zhangduo 0432d49d83 HBASE-20038 TestLockProcedure.testTimeout is flakey 2018-02-21 17:23:49 -08:00
Esteban Gutierrez c06c4f30d1 HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory 2018-02-21 16:34:53 -08:00
Michael Stack c677c30020 HBASE-20042 TestRegionServerAbort flakey; ADDENDUM, RETRY 2018-02-21 15:26:59 -08:00
Michael Stack 92bb4db9ef HBASE-2004 TestClientClusterStatus is flakey 2018-02-21 14:52:10 -08:00
Michael Stack baec532aa2 HBASE-20042 TestRegionServerAbort flakey 2018-02-21 13:24:39 -08:00
Andrew Purtell 67b7ce38fb HBASE-20027 Add test TestClusterPortAssignment
LocalHBaseCluster forces random port assignment for sake of concurrent unit test
execution friendliness, but we still need a positive test for RPC and info port
assignment.
2018-02-21 13:12:30 -08:00
tedyu 5c988c9960 HBASE-19166 AsyncProtobufLogWriter persists ProtobufLogWriter as class name for backward compatibility 2018-02-21 13:05:54 -08:00
Mike Drob d8ec10e20f HBASE-20039 MR tests out to hbase-mapreduce mobile 2018-02-21 14:53:30 -06:00
Josh Elser e10db9d29a HBASE-19953 Ensure post DDL hooks are only called after successful operations
The 1.x functionality of Master DDL operations is that "post" observer hooks
are only invoked when the DDL action was successful. With the async-ness of
ProcV2, we find ourselves in a case where the post-hook may be invoked before
the Procedure runs and fails. We need to introduce some blocking to wait and
see if the Procedure is going to fail on a precondition before invoking the hook.

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-21 13:53:09 -05:00
zhangduo 30c2dcd883
HBASE-20037 Race when calling SequenceIdAccounting.resetHighest 2018-02-21 10:04:33 -08:00
anastas 2b4df5e36e HBASE-19506: The CellChunkMap index chunks are usually small, so in order to prevent memory underutilization, HBASE-19506 presents small chunks preallocated in a small pool 2018-02-21 10:19:49 +02:00
Chia-Ping Tsai bc1ac49de2 HBASE-20017 BufferedMutatorImpl submit the same mutation repeatedly
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-20 16:59:48 -08:00
Ted Yu cbd138d41a HBASE-19954 Separate TestBlockReorder into individual tests to avoid ShutdownHook suppression error against hadoop3 2018-02-20 14:56:46 -08:00
Michael Stack 7e2978787d HBASE-20029 @Ignore TestQuotaThrottle and TestReplicasClient#testCancelOfMultiGet 2018-02-20 12:08:35 -08:00
Michael Stack 9be0360c5d HBASE-20024 TestMergeTableRegionsProcedure is STILL flakey 2018-02-20 11:07:36 -08:00
zhangduo 0152d5e2ae HBASE-20021 TestFromClientSideWithCoprocessor is flakey 2018-02-20 06:24:22 -08:00
zhangduo 34d3e847cc HBASE-20020 Make sure we throw DoNotRetryIOException when ConnectionImplementation is closed 2018-02-20 06:10:45 -08:00
zhangduo c1fe9f441c HBASE-19978 The keepalive logic is incomplete in ProcedureExecutor 2018-02-19 17:13:16 -08:00
Reid Chan 4ef6319af0 HBASE-19950 Introduce a ColumnValueFilter
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-20 05:05:19 +08:00
eshcar a458d7c400 HBASE-18294 Reduce global heap pressure: flush based on heap occupancy 2018-02-18 12:58:52 +02:00
Michael Stack 0082f55595 HBASE-20015 TestMergeTableRegionsProcedure and TestRegionMergeTransactionOnCluster flakey 2018-02-17 11:37:42 -08:00
Michael Stack 07bdd492f4 Revert "HBASE-20015 TestMergeTableRegionsProcedure and TestRegionMergeTransactionOnCluster flakey"
This reverts commit 72193650dd.
2018-02-16 23:21:56 -08:00
Michael Stack 3ea2ffcd57 HBASE-20013 TestZKPermissionWatcher is flakey 2018-02-16 23:19:07 -08:00
Michael Stack 72193650dd HBASE-20015 TestMergeTableRegionsProcedure and TestRegionMergeTransactionOnCluster flakey 2018-02-16 22:59:32 -08:00
Chia-Ping Tsai 34d5f2b70e HBASE-19680 BufferedMutatorImpl#mutate should wait the result from AP in order to throw the failed mutations 2018-02-17 07:27:23 +08:00
Michael Stack d59959d19e HBASE-20011 Disable TestRestoreSnapshotFromClientWithRegionReplicas; it is flakey. Needs attention. 2018-02-16 14:45:55 -08:00
Mike Drob 138f82c8c5 HBASE-19920 Lazy init for ProtobufUtil classloader 2018-02-16 09:11:45 -06:00
Umesh Agashe 70d3413ee2 HBASE-19988 Fixed chatty log from HRegion#lockRowsAndBuildMiniBatch() when interrupted while waiting on a row lock 2018-02-16 07:01:23 -08:00
Michael Stack 9f27fdaffc HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey
Remove assert in splittableregionprocedure. It was in the prepare.
Was causing fail in legit case where a region split follows a
table split BEFORE the parent has been GC'd. The region split
finds the parent in SPLIT state which is right. The assert was
having us fail. No need.

Also disabled TestHTrace since not supported in 2.0.0 and flakey.
2018-02-15 20:44:39 -08:00
Michael Stack 13bc4fe33c
HBASE-19998 Flakey TestVisibilityLabelsWithDefaultVisLabelService
Only call server.checkIfShouldMoveSystemRegionAsync if a node has been
added. Do not call it if only one regionserver in cluster. Make it
so ServerCrashProcedure runs before it. Add logging if
server.checkIfShouldMoveSystemRegionAsync was responsible for
MOVE (Previous was a mystery when it cut in).

Previous we'd call it when there was a nodeChildrenChanged. These
happen before nodeDeleted. If a server crashed,
checkIfShouldMoveSystemRegionAsync could run first, find the
server that had not yet registered as crashed, find system
tables on it and then try to move them. It would fail because
server would not respond to RPC. The region move would then
be waiting on the servercrashprocedure to wake it up when
done processing but this move had locked the region so
SCP couldn't run....
2018-02-15 19:41:17 -08:00
Thiruvel Thirumoolan 65ac99b675 HBASE-19996 Some nonce procs might not be cleaned up (follow up HBASE-19756)
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-15 04:48:24 -08:00
anastas af804a586b HBASE-19930: Fixing the bug, in the rare case when there is a merge into CCM and the one of the old segments has a big cell allocated on-heap and it needs to be copied to the MSLAB 2018-02-15 14:35:41 +02:00
zhangduo ce5f5e6bcd HBASE-20000 Remove the quantum logic in FairQueue, always put high priority queue in front 2018-02-15 16:40:21 +08:00
Ben Lau 0743bda059 HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of a KeeperException
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-14 17:23:09 -08:00
Toshihiro Suzuki 9cbf936f90 HBASE-19980 NullPointerException when restoring a snapshot after splitting a region
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-14 09:38:16 -08:00
Michael Stack 8a3b4cdc67 HBASE-19116 Currently the tail of hfiles with CellComparator* classname makes it so hbase1 can't open hbase2 written hfiles; fix
Serializing, if appropriate, write the hbase-1.x version of the
Comparator to the hfile trailer so hbase-1.x files can read hbase-2.x
hfiles (they are the same format).
2018-02-14 07:46:05 -08:00
Michael Stack 02bba3b788 HBASE-19998 Flakey TestVisibilityLabelsWithDefaultVisLabelService 2018-02-13 22:31:57 -08:00
Michael Stack 1f3c131371 HBASE-19965 Fix flaky TestAsyncRegionAdminApi; ADDENDUM do same to TestAsyncTableAdminAPI AGAIN (make TestAsyncTableAdminAPI3... already an TestAsyncTableAdminAPI2) 2018-02-13 21:05:34 -08:00
Pankaj Kumar 4594f7156d HBASE-19979 ReplicationSyncUp tool may leak Zookeeper connection 2018-02-13 08:43:52 -08:00
Michael Stack 562402ec23 Revert "HBASE-19970 Remove unused functions from TableAuthManager."
This reverts commit b19531f04b.
2018-02-13 06:18:35 -08:00
Chia-Ping Tsai 6923472f75 HBASE-19876 The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner 2018-02-13 21:20:21 +08:00
Vasudevan 3623089cba HBASE-19977 FileMmapEngine allocation of byte buffers should be
synchronized (Ram)
2018-02-13 15:51:21 +05:30
Michael Stack 5b95ea01da HBASE-19965 Fix flaky TestAsyncRegionAdminApi; ADDENDUM do same to TestAsyncTableAdminAPI 2018-02-12 22:53:29 -08:00
Michael Stack 7104364a4d HBASE-19965 Fix flaky TestAsyncRegionAdminApi 2018-02-12 22:24:50 -08:00
Apekshit Sharma b19531f04b HBASE-19970 Remove unused functions from TableAuthManager.
Functions deleted: setTableUserPermissions, setTableGroupPermissions, setNamespaceUserPermissions,
setNamespaceGroupPermissions, writeTableToZooKeeper, writeNamespaceToZooKeeper
2018-02-12 16:02:48 -08:00
Michael Stack e94f082776 HBASE-19986 If HBaseTestClassRule timesout a test, thread dump 2018-02-12 15:13:42 -08:00
Michael Stack 1ae64ccee0 HBASE-19972 Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink; ADDENDUM to fix TestReplicationSink 2018-02-12 10:48:21 -08:00
Michael Stack c7473df2c3 HBASE-16060 1.x clients cannot access table state talking to 2.0 cluster
This patch adds mirroring of table state out to zookeeper. HBase-1.x
clients look for table state in zookeeper, not in hbase:meta where
hbase-2.x maintains table state.

The patch also moves and refactors the 'migration' code that was put in
place by HBASE-13032.

D hbase-client/src/main/java/org/apache/hadoop/hbase/CoordinatedStateException.java
 Unused.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Move table state migration code from Master startup out to
TableStateManager where it belongs. Also start
MirroringTableStateManager dependent on config.

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MirroringTableStateManager.java

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
 Move migration from zookeeper of table state in here. Also plumb in
mechanism so subclass can get a chance to look at table state as we do
the startup fixup full-table scan of meta.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Bug-fix. Now we create regions in CLOSED state but we fail to check
table state; were presuming table always enabled. Meant on startup
there'd be an unassigned region that never got assigned.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMirroringTableStateManager.java
 Test migration and mirroring.
2018-02-12 08:22:14 -08:00
zhangduo 43f507dfa8 HBASE-19554 Dump thread info before failure 2018-02-12 16:06:50 +08:00
huzheng d793db7c7d HBASE-19972 Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink 2018-02-12 15:15:08 +08:00
zhangduo 3f127063ff HBASE-19966 The WriteEntry for WALKey maybe null if we failed to call WAL.append 2018-02-09 19:11:08 -08:00
Guanghao Zhang 4461cb7d7a HBASE-19942 (addendum) Fix flaky TestSimpleRpcScheduler 2018-02-09 13:38:05 +08:00
Aman Poonia bc080e7500 HBASE-19835 Use explicit casting to avoid side effects
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-08 12:33:08 -08:00
Michael Stack c93ba37dfc HBASE-19956 Remove category as a consideration timing out tests; set all test to timeout at 10minutes regardless 2018-02-07 20:10:36 -08:00
Michael Stack 8f1e01b6e5 HBASE-19951 Cleanup the explicit timeout value for test method 2018-02-07 16:39:54 -08:00
Mike Drob 7d449892af HBASE-19947 ITU should overwrite HTU local FS 2018-02-07 16:56:11 -06:00
zhangduo e71b0bf19d HBASE-19929 Call RS.stop on a session expired RS may hang 2018-02-07 15:53:08 +08:00
Chia-Ping Tsai e2636c81f7 HBASE-19900 Region-level exception destroy the result of batch 2018-02-07 08:38:04 +08:00
Guanghao Zhang f481386e3e HBASE-19942 Fix flaky TestSimpleRpcScheduler 2018-02-07 08:00:01 +08:00
Balazs Meszaros 4d24b19822 HBASE-19886 Display maintenance mode in shell, web UI 2018-02-06 14:53:48 -08:00
Mike Drob f155da3791 HBASE-19946 More test clean up for local-mode HTU 2018-02-06 15:39:17 -06:00
Michael Stack 89baad5bbe HBASE-19940 TestMetaShutdownHandler flakey; ADDENDUM: yet more debug 2018-02-06 09:48:26 -08:00
Michael Stack ca6b37d63c HBASE-19940 TestMetaShutdownHandler flakey; ADDENDUM: more debug 2018-02-06 06:35:02 -08:00
Guanghao Zhang fb480775d5 HBASE-19944 Fix timeout TestVisibilityLabelsWithCustomVisLabService
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-02-06 20:30:16 +08:00
Umesh Agashe e4aeb46177 HBASE-19941 Flaky TestCreateTableProcedure times out in nightly, needs to LargeTests 2018-02-05 22:11:53 -08:00
tedyu b9175680b6 HBASE-19934 HBaseSnapshotException when read replicas is enabled and online snapshot is taken after region splitting (Toshihiro Suzuki) 2018-02-05 21:06:20 -08:00
Umesh Agashe 53e5707224
HBASE-19939 Fixed NPE in tests TestSplitTableRegionProcedure#testSplitWithoutPONR() and testRecoveryAndDoubleExecution()
Value of 'htd' is null as it is initialized in the constructor but when the object is deserialized its null. Got rid of member variable htd and made it local to method.
2018-02-05 20:48:20 -08:00
zhangduo 8b6b3326a6 HBASE-19927 Addendum join on RegionServerThread instead of HRegionServer 2018-02-06 10:01:41 +08:00
Michael Stack 7723a3d60d HBASE-19940 TestMetaShutdownHandler flakey 2018-02-05 12:38:51 -08:00
Michael Stack a1600c1490 HBASE-19840 Flakey TestMetaWithReplicas; ADDENDUM Adding debug 2018-02-05 11:00:46 -08:00
Michael Stack 29016bc100 HBASE-19837 Flakey TestRegionLoad; ADDENDUM Report more often and wait less time on change (also add some debug on TestMetaShutdown test) 2018-02-05 08:39:52 -08:00
Rajeshbabu Chintaguntla f0a5f12d97 HBASE-19703 Functionality added as part of HBASE-12583 is not working after moving the split code to master
Co-authored-by: Michael Stack <stack@apache.org>

Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-05 23:41:32 +08:00
anastas 514eadbe95 HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress 2018-02-05 11:29:10 +02:00
zhangduo 30a7e4a35c HBASE-19554 reenable TestDLSAsyncFSWAL/TestDLSFSHLog for debugging 2018-02-05 08:58:30 +08:00
zhangduo 054acca866 HBASE-19927 TestFullLogReconstruction flakey 2018-02-05 08:41:37 +08:00
Michael Stack 32d1903377 HBASE-19931 TestMetaWithReplicas failing 100% of the time in testHBaseFsckWithMetaReplicas 2018-02-04 14:37:13 -08:00
Ashish Singhi 2d5b36d194 HBASE-19905 ReplicationSyncUp tool will not exit if a peer replication is disabled
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-02-04 18:12:46 +05:30
zhangduo 3b603d2c08 HBASE-19926 Use a separated class to implement the WALActionListener for Replication 2018-02-04 20:32:14 +08:00
zhangduo cb138c2da8 HBASE-19914 Refactor TestVisibilityLabelsOnNewVersionBehaviorTable 2018-02-04 13:37:09 +08:00
Michael Stack a3233572ac HBASE-19928 TestVisibilityLabelsOnNewVersionBehaviorTable fails 2018-02-03 10:17:16 -08:00
Michael Stack 00653a4d18 HBASE-19919 Tidying up logging; ADDENDUM Fix tests w/ mocked Servers 2018-02-03 09:25:45 -08:00
zhangduo 66a11f2ac0 HBASE-19904 Break dependency of WAL constructor on Replication 2018-02-03 17:11:30 +08:00
Michael Stack bac4687345 HBASE-19919 Tidying up logging 2018-02-02 22:42:30 -08:00
Michael Stack 897afb7d18
HBASE-19726 Failed to start HMaster due to infinite retrying on meta assign 2018-02-02 18:30:24 -08:00
Guanghao Zhang 3398637bcd HBASE-19918 Promote TestAsyncClusterAdminApi to LargeTests 2018-02-03 09:40:58 +08:00
Chia-Ping Tsai b0bf6f504e HBASE-19720 Rename WALKey#getTabnename to WALKey#getTableName 2018-02-02 20:02:04 +08:00
tedyu efed668f74 HBASE-19895 Add keepDeletedCells option in ScanOptions for customizing scanInfo in pre-hooks (Ankit Singhal) 2018-02-01 18:28:05 -08:00
Mike Drob a165bd766a HBASE-19841 Every HTU should be local until DFS starts 2018-02-01 20:23:56 -06:00
zhangduo 79ccb0b38e HBASE-19913 Split TestStochasticLoadBalancer2 2018-02-02 08:29:33 +08:00
Michael Stack ea8290f3dc HBASE-19147 TestCacheOnWrite Times Out 2018-02-01 16:15:50 -08:00
Chia-Ping Tsai 6a1ed25581 HBASE-19897 RowMutations should follow the fluent pattern 2018-02-02 05:24:08 +08:00
Peter Somogyi 7fe215db5f HBASE-19884 BucketEntryGroup's equals, hashCode and compareTo methods are not consistent
Move back to default equals and hashCode.
Remove compareTo and Comparator to PriorityQueue.

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-01 13:21:23 -08:00
kewang 86ca65218d HBASE-19912 Remove useless 'writeToWAL' flag of Region#checkAndRowMutate
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-01 12:34:46 -08:00
Umesh Agashe 4cdc13b86e HBASE-19839 Fixed flakey tests TestMergeTableRegionsProcedure#testRollbackAndDoubleExecution and TestSplitTableRegionProcedure#testRollbackAndDoubleExecution
* Added a comment in MergeTableRegionsProcedure and SplitTableRegionProcedure explaining specific rollbacks has side effect that AssignProcedure/s are submitted asynchronously and those procedures may continue to execute even after rollback() is done.
* Updated comments in tests with correct rollback state to abort
* Added overloaded method MasterProcedureTestingUtility#testRollbackAndDoubleExecution which takes additional argument for waiting on all procedures to finish before asserting conditions
* Updated TestMergeTableRegionsProcedure#testRollbackAndDoubleExecution and TestSplitTableRegionProcedure#testRollbackAndDoubleExecution to use newly added method

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-01 12:01:57 -08:00
Michael Stack 23471deb75 HBASE-19906 TestZooKeeper Timeout Includes move of TestQoSFunction from smalltests to mediumtests; it timedout in a run. 2018-02-01 10:53:51 -08:00
Michael Stack 4c210eb212 HBASE-19911 Convert some tests from small to medium because they are timing out: TestNettyRpcServer, TestClientClusterStatus; ADDENDUM changes TestCheckTestClasses to be medium sized test 2018-01-31 20:28:02 -08:00
Michael Stack ba265a7e51 HBASE-19911 Convert some tests from small to medium because they are timing out: TestNettyRpcServer, TestClientClusterStatus 2018-01-31 20:21:57 -08:00
Michael Stack 113432c9b9 HBASE-19910 TestBucketCache TimesOut 2018-01-31 18:17:22 -08:00
Michael Stack faea89ea4f HBASE-19909 TestRegionLocationFinder Timeout 2018-01-31 17:53:48 -08:00
Michael Stack 13f92b7daa HBASE-19908 TestCoprocessorShortCircuitRPC Timeout.... 2018-01-31 17:34:05 -08:00
Michael Stack 7a82126f8b Revert "HBASE-19906 TestZooKeeper Timeout"
Revert. Pushed prematurely.

This reverts commit 80af9bdef0.
2018-01-31 15:56:26 -08:00
Michael Stack e6d2682f0f HBASE-19907 TestMetaWithReplicas still flakey 2018-01-31 15:18:45 -08:00
Michael Stack 80af9bdef0 HBASE-19906 TestZooKeeper Timeout 2018-01-31 15:18:38 -08:00
Rahul Gidwani 4148d7c614 HBASE-19528 - Major Compaction Tool 2018-01-31 14:37:48 -08:00
zhangduo 28d13c54a9 HBASE-19887 Do not overwrite the surefire junit listener property in the pom of sub modules 2018-01-31 20:49:25 +08:00
binlijin b3824b8c92 HBASE-19728 Add lock to filesCompacting in all place. 2018-01-31 14:18:27 +08:00
Peter Somogyi 0ee280db22
HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey
Recategorize TestCoprocessorWhitelistMasterObserver as Large test
2018-01-30 06:01:50 -08:00
Michael Stack 364650bad5 HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey 2018-01-29 21:33:16 -08:00
Michael Stack 90a75fb052 HBASE-19888 Move branch-2 version from 2.0.0-beta-1 to 2.0.0-beta-2-SNAPSHOT 2018-01-29 14:17:54 -08:00
Peter Somogyi 27d73dec8a
HBASE-19811 Fix findbugs and error-prone warnings in hbase-server (branch-2) - addendum
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-29 14:02:42 -08:00
Duo Zhang bbf3bae72a
HBASE-19873 Add a CategoryBasedTimeout ClassRule for all UTs 2018-01-29 12:41:14 -08:00
Michael Stack 7cbb70945a
HBASE-19840 Flakey TestMetaWithReplicas; ADDENDUM to fix Checksyte 2018-01-29 09:35:52 -08:00
Michael Stack 77607e4961
HBASE-19840 Flakey TestMetaWithReplicas
Fix two issues:

 # Meta Replicas can all be assigned to the same server. This
 will call the test to hang when we do our kill of the server
 hosting meta because there'll be no replicas to read from
 as test intends. Check is to look for this condition on
 startup and adjust if we come across it. Replicas cross-cut
 assignment. They need work.
 # Other issue was shutdown. The master started toward the
 end of the test may not have come up fully by the time
 shutdown is called. We could be stuck assigning the
 meta replicas. Have shutdown shutdown the procedure
 executor engine.

There is other cleanup and notes in the below.

M HMaster
 Remove the silly stops in startup now we have real
 means of shutting down Master during init.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
 This replica stuff was doing stuff it shouldn't be doing
 like setting core Master state flags. It may have made
 sense once but now meta is assigned by a Pv2 Procedure
 so the flag setting in here is meddlesome. Clear out
 methods no longer needed.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Remove unused methods.
 Changes local variable names so they align w/ our naming elsewhere in
 code base.

M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java
 Check for all replicas on the one server.
2018-01-29 09:26:41 -08:00
zhangduo 1efa050ebf HBASE-19885 Promote TestAssignmentManager to LargeTests 2018-01-30 00:25:32 +08:00
zhangduo f8bde7b76c HBASE-19866 TestRegionServerReportForDuty doesn't timeout
Synchronized on authManager instead of RpcServer
2018-01-29 21:23:57 +08:00
zhangduo 5b0d7dc45b HBASE-19882 Promote TestProcedureManager to MediumTests 2018-01-29 17:50:29 +08:00
zhangduo cdda0a79ea HBASE-19881 Promote TestRegionReplicaReplicationEndpoint to LargeTests 2018-01-29 16:15:55 +08:00
zhangduo b75c85a075 HBASE-19880 Promote TestFuzzyRowFilterEndToEnd to LargeTests 2018-01-29 16:15:36 +08:00
zhangduo 5ef5c5a247 HBASE-19879 Promote TestAcidGuaranteesXXX to LargeTests 2018-01-29 14:57:31 +08:00