Commit Graph

7016 Commits

Author SHA1 Message Date
Ashish Singhi 5937202fd5 HBASE-16499 slow replication for small HBase clusters
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-04-04 09:54:41 +05:30
Michael Stack b1b0db3195 HBASE-20298 Doc change in read/write/total accounting metrics 2018-04-03 17:00:43 -07:00
Michael Stack c88ca54c16 HBASE-20259 Doc configs for in-memory-compaction and add detail to
in-memory-compaction logging

Adds logging of CompactingMemStore configuration on construction.

Add logging of detail about Store on creation including memstore type.

Add chapter to refguide on new in-memory compaction feature.
2018-04-02 09:53:12 -07:00
zhangduo 1e56938757 HBASE-20296 Remove last pushed sequence ids when removing tables from a peer 2018-04-01 21:14:31 +08:00
Jan Hentschel 97b3a04019 HBASE-17631 Changed Canary interval to 60 seconds 2018-04-01 13:03:52 +02:00
Mike Drob d57001ee2d HBASE-20282 Clean up tooling docs/help 2018-03-30 08:59:47 -07:00
tedyu e60f34fdb5 HBASE-20159 Support using separate ZK quorums for client - addendum fixes flaky TestSeparateClientZKCluster 2018-03-30 07:19:47 -07:00
Vasudevan cbd8b15e6b HBASE-20197 Review of ByteBufferWriterOutputStream.java (BELUGA BEHR)
Signed-off-by: Vasudevan <ramkrishna.s.vasudevan@gmail.com>
2018-03-30 12:13:41 +05:30
Mike Drob 621e2b5f5e HBASE-20273 Additional removed/changed configs 2018-03-29 11:47:14 -07:00
Yu Li 061a31fad1 HBASE-20159 Support using separate ZK quorums for client 2018-03-29 02:37:26 +08:00
Josh Elser 69f5d707b6 HBASE-20199 Add a unit test to verify flush and snapshot permission requirements aren't excessive
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-27 20:17:08 -04:00
Zach York 09ed7c7a10 HBASE-20280 Fix possibility of deadlocking in refreshFileConnections
When prefetch on open is specified, there is a deadlocking case
where if the prefetch is cancelled, the PrefetchExecutor interrupts
the threads if necessary, when that happens in FileIOEngine, it
causes an ClosedByInterruptException which is a subclass of
ClosedChannelException. If we retry all ClosedChannelExceptions,
this will lock as this access is expected to be interrupted.
This change removes calling refreshFileConnections for
ClosedByInterruptExceptions.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-03-27 16:53:01 -07:00
Rajeshbabu Chintaguntla d87139989f HBASE-20111 A region's splittable state now includes the configuration splitPolicy
The Master asks a RegionServer whether a Region can be split or not, primarily to
verify that the region is not closing, opening, etc. This change has the RegionServer
also consult the configured RegionSplitPolicy.

Signed-off-by: Josh Elser <elserj@apache.org>
2018-03-27 18:42:49 -04:00
zhangduo 056c3395d9 HBASE-20285 Delete all last pushed sequence ids when removing a peer or removing the serial flag for a peer 2018-03-27 12:20:51 +08:00
Josh Elser 15c398f7d2 HBASE-20223 Update to hbase-thirdparty 2.1.0
Remove commons-cli and commons-collections4 use. Account
for the newer internal protobuf version of 3.5.1.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Mike Drob <mdrob@apache.org>
2018-03-26 22:05:19 -04:00
Toshihiro Suzuki ff5250ca2b HBASE-20229 ConnectionImplementation.locateRegions() returns duplicated entries when region replication is on 2018-03-26 15:09:20 -07:00
Yuki Tawara 4cedd9914b HBASE-20289 Fix comparator for NormalizationPlan
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-26 14:30:25 -07:00
Reid Chan 83fa0ad9ed HBASE-20095 Redesign single instance pool in CleanerChore 2018-03-26 12:48:31 -05:00
zhangduo e9701a0595 HBASE-20127 Add UT for serial replication after failover 2018-03-26 20:16:02 +08:00
anoopsamjohn 3f7222df36 HBASE-17819 Reduce the heap overhead for BucketCache. 2018-03-25 16:36:30 +05:30
tedyu b50b2e51bf HBASE-20272 TestAsyncTable#testCheckAndMutateWithTimeRange fails due to TableExistsException 2018-03-24 06:27:20 -07:00
zhangduo c44e886860 HBASE-20271 ReplicationSourceWALReader.switched should use the file name instead of the path object directly 2018-03-24 21:12:40 +08:00
Jan Hentschel 64ccd2b295 HBASE-13300 Fixed casing in set/getTimeStamp for Mutations 2018-03-23 17:23:22 -07:00
Sakthi b8f999bf33 HBASE-20135 Fixed NullPointerException during reading bloom filter when upgraded from hbase-1 to hbase-2 2018-03-23 15:21:51 -07:00
Toshihiro Suzuki 88eac3ca18 HBASE-20261 Table page (table.jsp) in Master UI does not show replicaIds for hbase meta table
Signed-off-by: Josh Elser <elserj@apache.org>
2018-03-23 15:31:30 -04:00
Chia-Ping Tsai ad47c2daf4 HBASE-19504 Add TimeRange support into checkAndMutate
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-24 00:12:38 +08:00
zhangduo 64061f896f HBASE-20147 Serial replication will be stuck if we create a table with serial replication but add it to a peer after there are region moves 2018-03-23 14:31:20 +08:00
Umesh Agashe c614b9f3e8 HBASE-20224 Web UI is broken in standalone mode
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
2018-03-22 20:27:39 -07:00
Michael Stack 5d1b2110d1
Revert "HBASE-20224 Web UI is broken in standalone mode"
Broke shell tests.

This reverts commit dd9fe813ec.
2018-03-22 10:57:42 -07:00
Umesh Agashe 4cb40e6d84 HBASE-20224 Web UI is broken in standalone mode
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
2018-03-22 06:52:20 -07:00
Chia-Ping Tsai a6eeb26cc0 HBASE-20212 Make all Public classes have InterfaceAudience category
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-22 18:10:23 +08:00
Peter Somogyi 68b2f5502a HBASE-17449 Add explicit document on different timeout settings 2018-03-22 12:06:22 +05:30
Michael Stack 9601ab2272 HBASE-20237 Put back getClosestRowBefore and throw UnsupportedOperation instead... for asynchbase client Throw exception if an old client connects. 2018-03-21 21:51:25 -07:00
zhangduo d7475b92e8 HBASE-20242 The open sequence number will grow if we fail to open a region after writing the max sequence id file 2018-03-22 08:42:36 +08:00
Andrew Purtell 17ac2fe9c1 HBASE-19024 Provide a configurable option to hsync WAL edits to the disk for better durability (Harshal Jain) 2018-03-20 21:07:59 -07:00
tedyu e7cfdac7b5 HBASE-20090 Properly handle Preconditions check failure in MemStoreFlusher.run 2018-03-20 21:02:09 -07:00
Michael Stack acbdb86bb4 HBASE-20169 NPE when calling HBTU.shutdownMiniCluster
Adds a prepare step to RecoverMetaProcedure in which we test for
cluster up and master being up. If not up, we fail the run.

Modified hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileCleaner.java
Modified hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java
 Minor log cleanup.

Modified hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RecoverMetaProcedure.java
 Add pepare step.

Modified hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerMetrics.java
 Debug for the failing test....

Added hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRecoverMetaProcedure.java
 Test the prepare step goes down if master or cluster are down.
2018-03-20 13:11:57 -07:00
Michael Stack 74c28bdf44 HBASE-20232 [LOGGING] Formatting around close and flush 2018-03-20 10:34:49 -07:00
tedyu 2a3f4a0a4e HBASE-20214 Review of RegionLocationFinder Class - revert due to the pending removal of commons-collections4 dependency 2018-03-19 16:52:27 -07:00
tedyu df5de33a02 HBASE-20196 Maintain all regions with same size in memstore flusher 2018-03-19 08:10:39 -07:00
zhangduo 67f013430c HBASE-20206 WALEntryStream should not switch WAL file silently 2018-03-19 11:40:11 +08:00
zhangduo 00095a2ef9 Revert "HBASE-19665 Add table based replication peers/queues storage back"
This reverts commit 31978c31bb.

 Conflicts:
	hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/TableReplicationStorageBase.java
2018-03-17 20:25:27 +08:00
BELUGA BEHR 104f58701e HBASE-20214 Review of RegionLocationFinder Class 2018-03-16 16:09:40 -07:00
Michael Stack bedf849d83 HBASE-20213 [LOGGING] Aligning formatting and logging less (compactions,
in-memory compactions)

Log less. Log using same format as used elsewhere in log.

Align logs in HFileArchiver with how we format elsewhere. Removed
redundant 'region' qualifiers, tried to tighten up the emissions so
easier to read the long lines.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java
 Add a label for each of the chunkcreators we make (I was confused by
two chunk creater stats emissions in log file -- didn't know that one
was for data and the other index).

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java
 Formatting. Log less.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactionStrategy.java
 Make the emissions in here trace-level. When more than a few regions,
log is filled with this stuff.
2018-03-16 13:16:49 -07:00
Zach York aaa90d8069 [HBASE-20141] Fix TooManyFiles exception when RefreshingChannels
HBASE-19435 implements a fix for reopening file channels when they are unnexpected closed
to avoid disabling the BucketCache. However, it was missed that the the channels might not
actually be completely closed (the write or read channel might still be open
(see https://docs.oracle.com/javase/7/docs/api/java/nio/channels/ClosedChannelException.html)
This commit closes any open channels before creating a new channel.
2018-03-16 10:51:39 -07:00
Michael Stack 13f3ba3cee HBASE-20202 [AMv2] Don't move region if its a split parent or offlined
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/DoNotRetryRegionException.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/exceptions/MergeRegionException.java
 Allow passing cause to Constructor.

M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
 Add prepare step to move procedure.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
 Add check that regions to merge are actually online to the Constructor
so we can fail fast if they are offline

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
 Add prepare step. Check regions and context and skip move if not right.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
 Add check parent region is online to constructor.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java
 Add generic check region is online utility function for use by subclasses.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMove.java
 Add test that we fail if we try to move an offlined region.
2018-03-16 09:35:33 -07:00
Michael Stack c200bf8f78
HBASE-20190 Fix default for MIGRATE_TABLE_STATE_FROM_ZK_KEY 2018-03-15 10:34:02 -07:00
Chia-Ping Tsai 4f2133ee32 HBASE-20119 Introduce a pojo class to carry coprocessor information in order to make TableDescriptorBuilder accept multiple cp at once
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-16 01:21:38 +08:00
Ashish Singhi 82483fad7c HBASE-20146 Addendum Regions are stuck while opening when WAL is disabled
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-03-15 10:13:30 +08:00
Michael Stack d0e55429b3 HBASE-20178 [AMv2] Throw exception if hostile environment
Add Fail-Fast to Procedures by throwing exception out of Procedure
constructor so if move but table is disabled or if master is going
down, etc., we can give notice before the procedure is scheduled.
Will help guard against scheduling Procedures that will have a hard
time succeeding; e.g. a move when table is offline.

Also fixed bug around table state where we presumed ENABLED though no
entry in hbase:meta (we were using this mechanism for whether a table
existed or not).

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMove.java
 Test stolen from HBASE-20131

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableState.java
 Add convenience isEnabled/isDisabled

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Promote assert state to throw exception.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
 Add isClusterUp

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Move constructor now throws exception
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RestoreSnapshotProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java
 Do environment check at construction and fail-fast if hostile.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java
 Add preflightCheck utility method.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Removed setting time setting table state; broke when using other than
 default environment edge masked by presumption that no state meant
 active.
2018-03-14 14:56:22 -07:00
Mike Drob f63a7ff635 HBASE-20180 Avoid Class::newInstance 2018-03-14 13:15:05 -05:00
Apekshit Sharma 84ee32c723 HBASE-20185 Fix ACL check for MasterRpcServices#execProcedure 2018-03-14 19:02:19 +05:30
Chance Li 4f54a66782 HBASE-19389 Limit concurrency of put with dense (hundreds) columns to prevent write handler exhausted
Signed-off-by: Yu Li <liyu@apache.org>
2018-03-14 18:38:33 +08:00
Yu Li 9a26af37b9 Revert "HBASE-19389 Limit concurrency of put with dense (hundreds) columns to prevent write handler exhausted"
This reverts commit 98ac4f12b5.
2018-03-14 18:38:07 +08:00
Yu Li 98ac4f12b5 HBASE-19389 Limit concurrency of put with dense (hundreds) columns to prevent write handler exhausted
Signed-off-by: Yu Li <liyu@apache.org>
2018-03-14 18:25:17 +08:00
huzheng 31978c31bb HBASE-19665 Add table based replication peers/queues storage back 2018-03-14 15:42:16 +08:00
zhangduo b7308ee01c HBASE-20117 Cleanup the unused replication barriers in meta table 2018-03-14 12:08:15 +08:00
Mike Grimes b16e03c130 HBASE-17165 Make use of retry setting in LoadIncrementalHFiles & fix test 2018-03-13 15:00:36 -07:00
Sahil Aggarwal 1b66444846 HBASE-19075: Fix the 'tasks' table on master info page to not scroll up on clicking the tab 2018-03-13 14:26:16 -07:00
Michael Stack 72c3d27bf6 HBASE-20173 [AMv2] DisableTableProcedure concurrent to ServerCrashProcedure can deadlock
Allow that DisableTableProcedue can grab a region lock before
ServerCrashProcedure can. Cater to this cricumstance where SCP
was not unable to make progress by running the search for RIT
against the crashed server a second time, post creation of all
crashed-server assignemnts. The second run will uncover such as
the above DisableTableProcedure unassign and will interrupt its
suspend allowing both procedures to make progress.

M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
 Add new procedure step post-assigns that reruns the RIT finder method.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Make this important log more specific as to what is going on.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
 Better explanation as to what is going on.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java
 Add extra step and run handleRIT a second time after we've queued up
 all SCP assigns. Also fix a but. SCP was adding an assign of a RIT
 that was actually trying to unassign (made the deadlock more likely).
2018-03-13 06:04:36 -07:00
BELUGA BEHR f30dfc69bb HBASE-19449 Minor logging change in HFileArchiver
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-03-12 22:01:56 +05:30
zhangduo 6060d3ba56 HBASE-20167 Optimize the implementation of ReplicationSourceWALReader 2018-03-12 15:14:16 +08:00
Umesh Agashe 45bbee4905 HBASE-20120 Removed unused classes/ java files from hbase-server
deleted:    hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NoOpHeapMemoryTuner.java
deleted:    hbase-server/src/main/java/org/apache/hadoop/hbase/replication/BaseWALEntryFilter.java
deleted:    hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSMapRUtils.java
deleted:    hbase-server/src/main/java/org/apache/hadoop/hbase/util/ProtoUtil.java
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-03-10 13:08:38 +08:00
Guangxu Cheng 70240f9732 HBASE-20132 Change the "KV" to "Cell" for web UI
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-03-10 12:27:59 +08:00
zhangduo dd6f4525e7 HBASE-20148 Make serial replication as a option for a peer instead of a table 2018-03-10 09:04:44 +08:00
Umesh Agashe 974200fca1 HBASE-20024 Fixed flakyness of TestMergeTableRegionsProcedure
We assumed that we can run for loop from 0 to lastStep sequentially. MergeTableRegionProcedure skips step 2. So, when i is 0 the procedure is already at step 3.
Added a method StateMachineProcedure#getCurrentStateId that can be used from test code only.
2018-03-09 12:45:39 -08:00
Ashish Singhi 06550bc93b HBASE-20146 Regions are stuck while opening when WAL is disabled
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 21:10:16 +08:00
zhangduo 756cccecff HBASE-19598 Addendum fix typo 2018-03-09 15:37:22 +08:00
zhangduo 04798d6747 HBASE-19598 Addendum increase sync wait time 2018-03-09 15:34:24 +08:00
zhangduo 033485dff3 HBASE-19598 Fix TestAssignmentManagerMetrics flaky test 2018-03-09 11:47:55 +08:00
zhangduo a513678a79 HBASE-20160 TestRestartCluster.testRetainAssignmentOnRestart uses the wrong condition to decide whether the assignment is finished 2018-03-09 11:08:44 +08:00
zhangduo a03d09abd7 HBASE-20144 The shutdown of master will hang if there are no live region server 2018-03-08 15:05:57 +08:00
Michael Stack 37d91cdfbb Revert "HBASE-20137 TestRSGroups is flakey"
Revert. Fix is not right.

This reverts commit 6d1740d498.
2018-03-07 09:26:21 -08:00
zhangduo 6b77786dfc HBASE-20125 Add UT for serial replication after region split and merge 2018-03-07 14:52:23 +08:00
Michael Stack 1f5e93a8f8 HBASE-20137 TestRSGroups is flakey
On failed RPC we expire the server and suspend expecting the
resultant ServerCrashProcedure to wake us back up again. In tests,
TestRSGroup hung because it failed to schedule a server expiration
because the server was already expired undergoing processing (the
test was shutting down). Deal with this case by having expire
servers return false if unable to expire. Callers will then know
where a ServerCrashProcedure has been scheduled or not.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
  Have expireServer return true if successful.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
 The log that included an exception whose message was the current
procedure as a String totally baffled me. Make it more obvious what
exception is.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
 If failed expire of a server, wake our procedure -- do not suspend --
and presume ok to move region to CLOSED state (because going down or
concurrent crashed server processing ongoing).
2018-03-06 10:55:40 -08:00
zhangduo 1384da7137 HBASE-20129 Add UT for serial replication checker 2018-03-06 16:43:01 +08:00
Josh Elser 4a4c012049 HBASE-18135 Implement mechanism for RegionServers to report file archival for space quotas
This de-couples the snapshot size calculation from the
SpaceQuotaObserverChore into another API which both the periodically
invoked Master chore and the Master service endpoint can invoke. This
allows for multiple sources of snapshot size to reported (from the
multiple sources we have in HBase).

When a file is archived, snapshot sizes can be more quickly realized and
the Master can still perform periodical computations of the total
snapshot size to account for any delayed/missing/lost file archival RPCs.

Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-03-05 17:32:42 -05:00
zhangduo b7b8683925 HBASE-20115 Reimplement serial replication based on the new replication storage layer 2018-03-05 20:25:25 +08:00
haxiaolin 1d25b60831 HBASE-20114 Fix IllegalFormatConversionException in rsgroup.jsp
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-02 09:00:28 -08:00
Chia-Ping Tsai db131be39a HBASE-19437 Batch operation can't handle the null result for Append/Increment
Signed-off-by: anoopsamjohn <anoopsamjohn@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-02 23:31:56 +08:00
Sean Busbey 2a65066b35 HBASE-20070 refactor website generation
* rely on git plumbing commands when checking if we've built the site for a particular commit already
* switch to forcing '-e' for bash
* add command line switches for: path to hbase, working directory, and publishing
* only export JAVA/MAVEN HOME if they aren't already set.
* add some docs about assumptions
* Update javadoc plugin to consistently be version 3.0.0
* avoid duplicative site invocations on reactor modules
* update use of cp command so it works both on linux and mac
* manually skip enforcer plugin during build
* still doing install of all jars due to MJAVADOC-490, but then skip rebuilding during aggregate reports.
* avoid the pager on git-diff by teeing to a log file, which also helps later reviewing in the case of big changesets.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
2018-03-02 09:25:10 -06:00
huzheng 99d3edfc82 HBASE-20050 Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-02 15:53:14 +08:00
Umesh Agashe e86ffedb45 HBASE-20055 Removed declaration of un-thrown exceptions and unused setRegionStateBackToOpen() from MergeTableRegionsProcedure
Plus some minor cleanup.
2018-03-01 08:46:23 -08:00
Chia-Ping Tsai 776eb5d9cb HBASE-20093 (addendum) remove unused import of ServerLoad
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-03-01 23:47:36 +08:00
Michael Stack 0732ef5ebf Revert "HBASE-19598 Fix TestAssignmentManagerMetrics flaky test"
Pushed prematurely. Revert.

This reverts commit 016cf0c64b.
2018-02-28 23:39:54 -08:00
Michael Stack 016cf0c64b HBASE-19598 Fix TestAssignmentManagerMetrics flaky test
Master never left waitForMasterActive because it never checked state of
the clusterUp flag. The test here was aborting regionserver and then
just exiting. The minihbasecluster shutdown sets the cluster down flag
but we were never looking at it so Master thread was staying up.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
The tableOnMaster check in waitForMasterActive looks wrong. It was
making it so a 'normal' Master was getting stuck in here. This is not
the place to worry about tablesOnMaster. That is for the balancer to be
concerned with. There is a problem with Master hosting
system-tables-only. After further study, Master can carry regions like a
regionserver but making it so it carries system tables only is tricky
given meta assign happens ahead of all others which means that the
Master needs to have checked-in as a regionserver super early... It
needs work. Punted for now.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Mostly renaming so lists and maps of region infos have same name as they
have elsewhere in code base and cleaning up confusion that may arise
when we talk of servers-for-system-tables....It is talking about
something else in the code changes here that is other than the normal
understanding. It is about filtering regionservers by their version
numbers so we favor regions with higher version numbers. Needs to go
back up into the balancer.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
It was possible for the Master to be given regions if no regionservers
available (as per the failing unit test in this case).

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Minor reordering moving the waitForMasterActive later in the initialize
and wrapping each test in a check if we are to keep looping (which
checks cluster status flag).

M hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerMetrics.java
This was an old test from the days when Master carried system tables.
Updated test and fixed metrics. Metrics count the hbase:meta along with
the userspace region so upped expected numbers (previously the
hbase:meta was hosted on the master so metrics were not incremented).

M hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRegionsOnMasterOptions.java
I took a look at this test again but nope, needs a load of work still to
make it pass.

M hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
Stop being so whiney.
2018-02-28 22:25:24 -08:00
Michael Stack 372c88abd6
HBASE-19656 Disable TestAssignmentManagerMetrics for beta-1 2018-02-28 21:30:44 -08:00
tedyu d7adc58e52 HBASE-20107 Add a test case for HBASE-14317 (Zephyr Guo) 2018-02-28 21:17:03 -08:00
Josh Elser bdedcc5631 HBASE-18133 Decrease quota reaction latency by HBase
Certain operations in HBase are known to directly affect
the utilization of tables on HDFS. When these actions
occur, we can circumvent the normal path and notify the
Master directly. This results in a much faster response to
changes in HDFS usage.

This requires FS scanning by the RS to be decoupled from
the reporting of sizes to the Master. An API inside each
RS is made so that any operation can hook into this call
in the face of other operations (e.g. compaction, flush,
bulk load).

Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-02-28 14:11:20 -05:00
Sergey Soldatov 393ab302ab HBASE-19863 java.lang.IllegalStateException: isDelete failed when SingleColumnValueFilter is used
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
Signed-off-by: ramkrish86 <ramkrishna.s.vasudevan@gmail.com>
2018-02-28 13:58:37 -05:00
Vladimir Rodionov 6cfa208add HBASE-17825: Backup further optimizations
Signed-off-by: Josh Elser <elserj@apache.org>
2018-02-28 09:14:15 -05:00
Chia-Ping Tsai 62ee7d9502 HBASE-20097 Merge TableDescriptors#getAll and TableDescriptors#getAllDescriptors into one
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-28 15:21:21 +08:00
Chia-Ping Tsai 197bd79070 HBASE-20084 Refactor the RSRpcServices#doBatchOp
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-28 15:15:34 +08:00
Chia-Ping Tsai 7f6e971c4c HBASE-20093 Replace ServerLoad by ServerMetrics for ServerManager
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-28 14:57:50 +08:00
Michael Stack ba063abd2f HBASE-20100 TestEnableTableProcedure flakey
Allow OPEN as a possible state when update region transition state.
Usually state is OPENING but if crash before finish step is completed,
on replay, master may have read that the state is OPEN from meta table
and so will think it open... When we replay the procedure finish, allow
that the region is already OPEN.
2018-02-27 14:51:21 -08:00
BELUGA BEHR f01c90bc0e HBASE-19614 Use ArrayDeque as Queue instead of LinkedList in CompoundBloomFilterWriter
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-02-27 11:50:52 -08:00
gsheffi 0fa5d69fc7 HBASE-20074 [FindBugs] Same code on both branches in CompactingMemStore#initMemStoreCompactor
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-27 11:21:40 -06:00
Sean Busbey e4ce38df32 Revert "for creating patch HBASE-20074-V01.patch"
This reverts commit 73028d5bd9.

bad commit message
2018-02-27 11:20:54 -06:00
Chia-Ping Tsai dbd8013057 HBASE-20092 Fix TestRegionMetrics#testRegionMetrics
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-27 16:48:17 +08:00
zhangduo f06a89b531 HBASE-20066 Region sequence id may go backward after split or merge 2018-02-27 15:33:07 +08:00
Michael Stack d272ac908c HBASE-20069 fix existing findbugs errors in hbase-server; ADDENDUM Address review 2018-02-26 23:16:21 -08:00
Michael Stack b11e506664 HBASE-20069 fix existing findbugs errors in hbase-server 2018-02-26 16:01:31 -08:00
gsheffi 73028d5bd9 for creating patch HBASE-20074-V01.patch
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-26 09:57:22 -08:00
Vasudevan 7cfb46432f HBASE-20036 TestAvoidCellReferencesIntoShippedBlocks timed out (Ram) 2018-02-26 22:07:42 +05:30
zhangduo 2beda62a10 HBASE-20083 Fix findbugs error for ReplicationSyncUp 2018-02-26 22:13:13 +08:00
zhangduo 8c74d177f6 HBASE-20082 Fix findbugs errors only on master which are introduced by HBASE-19397 2018-02-26 15:50:51 +08:00
Michael Stack 549a6d93d4 HBASE-20043 ITBLL fails against hadoop3
Fix MoveRandomRegionOfTableAction. It depended on old AM behavior.
Make it do explicit move as is required in AMv3; w/o it, it was just
closing region causing test to fail.

Fix pom so hadoop3 profile specifies a different netty3 version.

Bunch of logging format change that came of trying trying to read
the spew from this test.
2018-02-24 17:29:54 -08:00
zhangduo ba5fb53d14 HBASE-20065 Revisit the timestamp usage in MetaTableAccessor 2018-02-25 09:13:56 +08:00
Sean Busbey f90b47e3b0 HBASE-20061 HStore synchronized member variable filesCompacting should be private
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-24 10:57:01 -06:00
tedyu 9a7645360b HBASE-20054 Forward port HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of KeeperException 2018-02-23 19:14:52 -08:00
Sean Busbey 216d2d4648 Revert "HBASE-19835 Use explicit casting to avoid side effects"
This reverts commit f1a81618fd.

 Conflicts:
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
	hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Segment.java

patch reverted changes that happened in parallel without explanation. see jira.
2018-02-23 16:50:15 -06:00
zhangduo ad5cd50dfc HBASE-20048 Revert serial replication feature 2018-02-23 13:58:31 +08:00
Toshihiro Suzuki 1bc996aa50 HBASE-20049 Region replicas of SPLIT and MERGED regions are kept in in-memory states until restarting master
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-22 20:06:21 -08:00
Apekshit Sharma 51f7b75f1f HBASE-19401 Add missing security checks in RSRpcServices 2018-02-22 16:09:53 -08:00
tedyu d68f697f39 HBASE-20052 TestRegionOpen#testNonExistentRegionReplica fails due to NPE 2018-02-22 12:35:12 -08:00
Umesh Agashe 8afa49f5d0 HBASE-19767 Fix for Master web UI shows negative values for Remaining KVs
Negative Remaining KVs and progress percent greater than 100 is because CompactionProgress#totalCompactingKVs is sometimes less than CompactionProgress#currentCompactedKVs.
Changes add a getter to CompactionProgress#totalCompactingKVs and from inside getter warning is logged. currentCompactedKVs are return when totalCompactingKVs are less than current.

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-22 10:10:21 -08:00
Michael Stack 3434e99e6c HBASE-20044 TestClientClusterStatus is flakey"
Revert "Revert "HBASE-2004 TestClientClusterStatus is flakey""
This is a revert of a revert, i.e. a reapplication, just so I can fix
the JIRA number.

This reverts commit 6796b8e21f.
2018-02-22 09:44:04 -08:00
Michael Stack 3d2e9e2125 Revert "HBASE-2004 TestClientClusterStatus is flakey"
Revert to fix JIRA number; should be 20044.

This reverts commit 92bb4db9ef.
2018-02-22 09:44:04 -08:00
Michael Stack 11af33af2a HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory; ADDEDNDUM to fix TestRegionOpen failure 2018-02-22 09:15:16 -08:00
Josh Elser 4cf846d085 HBASE-20035 Stabilize the flaky TestQuotaStatusRPCs
The test will fail if the quota moves to violation before
the second half of the test.

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-22 09:40:20 -05:00
Michael Stack 2440f807bf Revert "HBASE-19767 Fix for Master web UI shows negative values for Remaining KVs"
Applied prematurely.

This reverts commit 61b55166bf.
2018-02-21 18:02:25 -08:00
zhangduo b328807d25 HBASE-20038 TestLockProcedure.testTimeout is flakey 2018-02-21 17:24:19 -08:00
Esteban Gutierrez a0900857c7 HBASE-19391 Calling HRegion#initializeRegionInternals from a region replica can still re-create a region directory 2018-02-21 15:56:36 -08:00
Michael Stack 13223c217c HBASE-20042 TestRegionServerAbort flakey; ADDENDUM, RETRY 2018-02-21 15:26:21 -08:00
Umesh Agashe 61b55166bf HBASE-19767 Fix for Master web UI shows negative values for Remaining KVs
Negative Remaining KVs and progress percent greater than 100 is because CompactionProgress#totalCompactingKVs is sometimes less than CompactionProgress#currentCompactedKVs.
Changes add a getter to CompactionProgress#totalCompactingKVs and from inside getter warning is logged. currentCompactedKVs are return when totalCompactingKVs are less than current.
2018-02-21 15:14:24 -08:00
Michael Stack 3a3994223c HBASE-2004 TestClientClusterStatus is flakey 2018-02-21 14:53:31 -08:00
Michael Stack 3f82098d4b HBASE-20042 TestRegionServerAbort flakey 2018-02-21 13:26:10 -08:00
Andrew Purtell 173a5bf1f1 HBASE-20027 Add test TestClusterPortAssignment
LocalHBaseCluster forces random port assignment for sake of concurrent unit test
execution friendliness, but we still need a positive test for RPC and info port
assignment.
2018-02-21 13:12:30 -08:00
tedyu bf5f034463 HBASE-19166 AsyncProtobufLogWriter persists ProtobufLogWriter as class name for backward compatibility 2018-02-21 13:04:59 -08:00
Mike Drob 5d994a24fc HBASE-20039 MR tests out to hbase-mapreduce mobile 2018-02-21 14:54:30 -06:00
Josh Elser d9b8dcc1d3 HBASE-19953 Ensure post DDL hooks are only called after successful operations
The 1.x functionality of Master DDL operations is that "post" observer hooks
are only invoked when the DDL action was successful. With the async-ness of
ProcV2, we find ourselves in a case where the post-hook may be invoked before
the Procedure runs and fails. We need to introduce some blocking to wait and
see if the Procedure is going to fail on a precondition before invoking the hook.

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-21 14:02:49 -05:00
zhangduo a27ef55a40
HBASE-20037 Race when calling SequenceIdAccounting.resetHighest 2018-02-21 10:05:02 -08:00
anastas 92d04d5751 HBASE-19506: The CellChunkMap index chunks are usually small, so in order to prevent memory underutilization, HBASE-19506 presents small chunks preallocated in a small pool 2018-02-21 10:12:25 +02:00
Chia-Ping Tsai 79d9403a79 HBASE-20017 BufferedMutatorImpl submit the same mutation repeatedly
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-20 16:59:48 -08:00
Ted Yu b75a8f4047 HBASE-19954 Separate TestBlockReorder into individual tests to avoid ShutdownHook suppression error against hadoop3 2018-02-20 14:56:20 -08:00
Apekshit Sharma 6b84393661 HBASE-19400 Add missing security checks in MasterRpcServices
- Added ADMIN permission check for following rpc calls:
  normalize, setNormalizerRunning, runCatalogScan, enableCatalogJanitor, runCleanerChore,
  setCleanerChoreRunning, execMasterService, execProcedure, execProcedureWithRet
- Moved authorizationEnabled check to start of AccessChecker's functions. Currently, and IDK why,
  we call authManager.authorize() first and then discard its result if authorizationEnabled is false. Weird.
2018-02-20 13:37:20 -07:00
Michael Stack 148356a7bf HBASE-20029 @Ignore TestQuotaThrottle and TestReplicasClient#testCancelOfMultiGet 2018-02-20 12:09:47 -08:00
Michael Stack 51cea3e2c3 HBASE-20024 TestMergeTableRegionsProcedure is STILL flakey 2018-02-20 11:08:27 -08:00
zhangduo 69d2becc73 HBASE-20021 TestFromClientSideWithCoprocessor is flakey 2018-02-20 06:24:42 -08:00
zhangduo b7685307e4 HBASE-20020 Make sure we throw DoNotRetryIOException when ConnectionImplementation is closed 2018-02-20 06:12:13 -08:00
zhangduo 391790ddb0 HBASE-19978 The keepalive logic is incomplete in ProcedureExecutor 2018-02-19 17:13:47 -08:00
Reid Chan a9a6eed372 HBASE-19950 Introduce a ColumnValueFilter
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-20 04:56:13 +08:00
eshcar f3bb9b9613 HBASE-18294 Reduce global heap pressure: flush based on heap occupancy 2018-02-18 09:55:44 +02:00
Michael Stack f3ff55a2b4 HBASE-20015 TestMergeTableRegionsProcedure and TestRegionMergeTransactionOnCluster flakey 2018-02-17 11:38:26 -08:00
Michael Stack 68d509bc1f HBASE-20013 TestZKPermissionWatcher is flakey 2018-02-16 23:22:23 -08:00
Chia-Ping Tsai dad90f6cce HBASE-19680 BufferedMutatorImpl#mutate should wait the result from AP in order to throw the failed mutations 2018-02-17 07:33:58 +08:00
Michael Stack abf7de702c HBASE-20011 Disable TestRestoreSnapshotFromClientWithRegionReplicas; it is flakey. Needs attention. 2018-02-16 14:46:24 -08:00
Mike Drob 8cf7a9d51e HBASE-19920 Lazy init for ProtobufUtil classloader 2018-02-16 09:12:15 -06:00
Umesh Agashe 9497eac4e0 HBASE-19988 Fixed chatty log from HRegion#lockRowsAndBuildMiniBatch() when interrupted while waiting on a row lock 2018-02-16 07:01:56 -08:00
Michael Stack 40f8d20cf7 HBASE-20006 TestRestoreSnapshotFromClientWithRegionReplicas is flakey
Remove assert in splittableregionprocedure. It was in the prepare.
Was causing fail in legit case where a region split follows a
table split BEFORE the parent has been GC'd. The region split
finds the parent in SPLIT state which is right. The assert was
having us fail. No need.

Also disabled TestHTrace since not supported in 2.0.0 and flakey.
2018-02-15 20:46:32 -08:00
Michael Stack 50c705dad9 HBASE-19998 Flakey TestVisibilityLabelsWithDefaultVisLabelService
Only call server.checkIfShouldMoveSystemRegionAsync if a node has been
added. Do not call it if only one regionserver in cluster. Make it
so ServerCrashProcedure runs before it. Add logging if
server.checkIfShouldMoveSystemRegionAsync was responsible for
MOVE (Previous was a mystery when it cut in).

Previous we'd call it when there was a nodeChildrenChanged. These
happen before nodeDeleted. If a server crashed,
checkIfShouldMoveSystemRegionAsync could run first, find the
server that had not yet registered as crashed, find system
tables on it and then try to move them. It would fail because
server would not respond to RPC. The region move would then
be waiting on the servercrashprocedure to wake it up when
done processing but this move had locked the region so
SCP couldn't run....
2018-02-15 06:08:55 -08:00
Thiruvel Thirumoolan 816d860228 HBASE-19996 Some nonce procs might not be cleaned up (follow up HBASE-19756)
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-15 04:47:53 -08:00
anastas 07e93458fe HBASE-19930: Fixing the bug, in the rare case when there is a merge into CCM and the one of the old segments has a big cell allocated on-heap and it needs to be copied to the MSLAB 2018-02-15 14:34:12 +02:00
zhangduo c18e7a963d HBASE-20000 Remove the quantum logic in FairQueue, always put high priority queue in front 2018-02-15 13:49:54 +08:00
Toshihiro Suzuki d0f2d18ca7 HBASE-19980 NullPointerException when restoring a snapshot after splitting a region
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-14 09:37:16 -08:00
Michael Stack 8d26736bc2 HBASE-19116 Currently the tail of hfiles with CellComparator* classname makes it so hbase1 can't open hbase2 written hfiles; fix
Serializing, if appropriate, write the hbase-1.x version of the
Comparator to the hfile trailer so hbase-1.x files can read hbase-2.x
hfiles (they are the same format).
2018-02-14 07:46:30 -08:00
Michael Stack 2f1b3eab67 HBASE-19998 Flakey TestVisibilityLabelsWithDefaultVisLabelService 2018-02-13 22:32:20 -08:00
Michael Stack 6e35f5eab9 HBASE-19965 Fix flaky TestAsyncRegionAdminApi; ADDENDUM do same to TestAsyncTableAdminAPI AGAIN (make TestAsyncTableAdminAPI3... already an TestAsyncTableAdminAPI2) 2018-02-13 21:06:00 -08:00
Pankaj Kumar 39e191e559 HBASE-19979 ReplicationSyncUp tool may leak Zookeeper connection 2018-02-13 08:59:02 -08:00
Michael Stack ba402b1e7b Revert "HBASE-19970 Remove unused functions from TableAuthManager."
This reverts commit 7cc239fb5a.
2018-02-13 06:19:08 -08:00
Chia-Ping Tsai 2f48fdbb26 HBASE-19876 The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner 2018-02-13 21:08:59 +08:00
Vasudevan 16f1f5b494 HBASE-19977 FileMmapEngine allocation of byte buffers should be
synchronized (Ram)
2018-02-13 15:49:37 +05:30
Michael Stack b4622ffad7 HBASE-19965 Fix flaky TestAsyncRegionAdminApi; ADDENDUM do same to TestAsyncTableAdminAPI 2018-02-12 22:52:56 -08:00
Michael Stack cf57ea15f1 HBASE-19965 Fix flaky TestAsyncRegionAdminApi 2018-02-12 22:27:12 -08:00
Apekshit Sharma 7cc239fb5a HBASE-19970 Remove unused functions from TableAuthManager.
Functions deleted: setTableUserPermissions, setTableGroupPermissions, setNamespaceUserPermissions,
setNamespaceGroupPermissions, writeTableToZooKeeper, writeNamespaceToZooKeeper
2018-02-12 16:01:29 -08:00
Michael Stack c2ee82c909 HBASE-19986 If HBaseTestClassRule timesout a test, thread dump 2018-02-12 15:28:40 -08:00
Michael Stack 00f8877323 HBASE-19972 Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink; ADDENDUM to fix TestReplicationSink 2018-02-12 10:47:25 -08:00
Michael Stack 67b69fb2c7 HBASE-16060 1.x clients cannot access table state talking to 2.0 cluster
This patch adds mirroring of table state out to zookeeper. HBase-1.x
clients look for table state in zookeeper, not in hbase:meta where
hbase-2.x maintains table state.

The patch also moves and refactors the 'migration' code that was put in
place by HBASE-13032.

D hbase-client/src/main/java/org/apache/hadoop/hbase/CoordinatedStateException.java
 Unused.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
 Move table state migration code from Master startup out to
TableStateManager where it belongs. Also start
MirroringTableStateManager dependent on config.

A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MirroringTableStateManager.java

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
 Move migration from zookeeper of table state in here. Also plumb in
mechanism so subclass can get a chance to look at table state as we do
the startup fixup full-table scan of meta.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Bug-fix. Now we create regions in CLOSED state but we fail to check
table state; were presuming table always enabled. Meant on startup
there'd be an unassigned region that never got assigned.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMirroringTableStateManager.java
 Test migration and mirroring.
2018-02-12 08:47:02 -08:00
zhangduo 8ff783f76d HBASE-19554 Dump thread info before failure 2018-02-12 16:03:52 +08:00
huzheng 629eaf8b74 HBASE-19972 Should rethrow the RetriesExhaustedWithDetailsException when failed to apply the batch in ReplicationSink 2018-02-11 14:48:21 +08:00
Michael Stack 0593dda663 HBASE-19951 Cleanup the explicit timeout value for test method 2018-02-10 09:24:31 -08:00
zhangduo 32f235abee HBASE-19966 The WriteEntry for WALKey maybe null if we failed to call WAL.append 2018-02-09 19:09:40 -08:00
Guanghao Zhang 2b63af376e HBASE-19923 Reset peer state and config when refresh replication source failed 2018-02-09 18:54:03 +08:00
Guanghao Zhang 0a5f26324c HBASE-19942 (addendum) Fix flaky TestSimpleRpcScheduler 2018-02-09 13:37:06 +08:00
Aman Poonia f1a81618fd HBASE-19835 Use explicit casting to avoid side effects
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-02-08 12:32:36 -08:00
Michael Stack 277ce3d8a3
HBASE-19956 Remove category as a consideration timing out tests; set all test to timeout at 10minutes regardless 2018-02-07 23:10:12 -08:00
Mike Drob f5dbdf0dab HBASE-19947 ITU should overwrite HTU local FS 2018-02-07 16:53:54 -06:00
zhangduo dcbb331792 HBASE-19929 Call RS.stop on a session expired RS may hang 2018-02-07 15:52:04 +08:00
Chia-Ping Tsai d8b999e695 HBASE-19900 Region-level exception destroy the result of batch 2018-02-07 08:41:56 +08:00
Guanghao Zhang a5b86dd77a HBASE-19942 Fix flaky TestSimpleRpcScheduler 2018-02-07 07:54:39 +08:00
Balazs Meszaros 380083e7d3 HBASE-19886 Display maintenance mode in shell, web UI 2018-02-06 14:53:32 -08:00
Mike Drob b5ccfecc96 HBASE-19946 More test clean up for local-mode HTU 2018-02-06 15:20:39 -06:00
Michael Stack 001f9cc5ea HBASE-19940 TestMetaShutdownHandler flakey; ADDENDUM: yet more debug 2018-02-06 09:47:30 -08:00
Michael Stack 590bee78dc HBASE-19940 TestMetaShutdownHandler flakey; ADDENDUM: more debug 2018-02-06 06:35:36 -08:00
Guanghao Zhang 77153d4dc6 HBASE-19944 Fix timeout TestVisibilityLabelsWithCustomVisLabService
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-02-06 20:30:08 +08:00
Umesh Agashe 3fa27c2382 HBASE-19941 Flaky TestCreateTableProcedure times out in nightly, needs to LargeTests 2018-02-05 22:13:53 -08:00
tedyu 11e3a83870 HBASE-19934 HBaseSnapshotException when read replicas is enabled and online snapshot is taken after region splitting (Toshihiro Suzuki) 2018-02-05 21:03:56 -08:00
Umesh Agashe 3bb8daa605
HBASE-19939 Fixed NPE in tests TestSplitTableRegionProcedure#testSplitWithoutPONR() and testRecoveryAndDoubleExecution()
Value of 'htd' is null as it is initialized in the constructor but when the object is deserialized its null. Got rid of member variable htd and made it local to method.
2018-02-05 20:48:56 -08:00
zhangduo f5197979aa HBASE-19927 Addendum join on RegionServerThread instead of HRegionServer 2018-02-06 10:01:23 +08:00
Michael Stack 6307689a6e HBASE-19940 TestMetaShutdownHandler flakey 2018-02-05 12:37:48 -08:00
Michael Stack 9f2149f171 HBASE-19840 Flakey TestMetaWithReplicas; ADDENDUM Adding debug 2018-02-05 11:01:09 -08:00
Michael Stack c245bd5c03 HBASE-19837 Flakey TestRegionLoad; ADDENDUM Report more often and wait less time on change (also add some debug on TestMetaShutdown test) 2018-02-05 08:40:26 -08:00
Rajeshbabu Chintaguntla c5f86f2ce4 HBASE-19703 Functionality added as part of HBASE-12583 is not working after moving the split code to master
Co-authored-by: Michael Stack <stack@apache.org>

Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-02-05 23:56:18 +08:00
zhangduo ad3a1ba495 HBASE-19936 Introduce a new base class for replication peer procedure 2018-02-05 20:23:19 +08:00
zhangduo 88d6e06a1f HBASE-19554 reenable TestDLSAsyncFSWAL/TestDLSFSHLog for debugging 2018-02-05 08:58:16 +08:00
zhangduo e1cd10b002 HBASE-19927 TestFullLogReconstruction flakey 2018-02-05 08:41:32 +08:00
Michael Stack ab5a26ad5e HBASE-19931 TestMetaWithReplicas failing 100% of the time in testHBaseFsckWithMetaReplicas 2018-02-04 14:38:21 -08:00
anastas 170ffbba68 HBASE-19658 make the test testFlatteningToJumboCellChunkMap() stable, by eliminating the possibility of third cell to be added while in-memory-flush is still in progress 2018-02-04 14:59:10 +02:00
zhangduo 14420e1b41 HBASE-19926 Use a separated class to implement the WALActionListener for Replication 2018-02-04 20:36:43 +08:00
Ashish Singhi 397d34736e HBASE-19905 ReplicationSyncUp tool will not exit if a peer replication is disabled
Signed-off-by: Ashish Singhi <ashishsinghi@apache.org>
2018-02-04 17:52:38 +05:30
zhangduo 2e1ec3d3d8 HBASE-19914 Refactor TestVisibilityLabelsOnNewVersionBehaviorTable 2018-02-04 13:27:28 +08:00
Michael Stack bfd74686c7 HBASE-19928 TestVisibilityLabelsOnNewVersionBehaviorTable fails 2018-02-03 10:17:40 -08:00
Michael Stack 12f3c82a86 HBASE-19919 Tidying up logging; ADDENDUM Fix tests w/ mocked Servers 2018-02-03 09:25:16 -08:00
Michael Stack 06dec20582
HBASE-19919 Tidying up logging 2018-02-03 08:42:02 -08:00
zhangduo 6519b98ac3 Revert "HBASE-19919 Tidying up logging"
This reverts commit 40250f8c5f.

Wrong patch. Revert to fix compile error.
2018-02-03 18:18:09 +08:00
Michael Stack 40250f8c5f HBASE-19919 Tidying up logging 2018-02-02 22:52:41 -08:00
Michael Stack 41974efa85
HBASE-19726 Failed to start HMaster due to infinite retrying on meta assign 2018-02-02 18:29:33 -08:00
Guanghao Zhang ad580acc89 HBASE-19918 Promote TestAsyncClusterAdminApi to LargeTests 2018-02-03 09:39:40 +08:00
Chia-Ping Tsai 2f4d0b94bc HBASE-19720 Rename WALKey#getTabnename to WALKey#getTableName 2018-02-02 19:43:26 +08:00
zhangduo fc6d140adf HBASE-19904 Break dependency of WAL constructor on Replication 2018-02-02 14:10:29 +08:00
tedyu a11258599e HBASE-19895 Add keepDeletedCells option in ScanOptions for customizing scanInfo in pre-hooks (Ankit Singhal) 2018-02-01 18:27:18 -08:00
Mike Drob 99b9fff07b HBASE-19841 Every HTU should be local until DFS starts 2018-02-01 20:24:11 -06:00
zhangduo 94dad844af HBASE-19913 Split TestStochasticLoadBalancer2 2018-02-02 08:29:28 +08:00
Michael Stack 3622bb0333 HBASE-19147 TestCacheOnWrite Times Out 2018-02-01 16:14:51 -08:00
Chia-Ping Tsai adccbb7edf HBASE-19897 RowMutations should follow the fluent pattern 2018-02-02 05:33:08 +08:00
Peter Somogyi d472422475 HBASE-19884 BucketEntryGroup's equals, hashCode and compareTo methods are not consistent
Move back to default equals and hashCode.
Remove compareTo and Comparator to PriorityQueue.

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-01 13:21:58 -08:00
kewang 38c8144a06 HBASE-19912 Remove useless 'writeToWAL' flag of Region#checkAndRowMutate
Signed-off-by: Michael Stack <stack@apache.org>
2018-02-01 12:33:48 -08:00
Umesh Agashe 57911d02c6 HBASE-19839 Fixed flakey tests TestMergeTableRegionsProcedure#testRollbackAndDoubleExecution and TestSplitTableRegionProcedure#testRollbackAndDoubleExecution
* Added a comment in MergeTableRegionsProcedure and SplitTableRegionProcedure explaining specific rollbacks has side effect that AssignProcedure/s are submitted asynchronously and those procedures may continue to execute even after rollback() is done.
* Updated comments in tests with correct rollback state to abort
* Added overloaded method MasterProcedureTestingUtility#testRollbackAndDoubleExecution which takes additional argument for waiting on all procedures to finish before asserting conditions
* Updated TestMergeTableRegionsProcedure#testRollbackAndDoubleExecution and TestSplitTableRegionProcedure#testRollbackAndDoubleExecution to use newly added method

Signed-off-by: Michael Stack <stack@apache.org>
2018-02-01 12:01:30 -08:00
Michael Stack b21b8bfb91 HBASE-19906 TestZooKeeper Timeout Includes move of TestQoSFunction from smalltests to mediumtests; it timedout in a run. 2018-02-01 10:53:18 -08:00
Michael Stack 9272f40a5c HBASE-19911 Convert some tests from small to medium because they are timing out: TestNettyRpcServer, TestClientClusterStatus; ADDENDUM changes TestCheckTestClasses to be medium sized test 2018-01-31 20:27:34 -08:00
Michael Stack b9cb1187b5 HBASE-19911 Convert some tests from small to medium because they are timing out: TestNettyRpcServer, TestClientClusterStatus 2018-01-31 20:23:14 -08:00
Michael Stack e17529ba73 HBASE-19910 TestBucketCache TimesOut 2018-01-31 18:17:01 -08:00
Michael Stack 28ee4be0fe HBASE-19909 TestRegionLocationFinder Timeout 2018-01-31 17:55:16 -08:00
Michael Stack bc0e9ce8f4 HBASE-19908 TestCoprocessorShortCircuitRPC Timeout.... 2018-01-31 17:32:39 -08:00
Michael Stack 414b2d0889 HBASE-19907 TestMetaWithReplicas still flakey 2018-01-31 15:17:38 -08:00
Michael Stack 60827fc1ea HBASE-19528 Major Compaction Tool; ADDENDUM
Add @ClassRule to new tests and rename those added here so they
align w/ how we name other Test classes.
2018-01-31 14:18:24 -08:00
Rahul Gidwani 4b3b627abe HBASE-19528 - Major Compaction Tool 2018-01-31 10:18:03 -08:00
zhangduo 7c318cead9 HBASE-19887 Do not overwrite the surefire junit listener property in the pom of sub modules 2018-01-31 20:49:18 +08:00
binlijin f7faad0477 HBASE-19728 Add lock to filesCompacting in all place. 2018-01-31 14:05:30 +08:00
Peter Somogyi 194890b694
HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey
Recategorize TestCoprocessorWhitelistMasterObserver as Large test
2018-01-30 06:01:24 -08:00
Michael Stack 98c9d8f7a1 HBASE-19868 TestCoprocessorWhitelistMasterObserver is flakey 2018-01-29 21:35:28 -08:00
Peter Somogyi 34c6c99041
HBASE-19811 Fix findbugs and error-prone warnings in hbase-server (branch-2) - addendum
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-29 14:03:08 -08:00
Michael Stack 0b9a0dc951
HBASE-19840 Flakey TestMetaWithReplicas; ADDENDUM to fix Checksyte 2018-01-29 09:36:30 -08:00
Michael Stack 4f547b3817
HBASE-19840 Flakey TestMetaWithReplicas
Fix two issues:

 # Meta Replicas can all be assigned to the same server. This
 will call the test to hang when we do our kill of the server
 hosting meta because there'll be no replicas to read from
 as test intends. Check is to look for this condition on
 startup and adjust if we come across it. Replicas cross-cut
 assignment. They need work.
 # Other issue was shutdown. The master started toward the
 end of the test may not have come up fully by the time
 shutdown is called. We could be stuck assigning the
 meta replicas. Have shutdown shutdown the procedure
 executor engine.

There is other cleanup and notes in the below.

M HMaster
 Remove the silly stops in startup now we have real
 means of shutting down Master during init.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMetaBootstrap.java
 This replica stuff was doing stuff it shouldn't be doing
 like setting core Master state flags. It may have made
 sense once but now meta is assigned by a Pv2 Procedure
 so the flag setting in here is meddlesome. Clear out
 methods no longer needed.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
 Remove unused methods.
 Changes local variable names so they align w/ our naming elsewhere in
 code base.

M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMetaWithReplicas.java
 Check for all replicas on the one server.
2018-01-29 09:21:30 -08:00
zhangduo dab51c7900 HBASE-19885 Promote TestAssignmentManager to LargeTests 2018-01-30 00:24:45 +08:00
zhangduo 5b93ea1556 HBASE-19866 TestRegionServerReportForDuty doesn't timeout
Synchronized on authManager instead of RpcServer
2018-01-29 21:23:47 +08:00
zhangduo 305da1098a HBASE-19882 Promote TestProcedureManager to MediumTests 2018-01-29 17:48:44 +08:00
zhangduo f9480a56c7 HBASE-19881 Promote TestRegionReplicaReplicationEndpoint to LargeTests 2018-01-29 16:15:45 +08:00
zhangduo 94dfc7d60a HBASE-19880 Promote TestFuzzyRowFilterEndToEnd to LargeTests 2018-01-29 16:15:18 +08:00
zhangduo 80438924ff HBASE-19879 Promote TestAcidGuaranteesXXX to LargeTests 2018-01-29 14:55:04 +08:00
zhangduo 918599ef12 HBASE-19873 Add a CategoryBasedTimeout ClassRule for all UTs 2018-01-29 08:43:56 +08:00
Jan Hentschel 851e17987d HBASE-19869 TestLockManager now uses the right class for logging 2018-01-28 13:54:57 +01:00
zhangduo f1502a3aca HBASE-19867 Split TestStochasticLoadBalancer into several small tests 2018-01-27 18:25:46 +08:00
Michael Stack a5a8c4f3f2
Revert "HBASE-19841 LocalHTU to not enforce stream capabilities"
At Mike Drobs' request!
2018-01-26 15:11:50 -08:00
Mike Drob 9ea152d235 HBASE-19841 LocalHTU to not enforce stream capabilities 2018-01-26 10:50:16 -06:00
haxiaolin 94a1c051ba HBASE-19861 - Avoid using RPCs when querying table infos for master status pages
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-26 08:02:14 -08:00
Guanghao Zhang 39b912aae9 HBASE-19818 Scan time limit not work if the filter always filter row key 2018-01-26 17:02:22 +08:00
Apekshit Sharma 01c34243fe HBASE-19866 Add cateogry based timeout using Timeout Rule to TestRegionServerReportForDuty. 2018-01-26 00:30:56 -08:00
Apekshit Sharma 1beb687f4e HBASE-19862 addendum: Checkstyle fixes, remove unused imports and variables. Add @Rule for timeout. 2018-01-26 00:24:35 -08:00
Apekshit Sharma 61358985b1 HBASE-19862 Fix TestTokenAuthentication - mock RegionCoprocessorEnvironment should be of type HasRegionServerServices also. 2018-01-25 23:15:45 -08:00
Thiruvel Thirumoolan ce50830a0a HBASE-19756 Master NPE during completed failed proc eviction
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-01-24 16:42:58 -08:00
Michael Stack 970636c5af HBASE-19838 Can not shutdown backup master cleanly when it has already tried to become the active master
On Master@shutdown, close the shared Master connection to kill any
ongoing RPCs by hosted clients.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Call close ont the Master shared clusterconnection to kill any ongoing
rpcs.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
Remove guts of close; we were closing the Masters connection....not
our responsibility.

Added unit test written by Duo Zhang which demonstrates the case where
Master will not go down.

Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-23 11:07:00 +08:00
Chia-Ping Tsai 541f8ad8a8 HBASE-4224 Need a flush by regionserver rather than by table option 2018-01-23 09:51:05 +08:00
Peter Somogyi 93a182f286 HBASE-19811 Fix findbugs and error-prone warnings in hbase-server (branch-2)
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-22 17:20:59 -08:00
Michael Stack d49357f265 HBASE-19840 Flakey TestMetaWithReplicas
Adding debuging
2018-01-22 12:28:56 -08:00
Rahul Gidwani b577fa0814 HBASE-15321 - Ability to open a HRegion from hdfs snapshot. 2018-01-22 12:09:38 -08:00
Sergey Soldatov be496ba8ea HBASE-19774 incorrect behavior of locateRegionInMeta 2018-01-22 11:48:55 -08:00
Michael Stack c60359959c HBASE-19837 Flakey TestRegionLoad
Add debug, change name of the rows so readable.
2018-01-22 08:01:57 -08:00
zhangduo 86f4df5f74 HBASE-19836 Fix TestZooKeeper.testLogSplittingAfterMasterRecoveryDueToZKExpiry 2018-01-22 18:32:31 +08:00
Michael Stack f3c563fc00 HBASE-19837 Flakey TestRegionLoad
Add debug, change name of the rows so readable.
2018-01-21 23:24:23 -08:00
Michael Stack 27d00f5861 HBASE-19794 TestZooKeeper hangs
Kill backup master first

Add some cleanup around NamespaceManager

Shorten the timeout waiting on namespace manager as workaround
until we have better soln for interrupting ongoing client rpcs.
Do it in general for all tests.

Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-21 20:59:16 +08:00
Michael Stack 7fe4aa6fe4 HBASE-19828 Flakey TestRegionsOnMasterOptions.testRegionsOnAllServers
Rename the PE Worker threads.

Send an interrupt if worker taking a long time to go down
(it may be RPC'ing out to a dead server, retrying so
interrupt). Also join on the ProcedureExecutor shutting down.
This will make problems shutting down more obvious.

Disable TestRegionsOnMasterOptions. Master carrying Regions is broke.
2018-01-19 21:54:19 -08:00
Artem Ervits 11d6e6b1e6 HBASE-19829 hadoop-minicluster pulls zookeeper:test-jar:tests 3.4.6
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-19 15:18:22 -08:00
Michael Stack 579586c697
HBASE-19827 Addendum for Flakey TestAssignmentManager Don't clear old location from RTP... 2018-01-19 13:46:44 -08:00
Michael Stack c66b42fd9d Revert "HBASE-19827 Addendum for Flakey TestAssignmentManager"
Premature commit.

This reverts commit fa66c7e28e.
2018-01-19 11:47:28 -08:00
Reid Chan 298dc0c626 HBASE-19822 Override cancel(boolean) in LogCleaner (Addendum)
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-19 11:46:46 -08:00
Michael Stack fa66c7e28e HBASE-19827 Addendum for Flakey TestAssignmentManager
Don't clear old location from RTP...
2018-01-19 10:59:47 -08:00
Michael Stack 8b520ce50d HBASE-19822 HFileCleaner threads stick around after shutdown stuck on queue#take Interrupt wait on queue when cancel is called. 2018-01-18 13:52:51 -08:00
Guangxu Cheng cde1f821eb HBASE-19799 Add web UI to rsgroup
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-18 12:49:43 -08:00
Michael Stack 581fabe7b2 HBASE-19815 Flakey TestAssignmentManager.testAssignWithRandExec (Part Two).
Part One cleaned up a ClassCastException.

Part Two adds the ServerCrashProcedure#handleRIT behavior to RecoverMetaProcedure.

Adds debug in the test.
2018-01-18 11:32:21 -08:00
Michael Stack 646770dd51 HBASE-19527 Make ExecutorService threads daemon=true
Set the ProcedureExcecutor worker threads as daemon.
Ditto for the timeout thread.

Remove hack from TestRegionsOnMasterOptions that was
put in place because the test would not go down.
2018-01-18 11:30:15 -08:00
lixiang b3eff31432 HBASE-19821 TestCleanerChore#testOnConfigurationChange() requires at least 4 processors to get passed
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-18 11:20:29 -08:00
Apekshit Sharma 09ffbb5b68 HBASE-19820 Restore public constructor of MiniHBaseCluster (API compat). 2018-01-18 09:05:30 -08:00
tedyu 7573e07cc6 HBASE-19806 Lower max versions for table column family of hbase:meta 2018-01-18 07:57:22 -08:00
zhangduo 678b98a726 HBASE-19812 TestFlushSnapshotFromClient fails because of failing region.flush 2018-01-18 16:23:07 +08:00
Scott Wilson e3d15834cb HBASE-19816 Refresh repliation sinks on UnknownHostException
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-17 16:24:48 -08:00
Michael Stack eeb40ff66c
Revert "HBASE-19791 TestZKAsyncRegistry hangs"
Premature push

This reverts commit d3a306d81d.
2018-01-17 15:19:48 -08:00
Michael Stack 53d0c2388d HBASE-19815 Flakey TestAssignmentManager.testAssignWithRandExec 2018-01-17 13:52:39 -08:00
Michael Stack a3c98b2dd8
HBASE-19808 Reenable TestMultiParallel 2018-01-17 08:58:37 -08:00
Reid Chan 8a8be33c71 HBASE-19736 Remove BaseLogCleanerDelegate deprecated #isLogDeletable(FileStatus) and use #isFileDeletable(FileStatus) instead
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2018-01-17 21:16:23 +08:00
zhangduo d8d6ecdad1 HBASE-19792 TestReplicationSmallTests.testDisableEnable fails 2018-01-17 14:23:05 +08:00
Michael Stack d3a306d81d
HBASE-19791 TestZKAsyncRegistry hangs 2018-01-16 12:34:05 -08:00
Vasudevan d155a4e33d HBASE-19732 Replica regions does not return back the MSLAB chunks to pool
(Ram)
2018-01-15 22:20:27 +05:30
zhangduo 3a66f15557 HBASE-19797 Operator priority leads to wrong logic in ReplicationSourceWALReader 2018-01-15 18:10:50 +08:00
zhangduo 3b638f7ea1 HBASE-19793 Minor improvements on Master/RS startup 2018-01-15 17:53:28 +08:00
zhangduo e546034f09 HBASE-19795 Move the tests which only need zookeeper in TestZooKeeper to hbase-zookeeper module 2018-01-15 10:57:51 +08:00
zhangduo c554340a91 HBASE-19768 RegionServer startup failing when DN is dead 2018-01-14 17:30:50 +08:00
Michael Stack 4ddfecac56 HBASE-19787 Fix or disable tests broken in branch-2 so can cut beta-1
M dev-support/make_rc.sh
  Disable checkstyle building site. Its an issue being fixed over in HBASE-19780

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
  The clusterid was being set into the process only after the
  regionserver registers with the Master. That can be too late for some
  test clients in particular. e.g. TestZKAsyncRegistry needs it as soon
  as it goes to run which could be before Master had called its run
  method  which is regionserver run method which then calls back to the
  master to register itself... and only then do we set the clusterid.
  HBASE-19694 changed start order which made it so this test failed.
  Setting the clusterid right after we set it in zk makes the test pass.

  Another change was that backup masters were not going down on stop.
  Backup masters were sleeping for the default zk period which is 90
  seconds. They were not being woken up to check for stop. On stop
  master now tells active master manager.

M hbase-server/src/test/java/org/apache/hadoop/hbase/TestJMXConnectorServer.java
  Prevent creation of acl table. Messes up our being able to go down
  promptly.

M hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestRegionsOnMasterOptions.java
M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerReadRequestMetrics.java
  Disabled for now because it wants to run with regions on the Master...
  currently broke!

M hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestZKAsyncRegistry.java
  Add a bit of debugging.

M hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDLSAsyncFSWAL.java
  Disabled. Fails 40% of the time.

M hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDLSFSHLog.java
  Disabled. Fails 33% of the time.

Disabled stochastic load balancer for favored nodes because it fails on
occasion and we are not doing favored nodes in branch-2.
2018-01-12 14:09:56 -08:00
Michael Stack c799b0e69b
HBASE-19694 The initialization order for a fresh cluster is incorrect Addendum. Adds in a missing line that helps debugging. 2018-01-12 13:09:05 -08:00
Guanghao Zhang 20ccaef841 HBASE-19783 Change replication peer cluster key/endpoint from a not-null value to null is not allowed
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-12 21:41:51 +08:00
Chia-Ping Tsai 4bd6ac3e10
HBASE-19746 Add default impl to Cell#getType 2018-01-11 15:03:20 -08:00
Michael Stack 3787c60cd1
HBASE-19694 The initialization order for a fresh cluster is
incorrect; ADDENDUM Fix compile issue.
2018-01-11 14:37:18 -08:00
Michael Stack d0aff98a67
HBASE-19694 The initialization order for a fresh cluster is incorrect
Become active Master before calling the super class's run method. Have
the wait-on-becoming-active-Master be in-line rather than off in a
background thread (i.e. undo running thread in startActiveMasterManager)

Purge the fragile HBASE-16367 hackery that attempted to fix this issue
previously by adding a latch to try and hold up superclass RegionServer
until cluster id set by subclass Master.
2018-01-11 14:29:11 -08:00
zhangduo e0eea94c98 HBASE-19758 Split TestHCM to several smaller tests
Signed-off-by: Michael Stack <stack@apache.org>
2018-01-11 11:39:51 -08:00
Rahul Gidwani 200c3d3b8e HBASE-11409 - Add more flexibility for input directory structure to LoadIncrementalHFiles 2018-01-11 11:05:11 -08:00
Rahul Gidwani 9f69be0011 Revert due to missing JIRA ticket - "Add more flexibility for input directory structure to LoadIncrementalHFiles"
This reverts commit ee3accb370.
2018-01-11 10:42:17 -08:00
Chia-Ping Tsai c88e444408 HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by increasing scanner timeout.
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-01-11 10:34:54 -08:00
Apekshit Sharma 15bb49d803 Revert due to missing author. "HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by increasing scanner timeout."
This reverts commit 0b9dc14dbc.
2018-01-11 10:34:42 -08:00
Apekshit Sharma 0b9dc14dbc HBASE-19685 Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors by increasing scanner timeout. 2018-01-11 10:31:45 -08:00
zhangduo 71a1192d67 HBASE-19751 Use RegionInfo directly instead of an identifier and a namespace when getting WAL 2018-01-11 15:47:34 +08:00
huzheng 8bf6adbe75 HBASE-19749 Revisit logic of UserScanQueryMatcher#mergeFilterResponse method 2018-01-11 12:05:02 +08:00
tedyu b568cf4ebd HBASE-19740 Repeated error message for NamespaceExistException 2018-01-10 18:25:45 -08:00
Rahul Gidwani ee3accb370 Add more flexibility for input directory structure to LoadIncrementalHFiles 2018-01-10 13:37:14 -08:00
huzheng 67a2c62a6a HBASE-19748 TestRegionReplicaFailover and TestRegionReplicaReplicationEndpoint UT hangs 2018-01-10 18:17:35 +08:00
Michael Stack 09c11b0ca4 HBASE-19743 Disable TestMemstoreLABWithoutPool
This is a revert of a revert, i.e., a reapplication only this time
I'm pushing w/ the right JIRA number
2018-01-09 18:21:38 -08:00
Michael Stack 88864f3d5d Revert "HBASE-19742 Disable TestMemstoreLABWithoutPool"
Revert because have wrong JIRA number.

This reverts commit 1454c44bcb.
2018-01-09 18:21:20 -08:00
Michael Stack 669b08f8ec HBASE-19742 Disable TestMemstoreLABWithoutPool 2018-01-09 14:33:29 -08:00
Guangxu Cheng 81ea657ed1 HBASE-19483 Add proper privilege check for rsgroup commands addendum
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-09 07:58:20 -08:00
Guanghao Zhang f458b89c05 HBASE-19139 Create Async Admin methods for Clear Block Cache 2018-01-09 14:43:01 +08:00
zhangduo eb88b69845 HBASE-19719 Fix checkstyle issues 2018-01-09 13:11:01 +08:00
Guanghao Zhang 2d5267331e HBASE-19711 TestReplicationAdmin.testConcurrentPeerOperations hangs
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
zhangduo ec364d0d6c HBASE-19707 Race in start and terminate of a replication source after we async start replicatione endpoint 2018-01-09 13:11:01 +08:00
Guanghao Zhang 01f1fc8284 HBASE-19636 All rs should already start work with the new peer change when replication peer procedure is finished
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
zhangduo 1a89793ee8 HBASE-19634 Add permission check for executeProcedures in AccessController 2018-01-09 13:11:01 +08:00
zhangduo 8eeedf0403 HBASE-19697 Remove TestReplicationAdminUsingProcedure 2018-01-09 13:11:01 +08:00
huzheng a5cfc51cbe HBASE-19661 Replace ReplicationStateZKBase with ZKReplicationStorageBase 2018-01-09 13:11:01 +08:00
zhangduo 368db315a6 HBASE-19687 Move the logic in ReplicationZKNodeCleaner to ReplicationChecker and remove ReplicationZKNodeCleanerChore 2018-01-09 13:11:01 +08:00
Guanghao Zhang d95ee41583 HBASE-19544 Add UTs for testing concurrent modifications on replication peer
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
zhangduo d357920988 HBASE-19686 Use KeyLocker instead of ReentrantLock in PeerProcedureHandlerImpl 2018-01-09 13:11:01 +08:00
zhangduo a85a215f36 HBASE-19623 Create replication endpoint asynchronously when adding a replication source 2018-01-09 13:11:01 +08:00
zhangduo 19707a85dd HBASE-19633 Clean up the replication queues in the postPeerModification stage when removing a peer 2018-01-09 13:11:01 +08:00
huzheng f89920a60f HBASE-19622 Reimplement ReplicationPeers with the new replication storage interface 2018-01-09 13:11:01 +08:00
zhangduo 62a4f5bb46 HBASE-19635 Introduce a thread at RS side to call reportProcedureDone 2018-01-09 13:11:01 +08:00
zhangduo f4703c6ed3 HBASE-19617 Remove ReplicationQueues, use ReplicationQueueStorage directly 2018-01-09 13:11:01 +08:00
zhangduo 5fc90244a8 HBASE-19642 Fix locking for peer modification procedure 2018-01-09 13:11:01 +08:00
zhangduo fdc907140b HBASE-19592 Add UTs to test retry on update zk failure 2018-01-09 13:11:01 +08:00
Guanghao Zhang 1025388da6 HBASE-19630 Add peer cluster key check when add new replication peer
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
Guanghao Zhang 65eb91f84e HBASE-19573 Rewrite ReplicationPeer with the new replication storage interface 2018-01-09 13:11:01 +08:00
zhangduo c4fa568b47 HBASE-19599 Remove ReplicationQueuesClient, use ReplicationQueueStorage directly 2018-01-09 13:11:01 +08:00
zhangduo 5e6c303528 HBASE-19543 Abstract a replication storage interface to extract the zk specific code 2018-01-09 13:11:01 +08:00
huzheng 8f5e54a456 HBASE-19525 RS side changes for moving peer modification from zk watcher to procedure 2018-01-09 13:11:01 +08:00
zhangduo 7afae59323 HBASE-19580 Use slf4j instead of commons-logging in new, just-added Peer Procedure classes 2018-01-09 13:11:01 +08:00
Guanghao Zhang 17762c60cc HBASE-19520 Add UTs for the new lock type PEER
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
zhangduo 62496b5f36 HBASE-19564 Procedure id is missing in the response of peer related operations 2018-01-09 13:11:01 +08:00
zhangduo 7f4bd0d371 HBASE-19524 Master side changes for moving peer modification from zk watcher to procedure 2018-01-09 13:11:01 +08:00
zhangduo f17198ff19 HBASE-19216 Implement a general framework to execute remote procedure on RS 2018-01-09 13:11:01 +08:00
Guangxu Cheng 7ddf79946d HBASE-19483 Add proper privilege check for rsgroup commands
Signed-off-by: tedyu <yuzhihong@gmail.com>
2018-01-08 19:48:37 -08:00
huzheng c5277d5f88 HBASE-19729 UserScanQueryMatcher#mergeFilterResponse should return INCLUDE_AND_SEEK_NEXT_ROW when filterResponse is INCLUDE_AND_SEEK_NEXT_ROW 2018-01-09 07:19:40 +08:00
zhangduo 2509a150c0
HBASE-19731 TestFromClientSide#testCheckAndDeleteWithCompareOp and testNullQualifier are flakey 2018-01-08 10:35:36 -08:00
Jan Hentschel 2af61718b1 HBASE-19541 Removed unnecessary semicolons in hbase-server 2018-01-08 16:43:43 +01:00
BELUGA BEHR afc2cdbaff HBASE-19684 BlockCacheKey toString Performance
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-08 17:06:21 +08:00
Chia-Ping Tsai 7378dad5a9 HBASE-19712 Fix TestSnapshotQuotaObserverChore#testSnapshotSize 2018-01-08 08:47:18 +08:00
tedyu 5a66eb978c HBASE-19696 Filter returning INCLUDE_AND_NEXT_COL doesn't skip remaining versions when scan has explicit columns (Ankit Singhal) 2018-01-07 07:50:00 -08:00
Josh Elser 45e46bb242 HBASE-19709 Ensure that we don't set a poolSize of 0
In some situations, Runtime.getRuntime().getAvailableProcessors()
may return 0 which would result in calculatePoolSize returning 0
which will trigger an exception. Guard against this case.

Signed-off-by: Reid Chan <reidddchan@outlook.com>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2018-01-05 18:44:35 -05:00
Beluga Behr a30d9fe8d9
HBASE-19651 Remove LimitInputStream 2018-01-05 16:59:29 -06:00
Michael Stack 2bf2b5b2c1
Revert "Revert "Revert "HBASE-19651 Remove LimitInputStream"""
This reverts commit 032701c96c.
2018-01-05 16:58:26 -06:00
Michael Stack 032701c96c
Revert "Revert "HBASE-19651 Remove LimitInputStream""
This reverts commit e5f7030ded.
2018-01-05 16:55:59 -06:00
Beluga Behr e5f7030ded
Revert "HBASE-19651 Remove LimitInputStream"
Revert to put Beluga in place as author.

This reverts commit 032fdc53de.
2018-01-05 16:55:01 -06:00
Janos Gub 9cd0e1a490 HBASE-17436 Add column to master web UI for displaying region states when listing regions of a table.
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-01-05 14:32:27 -08:00
tedyu 5ce9351129 HBASE-19483 Add proper privilege check for rsgroup commands - revert due to pending review comment 2018-01-05 12:06:31 -08:00