Commit Graph

6277 Commits

Author SHA1 Message Date
zhangduo 03cb581585 HBASE-19410 Move zookeeper related UTs to hbase-zookeeper and mark them as ZKTests 2017-12-06 16:38:58 +08:00
anoopsamjohn 78a6e0532e Revert "HBASE-19357 Bucket cache no longer L2 for LRU cache."
This reverts commit d952c5df9f.
2017-12-06 11:36:19 +05:30
anoopsamjohn d952c5df9f HBASE-19357 Bucket cache no longer L2 for LRU cache. 2017-12-06 11:26:10 +05:30
Michael Stack e048b2cbdb
HBASE-19323 Make netty engine default in hbase2 2017-12-05 15:29:57 -08:00
Michael Stack c77778dd13
HBASE-19426 Move has() and setTimestamp() to Mutation (Chia-Ping Tsai)
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-12-05 09:42:13 -08:00
Michael Stack db49391683 HBASE-19384 Results returned by preAppend hook in a coprocessor are replaced with
null from other coprocessor even on bypass

If 'bypass' is set by a Coprocessor, skip out on calling any subsequent
Coprocessors that might be chained to a bypassable method.

This patch restores some of the 'complete' behavior removed by
HBASE-19123 only 'bypass' now triggers 'complete'.
2017-12-05 08:07:23 -08:00
Ted Yu b906980e14 HBASE-19056 TestCompactionInDeadRegionServer is top of the flakies charts!
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-12-04 14:13:49 +08:00
zhangduo c61ed5dc7e HBASE-19399 Purge curator dependency from hbase-client 2017-12-03 08:30:43 +08:00
Chia-Ping Tsai 5f8b459cc1 HBASE-19408 Remove WALActionsListener.Base 2017-12-03 01:17:03 +08:00
Chia-Ping Tsai a5853221b4 HBASE-19339 Eager policy results in the negative size of memstore 2017-12-03 00:50:25 +08:00
Guanghao Zhang 6c6910b0b4 HBASE-19396 Fix flaky test TestHTableMultiplexerFlushCache 2017-12-02 20:05:59 +08:00
Andrew Purtell 99790a52ae HBASE-19406 Fix CompactionRequest equals and hashCode 2017-12-01 19:15:11 -08:00
Josh Elser fa5473c299 HBASE-19285 Implements table-level latency histograms
For a egionserver's view of a table (the regions
that belong to a table hosted on a regionserver),
this change tracks the latencies of operations that
affect the regions for this table.

Tracking at the per-table level avoids the memory bloat
and performance impact that accompanied the previous
per-region latency metrics while still providing important
details for operators to consume.

Signed-Off-By: Andrew Purtell <apurtell@apache.org>
2017-12-01 17:34:43 -05:00
Guangxu Cheng 01d366da4a HBASE-19326 Remove decommissioned servers from rsgroup
Signed-off-by: Michael Stack <stack@apache.org>

Conflicts:
	hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java
2017-12-01 11:12:34 -08:00
Michael Stack 752be198e7 HBASE-19392 TestReplicaWithCluster#testReplicaGetWithPrimaryAndMetaDown failure in master
Signed-off-by: Huaxiang Sun <hsun@apache.org>
2017-11-30 22:46:00 -08:00
zhangduo df3668818d HBASE-19344 improve asyncWAL by using Independent thread for netty #IO in FanOutOneBlockAsyncDFSOutput 2017-12-01 11:19:09 +08:00
Harshal Deepakkumar Jain 49eaa7a8f9 HBASE-19388 - Incorrect value is being set for Compaction Pressure in RegionLoadStats object inside HRegion class
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-11-30 08:48:27 -08:00
Toshihiro Suzuki 29079886ca HBASE-18112 Write RequestTooBigException back to client for NettyRpcServer
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-11-30 17:07:22 +08:00
zhangduo e2e08866f6 HBASE-19346 Use EventLoopGroup to create AsyncFSOutput 2017-11-30 15:23:22 +08:00
Michael Stack 0e6f1a0240 HBASE-19385 [1.3] TestReplicator failed 1.3 nightly 2017-11-29 23:13:25 -08:00
Apekshit Sharma 4f4aac77e1 HBASE-19367 Refactoring in RegionStates, and RSProcedureDispatcher
- Adding javadoc comments
- Bug: ServerStateNode#regions is HashSet but there's no synchronization to prevent concurrent addRegion/removeRegion. Let's use concurrent set instead.
- Use getRegionsInTransitionCount() directly to avoid instead of getRegionsInTransition().size() because the latter copies everything into a new array - what a waste for just the size.
- There's mixed use of getRegionNode and getRegionStateNode for same return type - RegionStateNode. Changing everything to getRegionStateNode. Similarly rename other *RegionNode() fns to *RegionStateNode().
- RegionStateNode#transitionState() return value is useless since it always returns it's first param.
- Other minor improvements
2017-11-29 22:42:39 -08:00
Guanghao Zhang 9e7ab9ffe2
HBASE-19359 Revisit the default config of hbase client retries number 2017-11-29 10:33:20 -08:00
zhangduo 22b90c4a64
HBASE-19372 Remove the Span object in SyncFuture as it is useless now
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-29 09:13:02 -08:00
binlijin 64ddce303e HBASE-19290 Reduce zk request when doing split log 2017-11-29 18:43:41 +08:00
Jerry He 0c4c395538 HBASE-19096 Add RowMutions batch support in AsyncTable
Signed-off-by: Jerry He <jerryjch@apache.org>
2017-11-28 18:49:08 -08:00
zhangduo 8688da9e9c HBASE-17049 Do not issue sync request when there are still entries in ringbuffer 2017-11-29 10:19:53 +08:00
Balazs Meszaros e946d9d841
HBASE-19242 Add MOB compact support for AsyncAdmin
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Guanghao Zhang <zghaobac@gmail.com>
2017-11-28 15:04:56 -08:00
Peter Somogyi 1447956846 HBASE-19363 Tests under TestCheckAndMutate are identical
Remove testCheckAndMutateUsingNewComparisonOperatorInstead

Signed-off-by: Michael Stack <stack@apache.org>
2017-11-28 12:19:13 -08:00
Josh Elser 4fef4cfc30 HBASE-19267 Remove compiler-plugin mapping executions as it breaks Java8 detection
It seems like the original reason this execution filter was added is no
longer an issue for 2.0. Actually, these entries actually preclude
Eclipse from correctly using the Java8 source/target version that we
have specified (which creates numerous compilation errors in Eclipse)

Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-11-28 14:54:32 -05:00
Peter Somogyi 12efae4829 HBASE-19351 Deprecated is missing in Table implementations
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-28 11:18:07 -08:00
Chia-Ping Tsai 8b6f305ac7 HBASE-19342 fix TestTableBasedReplicationSourceManagerImpl#testRemovePeerMetricsCleanup 2017-11-28 18:12:55 +08:00
Balazs Meszaros ed16667208
HBASE-19328 Remove asked if splittable log messages
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-11-27 21:07:59 -08:00
binlijin a62010c9e7 HBASE-19338 Performance regression in RegionServerRpcQuotaManager to get ugi 2017-11-28 09:58:39 +08:00
Apekshit Sharma 0384561423 HBASE-19335 Fix waitUntilAllRegionsAssigned(). Ignore assignments to killed servers and when region state != OPEN.
Update timeouts for TestRegionObserverInterface.
Reason: There are ~10 tests there, each with 5 min individual timeout. Too much. The test class is labelled MediumTests,
let's used that with our standard CategoryBasedTimeout. 3 min per test function should be enough even on slower Apache machines.
2017-11-27 16:25:54 -08:00
Balazs Meszaros 95e4f059a3 HBASE-18601 Update Htrace to 4.2 (addendum)
- TraceTree fix

Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-11-28 06:59:30 +08:00
Josh Elser e42d20f8dd HBASE-19318 Use the PB service interface as the judge of whether some security feature exists
Hard-coded checks on HBase implementations (e.g. AccessController and
VisibilityController) preclude custom implementations of authorization
and visibility labels (Apache Ranger).

Signed-off-by: Ted Yu <tedyu@apache.org>
2017-11-27 17:49:41 -05:00
Apekshit Sharma 96e63ac7b8 HBASE-19319 Fix bug in synchronizing over ProcedureEvent
Also moves event related functions (wake/wait/suspend) from ProcedureScheduler to ProcedureEvent class
2017-11-27 12:06:07 -08:00
Guangxu Cheng 44bd94721c HBASE-19325 Pass a list of server name to postClearDeadServers
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-11-28 02:53:19 +08:00
Vasudevan 6ac6ae3fa2 HBASE-19092 Make Tag IA.LimitedPrivate and expose for CPs (Ram)
Signed-off-by: Chia-ping Tsai, Anoop Sam John, Stack
2017-11-24 12:38:42 +05:30
Reid Chan 2838cf3e05 HBASE-18309 Support multi threads in CleanerChore
Signed-off-by: Yu Li <liyu@apache.org>
2017-11-24 13:36:55 +08:00
Chia-Ping Tsai 66e650c39b HBASE-19266 TestAcidGuarantees should cover adaptive in-memory compaction 2017-11-24 00:19:06 +08:00
Guanghao Zhang ca6e7e68f4 HBASE-16868 Add a replicate_all flag to avoid misuse the namespaces and table-cfs config of replication peer
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-11-23 15:08:03 +08:00
Gary Helmling 135bb5583b HBASE-19332 DumpReplicationQueues misreports total WAL size 2017-11-22 16:40:53 -08:00
Josh Elser 4e387a948f HBASE-19317 Set a high NodeManager max disk utilization if not already set
This avoids the situation where the build machine has sufficient disk
space (a few GB's at most) to run an HBase test, but the default YARN
configuration would preclude the NM's from starting correctly. This
should eliminate a trivial source of build flakiness based on the host
machines being used.

Signed-off-by: Ted Yu <tedyu@apache.org>
2017-11-22 17:23:53 -05:00
Michael Stack 3c24c6eda6
HBASE-19122 preCompact and preFlush can bypass by returning null scanner; shut it down
Removed TestHRegion#testMemstoreSizeWithFlushCanceling test.
CPs are not able to cancel a flush any more so this test was
failing; removed it (It was testing memory accounting kept
working across a cancelled flush).

Signed-off-by: Michael Stack <stack@apache.org>
2017-11-22 08:02:09 -08:00
zhangduo 9c29a6211b HBASE-19311 Promote TestAcidGuarantees to LargeTests and start mini cluster once to make it faster 2017-11-22 14:10:17 +08:00
Apekshit 8f0f820f22 HBASE-19291 Use common header and footer for JSP pages
Use header and footer in our *.jsp pages to avoid unnecessary redundancy (copy-paste of code)

Misc edits:
- Due to redundancy, new additions make it to some places but not others. For eg there are missing links to "/logLevel", "/processRS.jsp" in few places.
- Fix processMaster.jsp wrongly pointing to rs-status instead of master-status (probably due to copy paste from processRS.jsp)
- Deleted a bunch of extraneous "</a>" in processMaster.jsp & processRS.jsp
- Added missing </div> tag in snapshot.jsp
- Deleted fossils of html5shiv.js. It's uses and the js itself were deleted in the commit "819aed4ccd073d818bfef5931ec8d248bfae5f1f"
- Fixed wrongly matched heading tags
- Deleted some unused variables

Tested:
Ran standalone cluster and opened each page to make sure it looked right.

Sidenote:
Looks like HBASE-3835 started the work of converting from jsp to jamon, but the work didn't finish. Now we have a mix of jsp and jamon. Needs reconciling, but later.
2017-11-21 13:56:00 -08:00
Peter Somogyi bcd367e293
HBASE-19315 Incorrect snapshot version is used for 2.0.0-beta-1
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-21 10:41:50 -08:00
anoopsamjohn f7331f9ceb HBASE-19301 Provide way for CPs to create short circuited connection with custom configurations. 2017-11-21 19:06:39 +05:30
Guanghao Zhang c978f8ab23 HBASE-19293 Support add a disabled state replication peer directly 2017-11-21 15:37:33 +08:00
Michael Stack 719a935b9d
HBASE-19123 Purge 'complete' support from Coprocesor Observers 2017-11-20 22:19:06 -08:00
tedyu b554bbc65e HBASE-19288 Intermittent test failure in TestHStore.testRunDoubleMemStoreCompactors 2017-11-19 20:23:49 -08:00
Apekshit Sharma e0c4f374b5 HBASE-19114 Split out o.a.h.h.zookeeper from hbase-server and hbase-client
- Moved DrainingServerTracker and RegionServerTracker to hbase-server:o.a.h.h.master.
- Moved SplitOrMergeTracker to oahh.master (because it depends on a PB)
- Moving hbase-client:oahh.zookeeper.*  to hbase-zookeeper module.  After HBASE-19200, hbase-client doesn't need them anymore (except 3 classes).
- Renamed some classes to use a consistent naming for classes - ZK instead of mix of ZK, Zk , ZooKeeper. Couldn't rename following public classes: MiniZooKeeperCluster, ZooKeeperConnectionException. Left RecoverableZooKeeper for lack of better name. (suggestions?)
- Sadly, can't move tests out because they depend on HBaseTestingUtility (which defeats part of the purpose - trimming down hbase-server tests. We need to promote more use of mocks in our tests)
2017-11-17 13:23:28 -08:00
Guangxu Cheng 63b02a072a HBASE-19274 Log IOException when unable to determine the size of committed file
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-11-17 03:00:59 -08:00
huzheng 57291108ed HBASE-19252 Move the transform logic of FilterList into transformCell() method to avoid extra ref to question cell 2017-11-17 10:42:58 +08:00
Guanghao Zhang 8b30adb834 HBASE-18911 Unify Admin and AsyncAdmin's methods name 2017-11-17 09:50:41 +08:00
Michael Stack 9fecb3b2c8
HBASE-19268 Enable Replica tests that were disabled by Proc-V2 AM in HBASE-14614 2017-11-16 13:49:11 -08:00
Michael Stack 2c1ded5425 HBASE-19276 RegionPlan should correctly implement equals and hashCode
Andrew's patch that adds equals and hash but revamping compare also.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-11-16 13:27:52 -08:00
Umesh Agashe e9612e6c89
HBASE-18964 Deprecated RowProcessor and Region#processRowsWithLocks() methods that take RowProcessor as an argument
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-15 23:40:20 -08:00
Michael Stack 853ab2f943
HBASE-19270 Reenable TestRegionMergeTransactionOnCluster#testMergeWithReplicas disable by HBASE-14614 2017-11-15 23:38:07 -08:00
Michael Stack 99fbe73552
HBASE-18356 Enable TestFavoredStochasticBalancerPickers#testPickers that was disabled by Proc-V2 AM in HBASE-14614
Rebase/Fixup

Signed-off-by: Thiruvel Thirumoolan <thiruvel at yahoo-inc dot com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-15 23:36:23 -08:00
Michael Stack 8874890798
HBASE-19278 Reenable cleanup in test teardown in TestAccessController3 disabled by HBASE-14614
Remove a few unused imports.

Remove TestAsyncRegionAdminApi#testOffline, a test for a condition that
no longer exists (no offlining supported in hbase2).

M hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController3.java
 Uncomment cleanup called in test teardown.
2017-11-15 23:27:28 -08:00
zhangduo bc3542c0fb HBASE-19251 Merge RawAsyncTable and AsyncTable 2017-11-16 14:37:51 +08:00
Michael Stack e063b231da HBASE-18357 Enable disabled tests in TestHCM that were disabled by Proc-V2 AM in HBASE-14614
Restore testRegionCaching and testMulti to working state (required
fixing move procedure and looking for a new exception).

testClusterStatus is broke because multicast is broken.
2017-11-15 18:40:00 -08:00
Guanghao Zhang d885e2232d HBASE-19009 implement modifyTable and enable/disableTableReplication for AsyncAdmin 2017-11-16 07:19:34 +08:00
Sean Busbey 9c85d0017f HBASE-19248 Move tests that need to look at Connection internals to test of said internals.
Signed-off-by: zhangduo <zhangduo@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-11-15 08:40:03 -06:00
anoopsamjohn a1d86d90ba HBASE-19235 CoprocessorEnvironment should be exposed to CPs. 2017-11-15 14:51:04 +05:30
Umesh Agashe 7139113fde
HBASE-18963 Removed MultiRowMutationProcessor and modified mutateRows...() to use batchMutate() with atomic set to true 2017-11-14 21:45:01 -08:00
Lars Hofhansl 3d4fd90947 HBASE-12091 Optionally ignore edits for dropped tables for replication. 2017-11-14 17:16:56 -08:00
zhangduo ffb8c641d9 HBASE-19243 Start mini cluster once before class for TestFIFOCompactionPolicy 2017-11-14 15:40:30 +08:00
Andrew Purtell efe116f11a HBASE-19250 TestClientClusterStatus is flaky 2017-11-13 18:02:21 -08:00
Yi Liang 57aa6b32a2 HBASE-19237: TestMaster.testMasterOpsWhileSplitting fails
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-11-13 17:55:30 -08:00
Michael Stack 18e484c6c4 HBASE-18601: Update Htrace to 4.2; ADDENDUM; hack failing TestHTraceHooks test 2017-11-13 10:39:22 -08:00
Tamas Penzes 7a69ebc73e HBASE-18601: Update Htrace to 4.2
Updated HTrace version to 4.2
Created TraceUtil class to wrap htrace methods. Uses try with resources.

Signed-off-by: Balazs Meszaros <balazs.meszaros@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-13 10:38:36 -08:00
zhangduo 8a5273f38c HBASE-19241 Improve javadoc for AsyncAdmin and cleanup warnings for the implementation classes 2017-11-13 17:00:14 +08:00
Mike Drob cd681f26bc HBASE-19240 more error-prone results 2017-11-12 19:47:38 -06:00
Guanghao Zhang bc8048cf6c HBASE-19035 Miss metrics when coprocessor use region scanner to read data 2017-11-12 15:16:37 +08:00
anoopsamjohn 3ee76f8573 HBASE-19187 Remove option to create on heap bucket cache. 2017-11-11 12:41:09 +05:30
zhangduo 30f55f2316 HBASE-19200 Make hbase-client only depend on ZKAsyncRegistry and ZNodePaths
- Removes zookeeper connection from ClusterConnection
- Deletes class ZooKeeperKeepAliveConnection
- Removes Registry, ZooKeeperRegistry, and RegistryFactory
2017-11-10 10:09:04 -08:00
Michael Stack b41dae3149 HBASE-18423 Fix TestMetaWithReplicas 2017-11-09 15:00:53 -08:00
Guangxu Cheng b6b4cd40cd HBASE-19199 RatioBasedCompactionPolicy#shouldPerformMajorCompaction() always return true when only one file needs to compact
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-11-09 12:38:45 -08:00
Umesh Agashe 85227d6a72
HBASE-18962 Support atomic (all or none) BatchOperations through batchMutate()
Signed-off-by: Apekshit Sharma <appy@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-09 12:15:16 -08:00
Yi Liang 07b0ac4161 HBASE-19127: Set State.SPLITTING, MERGING, MERGING_NEW, SPLITTING_NEW properly in RegionStatesNode
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-09 11:34:53 -08:00
Michael Stack 0aafbe7aa9 HBASE-19165 TODO Handle stuck in transition: rit=OPENING, location=ve0538.... 2017-11-09 10:32:42 -08:00
Zach York 77e7c5ff27 HBASE-18624 Added support for clearing BlockCache based on tablename
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-11-09 04:03:35 +08:00
Mike Drob 0d4f33ca27 HBASE-19195 error-prone fixes for client, mr, and server 2017-11-08 12:04:46 -06:00
Apekshit Sharma a6d8023ff5 HBASE-19128 Purge Distributed Log Replay from codebase, configurations, text; mark the feature as unsupported, broken. 2017-11-07 17:48:52 -08:00
Umesh Agashe fa3cc6c3bc
HBASE-18961 doMiniBatchMutate() is split into smaller member methods of BatchOperation and it's sub-classes
There is no functionality change except for below:
* Variable lastIndexExclusive was getting incremented while locking rows corresponding to input
  operations. As a result when getRowLockInternal() method throws TimeoutIOException only operations
  in range [nextIndexToProcess, lastIndexExclusive) was getting marked as FAILED before raising
  exception up the call stack. With these changes all operations are getting marked as FAILED.
* Cluster Ids of first mutation is used consistently for entire batch. Previous behavior was to use
  cluster ids of first mutation in a mini-batch

Signed-off-by: Michael Stack <stack@apache.org>
2017-11-07 10:01:28 -08:00
QilinCao 1110910b3a HBASE-19103 Add BigDecimalComparator for filter
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-11-07 03:30:43 -08:00
Michael Stack 57c0fb2561 HBASE-19198 TestIPv6NIOServerSocketChannel fails; unable to bind 2017-11-06 21:21:06 -08:00
Michael Stack f13cf56f1c
HBASE-19197 Move version on branch-2 from 2.0.0-alpha4 to 2.0.0-beta-1.SNAPSHOT 2017-11-06 20:46:38 -08:00
Guanghao Zhang 40dac699bd HBASE-19186 Unify to use bytes to show size in master/rs ui 2017-11-07 10:07:52 +08:00
Josh Elser f4a4144f35 HBASE-19111 Add CellUtil#isPut and deprecate methods returning/expecting non public-api data
KeyValue.Type, and its corresponding byte value, are not public API. We
shouldn't have methods that are expecting them. Added a basic sanity
test for isPut and isDelete.

Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2017-11-06 15:37:16 -05:00
Mike Drob cfddfcf23c HBASE-19160 expose CellComparator as IA.Public 2017-11-06 10:15:55 -06:00
Guanghao Zhang 47c614c706 HBASE-18950 Remove Optional parameters in AsyncAdmin interface 2017-11-06 20:41:20 +08:00
anastas 80a9bf0eb6 HBASE-18232 Jumbo Chunks support for the future CellChunkMap index usage 2017-11-05 15:29:48 +02:00
zhangduo 143e4949fe HBASE-19095 Add CP hooks in RegionObserver for in memory compaction 2017-11-05 20:28:11 +08:00
Mike Drob 33ae6dce42 HBASE-18983 fixes from update error-prone to 2.1.1 2017-11-04 21:29:48 -05:00
Chia-Ping Tsai d592b29619 HBASE-19131 Add the ClusterStatus hook and cleanup other hooks which can be replaced by ClusterStatus hook 2017-11-05 09:56:04 +08:00
Chia-Ping Tsai b464c1eb35 HBASE-19027 Honor the CellComparator of ScanInfo in scanning over a store 2017-11-05 01:17:55 +08:00