25ff9d0bbf
HBASE-18503 Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor
2017-08-24 13:03:38 +08:00
12f2b02a80
Revert "HBASE-18503 Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor"
...
Wrong author information
This reverts commit b03348630c
2017-08-24 13:02:11 +08:00
b03348630c
HBASE-18503 Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor
2017-08-24 12:56:58 +08:00
26e6c2ceb4
HBASE-17442 Move most of the replication related classes from hbase-client to new hbase-replication package. (Guanghao Zhang).
...
Change-Id: Ie0e24cc617ab4bf56de8b1747062d1b78a5d4669
2017-08-23 14:41:58 -07:00
6b21f8881b
HBASE-18575 [AMv2] Fixed and enabled TestRestartCluster#testRetainAssignmentOnRestart on master
...
* Fixed ServerCrashProcedure to set forceNewPlan to false for instances AssignProcedure. This enables balancer to find most suitable target server
* Fixed and enabled TestRestartCluster#testRetainAssignmentOnRestart on master
* Renamed method ServerName@isSameHostnameAndPort() to isSameAddress()
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-23 10:10:56 -07:00
2dda371209
HBASE-17614: Move Backup/Restore into separate module (Vladimir Rodionov)
...
Signed-off-by: Josh Elser <elserj@apache.org>
2017-08-23 12:40:57 -04:00
6c0e219dd4
HBASE-18632 TestMultiParallel#testFlushCommitsWithAbort fails in master branch
2017-08-23 09:35:08 -07:00
04f114b85c
HBASE-18532 Improve cache related stats rendered on RS UI
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-23 07:53:55 -07:00
dcd3e9abf4
HBASE-18655 TestAsyncClusterAdminApi2 failing sometimes
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-22 21:24:10 -07:00
3be5e8ce00
HBASE-18614 Setting BUCKET_CACHE_COMBINED_KEY to false disables stats on RS UI
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-22 18:01:43 -07:00
51d458872d
HBASE-12349 Add custom error-prone module
2017-08-22 16:38:17 -05:00
5b42d81950
HBASE-18628 Fix event pre-emption in ZKPermWatcher
...
Instead of using an Atomic Reference to data and aborting when we detect
that new data comes in, use the native cancellation/pre-emption features
of Java Future.
2017-08-22 11:19:38 -05:00
05f07f692b
HBASE-18645 Loads of tests timing out.... Revert "HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are unreachable"
...
This reverts commit 1ab6882f62
2017-08-22 01:41:18 -07:00
5895538a32
HBASE-18103 [AMv2] Changed master to throw YouAreDeadException when it receives regionServerReport() with incorrect region assignment. Added test to verify rogue region server behavior.
...
Behavior prior to these changes is to call expireServer(), log exception and suppress it. These changes will result in RS receiving the YouAreDeadException and treating it as a fatal error. This 'fail fast' approach will help us stabilize the code. This behavior can be reconsidered later if necessary.
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-21 17:23:10 -07:00
bf343da4a4
HBASE-18627 Fix TestRegionServerReadRequestMetrics (Chia-Ping Tsai)
2017-08-21 15:38:15 -07:00
078a6be7d0
HBASE-18634 HBASE-Fix client.TestClientClusterStatus (Chia-Ping Tsai)
...
This is a revert of a revert; i.e. a reapply just to fix commit message.
This reverts commit fbbae8774c
2017-08-21 14:38:58 -07:00
fbbae8774c
Revert "HBASE-Fix client.TestClientClusterStatus (Chia-Ping Tsai)"
...
Revert because missing JIRA number
This reverts commit 7b3291fa8b
2017-08-21 14:38:22 -07:00
7b3291fa8b
HBASE-Fix client.TestClientClusterStatus (Chia-Ping Tsai)
2017-08-21 14:21:30 -07:00
70d48b113e
HBASE-18623 Frequent failed to parse at EOF warnings from WALEntryStream
2017-08-21 09:29:24 -07:00
56f9e1a60a
HBASE-18471 The DeleteFamily cell is skipped when StoreScanner seeks to next column
2017-08-19 02:15:12 +08:00
e9bafeb091
HBASE-18572 Delete can't remove the cells which have no visibility label
2017-08-19 01:55:45 +08:00
e2532ecd1e
HBASE-18617 FuzzyRowKeyFilter should not modify the filter pairs (vinisha)
2017-08-18 09:10:20 -07:00
2af2b4c040
HBASE-18489 Addendum revert protected to private, give shaded import a separated block
2017-08-18 10:54:49 +08:00
1ab6882f62
HBASE-14498 Master stuck in infinite loop when all Zookeeper servers are unreachable
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-17 19:02:39 -07:00
75a6b36849
HBASE-18375: Fix the bug where the pool chunks from ChunkCreator are deallocated and not returned to pool, because there is no reference to them
2017-08-17 18:23:19 +03:00
092dc6de84
HBASE-18608 AsyncConnection should return AsyncAdmin interface instead of the implemenation
2017-08-17 09:47:39 +08:00
473446719b
HBASE-18511 Default no regions on master
...
Changes the configuration hbase.balancer.tablesOnMaster from list of
table names to instead be a boolean; true if master carries
tables/regions and false if it does not.
Adds a new configuration hbase.balancer.tablesOnMaster.systemTablesOnly.
If true, hbase.balancer.tablesOnMaster is considered true but only
system tables are put on the master.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Master was claiming itself active master though it had stopped. Fix
the activeMaster flag. Set it to false on exit.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java
Add new configs and convenience methods for getting current state of
settings.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java
Move configs up into super Interface and now the settings mean
different, remove the no longer needed processing.
2017-08-16 08:39:36 -07:00
acf9b87dca
HBASE-18493 [AMv2] Skipped re-assignment of regions on crashed server through AssignmentManager.checkIfShouldMoveSystemRegionAsync() as those regions are handled by ServerCrashProcedure
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-16 08:12:05 -07:00
4c74a73d57
HBASE-18553 Expose scan cursor for asynchronous scanner
2017-08-16 21:04:57 +08:00
2a9cdd5e75
HBASE-17994 Add async client test to Performance Evaluation tool
2017-08-16 14:05:07 +08:00
9da4e6906e
HBASE-18251 Remove unnecessary traversing to the first and last keys in
...
the CellSet (Toshihoro Suzuki)
2017-08-16 11:06:31 +05:30
b0878184a3
HBASE-18437 Revoke access permissions of a user from a table does not work as expected
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-08-15 22:29:16 -07:00
59ffb6119b
HBASE-18598 AsyncNonMetaRegionLocator use FIFO algorithm to get a candidate locate request
2017-08-16 13:08:40 +08:00
665fd0d07e
HBASE-18424 Fix TestAsyncTableGetMultiThreaded
...
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-08-16 11:29:34 +08:00
5280c100ff
HBASE-18587 Fix flaky TestFileIOEngine
...
This short circuits reads and writes with 0 length and also removes flakiness in TestFileIOEngine
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-15 14:57:10 -07:00
2b88edfd8d
HBASE-18581 Removed dead code and some tidy up work in BaseLoadBalancer
...
* calls to methods getLowestLocalityRegionServer() & getLeastLoadedTopServerForRegion() got removed in HBASE-18164
* call to calculateRegionServerLocalities() got removed in HBASE-15486
* Some other minor improvements
Change-Id: Ib149530d8d20c019b0891c026e23180e260f59db
Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-08-15 14:55:52 -07:00
310934d060
HBASE-18544 Move the HRegion#addRegionToMETA to TestDefaultMemStore
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-15 14:52:33 -07:00
70c4f78ce0
HBASE-18599 Add missing @Deprecated annotations
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-15 10:44:50 -07:00
effd1093b5
HBASE-17064 Add TaskMonitor#getTasks() variant which accepts type selection
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-15 09:45:19 -07:00
d37266f63c
HBASE-2631 Decide between InMB and MB as suffix for field names in ClusterStatus objects
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-15 21:42:31 +08:00
0e32869f01
HBASE-18533 Expose BucketCache values to be configured
...
Before this commit, BucketCache always used the default values.
This commit adds the ability to configure these values.
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-14 13:27:26 -07:00
0b26ccdaa1
HBASE-18303 Clean up @Parameter boilerplate
2017-08-14 14:23:24 -05:00
096dac2e83
HBASE-18522 Add RowMutations support to Batch
2017-08-14 10:39:46 -07:00
e2b797be39
HBASE-18528 DON'T allow user to modify the passed table/column descriptor
2017-08-14 14:02:30 +08:00
c6bf4d5192
HBASE-18566 [RSGROUP]Log the client IP/port of the rsgroup admin
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-13 20:29:12 -07:00
05e6e56950
HBASE-14135 Merge backup images (Vladimir Rodionov)
2017-08-13 20:55:58 -04:00
c6ac04ab39
HBASE-18271 Shade netty Purge mention of netty-all.
...
Add in hbase-thirdparty hbase-shaded-netty instead.
s/io.netty/org.apache.hadoop.hbase.shaded.io.netty/ everywhere in hbase.
Also set a system property when running tests and when starting
hbase; required by netty so can find the relocation files in the
bundled .so.
2017-08-13 12:46:21 -07:00
923195c39e
HBASE-15511 ClusterStatus should be able to return responses by scope
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-14 01:02:39 +08:00
95e883967c
HBASE-18557: Change splitable to mergeable in MergeTableRegionsProcedure
...
Signed-off-by: Jerry He <jerryjch@apache.org>
2017-08-11 22:45:22 -07:00
aa8f67a148
HBASE-18526 FIFOCompactionPolicy pre-check uses wrong scope (Vladimir Rodionov)
2017-08-11 16:41:40 -07:00
1070888fff
HBASE-18551 [AMv2] UnassignProcedure and crashed regionservers; AMENDMENT -- disable TestAM#testSocketTimeout... mock is insufficent for new processing
2017-08-11 14:20:35 -07:00
71a9a9a944
HBASE-18025 CatalogJanitor should collect outdated RegionStates from the AM
2017-08-11 13:36:38 -05:00
6f44b24860
HBASE-18551 [AMv2] UnassignProcedure and crashed regionservers
...
If an unassign is unable to communicate with its target server,
expire the server and then wait on a signal from ServerCrashProcedure
before proceeding. The unassign has lock on the region so no one else
can proceed till we complete. We prevent any subsequent assign from
running until logs have been split for crashed server.
In AssignProcedure, do not assign if table is DISABLING or DISABLED.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
Change remoteCallFailed so it returns boolean on whether implementor
wants to stay suspended or not.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
Doc. Also, if we are unable to talk to remote server, expire it and
then wait on SCP to wake us up after it has processed logs for failed
server.
2017-08-11 07:16:33 -07:00
cabdbf181a
HBASE-18500 Performance issue: Don't use BufferedMutator for HTable's put method
2017-08-11 16:33:35 +08:00
679f34e881
HBASE-18469 Correct RegionServer metric of totalRequestCount
2017-08-11 14:13:18 +08:00
4dd24c52b8
HBASE-17125 Inconsistent result when use filter to read data
2017-08-11 10:58:00 +08:00
efd211debd
HBASE-18024 HRegion#initializeRegionInternals should not re-create .hregioninfo file when the region directory no longer exists
2017-08-10 17:56:17 -05:00
e4ba404a5a
Revert "HBASE-18551 [AMv2] UnassignProcedure and crashed regionservers"
...
This reverts commit 2dd75d10f8
2017-08-10 14:59:52 -07:00
e98b38bf6c
HBASE-18560 Fixed master.assignment.TestAssignmentManager hangs on master and it shows up in flaky list
2017-08-10 14:58:52 -07:00
2dd75d10f8
HBASE-18551 [AMv2] UnassignProcedure and crashed regionservers
...
If an unassign is unable to communicate with its target server,
expire the server and then wait on a signal from ServerCrashProcedure
before proceeding. The unassign has lock on the region so no one else
can proceed till we complete. We prevent any subsequent assign from
running until logs have been split for crashed server.
In AssignProcedure, do not assign if table is DISABLING or DISABLED.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionTransitionProcedure.java
Change remoteCallFailed so it returns boolean on whether implementor
wants to stay suspended or not.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/UnassignProcedure.java
Doc. Also, if we are unable to talk to remote server, expire it and
then wait on SCP to wake us up after it has processed logs for failed
server.
2017-08-10 14:53:35 -07:00
ded0842caf
HBASE-18398: Snapshot operation fails with FileNotFoundException
2017-08-10 14:20:08 -07:00
d5f34adcdb
HBASE-18543 Disabled test TestMasterFailover
...
This test as it is written currently will not work with AMv2. This needs to be re-written after HBASE-18511 is committed. Disabled the test and update JIRA to re-enable it with dependency on HBASE-18511.
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-10 11:01:27 -07:00
624652373e
HBASE-18489 Expose scan cursor in RawScanResultConsumer
2017-08-10 10:11:40 +08:00
d0941127d4
HBASE-18248 Warn if monitored RPC task has been tied up beyond a configurable threshold
2017-08-09 18:16:38 -07:00
67eddf5874
HBASE-18525 [AMv2] Fixed test TestAssignmentManager#testSocketTimeout on master branch
2017-08-09 10:15:37 -07:00
f314b5911b
HBASE-18492 [AMv2] Embed code for selecting highest versioned region server for system table regions in AssignmentManager.processAssignQueue()
...
* Modified AssignmentManager.processAssignQueue() method to consider only highest versioned region servers for system table regions when
destination server is not specified for them. Destination server is retained, if specified.
* Modified MoveRegionProcedure to allow null value for destination server i.e. moving a region from specific source server to non-specific/ unknown
destination server (picked by load-balancer) is supported now.
* Removed destination server selection from HMaster.checkIfShouldMoveSystemRegionAsync(), as destination server will be picked by load balancer
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-08 14:02:11 -07:00
03390684cc
Revert "HBASE-18511 Default no regions on master"
...
This reverts commit a8e0267c00
2017-08-08 13:37:56 +08:00
a8e0267c00
HBASE-18511 Default no regions on master
2017-08-08 12:11:02 +08:00
fd76eb39d7
HBASE-18502 Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
2017-08-07 11:26:15 +08:00
a7014ce46c
HBASE-18515 Introduce Delete.add as a replacement for Delete#addDeleteMarker
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-08-07 11:05:23 +08:00
5915d73a70
HBASE-18485 Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner
2017-08-07 10:35:19 +08:00
637f7abf0b
HBASE-18520 Add jmx value to determine true Master Start time
...
This is to determine how long it took in total for the master to start and finish initializing.
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-08-05 22:32:33 -07:00
62deb8172e
HBASE-18516 Removed dead code in ServerManager resulted mostly from AMv2 refactoring
...
* Call to methods sendRegionOpen(), isServerReachable(), removeRequeuedDeadServers(), getRequeuedDeadServers() got removed in HBASE-14614
* Call to method ServerManager.sendFavoredNodes() got removed in HBASE-17198
2017-08-04 13:47:59 -07:00
de696cf6b6
HBASE-18231 Deprecate Admin#closeRegion*() commands in favor of Admin#unassign().
...
Other changes:
- Update corresponding tests in TestAdmin2. Removed tests centered around serverName part of old functions.
- Remove dead functions from ProtobufUtil and ServerManager
- Rename closeRegion* functions in HBTU to unassignRegion*
Change-Id: Ib9bdeb185e10750daf652be0bb328306accb73ab
2017-08-02 15:19:06 -07:00
7a6de1bd42
HBASE-17056 Remove checked in PB generated files
...
Selective add of dependency on hbase-thirdparty jars.
Update to READMEs on how protobuf is done (and update to refguide).
Removed all checked in generated protobuf files. They are generated
on the fly now as part of mainline build.
2017-08-02 09:33:20 -07:00
f260f09865
HBASE-18480 The cost of BaseLoadBalancer.cluster is changed even if the rollback is done
2017-08-02 08:48:01 +08:00
ba5e8706de
HBASE-18491 [AMv2] Fail UnassignProcedure if source Region Server is not online.
...
The patch also enables TestServerCrashProcedure.testRecoveryAndDoubleExecutionOnRsWithMeta()
Signed-off-by: Michael Stack <stack@apache.org>
2017-08-01 17:05:00 -07:00
422a57223a
HBASE-18487 Minor fixes in row lock implementation
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-08-01 11:28:02 -07:00
a5db120e60
HBASE-18261 Created RecoverMetaProcedure and used it from ServerCrashProcedure and HMaster.finishActiveMasterInitialization().
...
This procedure can be used from any code before accessing meta, to initialize/ recover meta
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-31 14:25:03 -07:00
331a6cface
HBASE-18475 ensure only non-null procedures are sent to requireTableExclusiveLock
...
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-31 11:05:16 -05:00
95808b4672
HBASE-18374 RegionServer Metrics improvements
2017-07-31 12:42:41 +05:30
df90ba58db
HBASE-18481 The autoFlush flag was not used in PE tool
2017-07-31 10:54:45 +08:00
00c1b56665
HBASE-18465: [AMv2] remove old split region code that is no longer needed
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-30 15:24:58 -05:00
0e9390bd6d
HBASE-18406 Remove NO-OP Method
...
This patch removes start(MasterProcedureEnv) from
ServerCrashProcedure.java which was a misnomer as a no-op. It
did not start anything.
Change-Id: I4e91864ace912e137471bfce03516746c4aff83e
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-30 13:59:53 +01:00
2d06a06ba4
HBASE-15134 Add visibility into Flush and Compaction queues
2017-07-28 12:59:09 +05:30
9a1661832d
HBASE-18362 hbck should not report split replica parent region from meta as errors (Huaxiang Sun)
...
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
2017-07-27 15:58:16 -05:00
3d81f7b9e7
HBASE-18449 Fix client.locking.TestEntityLocks
2017-07-26 20:26:24 +08:00
421029d0c7
HBASE-18434 Address some alerts raised by lgtm.com
...
Signed-off-by: Ramkrishna <ramkrishna.s.vasudevan@intel.com>
2017-07-26 10:00:23 +05:30
2fd8e824d5
HBASE-18054 log when we add/remove failed servers in client (Ali)
2017-07-25 18:53:09 -07:00
746d1b1819
HBASE-18427 minor cleanup around AssignmentManager
...
- unused imports
- superfluous exception in method definitions
Change-Id: I156383b9895fa718fe9d5227003c23bd945cf999
Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-07-25 17:46:04 -07:00
5cd7f630c2
HBASE-18023 Update row threshold warning from 1k to 5k (addendum)
2017-07-25 18:27:53 -04:00
e1cd59bbc4
HBASE-15968 (addendum) revert unrelated PE changing
2017-07-25 15:16:02 +08:00
1ac4152b19
HBASE-15968 New behavior of versions considering mvcc and ts rather than ts only
2017-07-25 15:00:36 +08:00
fabab8c23f
HBASE-18354 Fix TestMasterMetrics that were disabled by Proc-V2 AM in HBASE-14614 (Vladimir Rodionov)
2017-07-24 14:52:04 -07:00
8f006582e3
HBASE-18367 Reduce ProcedureInfo usage
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-24 10:41:03 +01:00
e9d8a7b6d5
HBASE-18107: [AMv2] Remove DispatchMergingRegionsRequest & DispatchMergingRegions
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-23 10:44:34 +01:00
317ce73963
HBASE-18433 Convenience method for creating simple ColumnFamilyDescriptor
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-07-22 23:42:33 +08:00
ec3cb19664
HBASE-15816 Provide client with ability to set priority on Operations
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-07-21 17:12:16 -07:00
890d92a90c
HBASE-17908 Upgrade guava
...
Pull in guava 22.0 by using the shaded version up in new hbase-thirdparty project.
In poms, exclude guava everywhere except on hadoop-common. Do this so
we minimize transitive includes. hadoop-common is needed because hadoop
Configuration uses guava doing preconditions.
Everywhere we used guava, instead use shaded so fix a load of imports.
Stopwatch API changed as did hashing and toStringHelper which is now
in MoreObjects class. Otherwise, minimal changes to come up on 22.0
2017-07-21 15:28:08 +01:00
bc93b6610b
HBASE-16993 BucketCache throw java.io.IOException: Invalid HFile block magic when configuring hbase.bucketcache.bucket.sizes.
2017-07-20 22:59:06 +05:30
01db60d65b
HBASE-18330 NPE in ReplicationZKLockCleanerChore
2017-07-19 15:46:08 -07:00
3574757f74
HBASE-18308 Eliminate the findbugs warnings for hbase-server
2017-07-20 00:35:07 +08:00
f10f8198af
HBASE-16312 update jquery version
...
Upgrade jquery from 1.8.3 to 3.2.1 in hbase-server and hbase-thrift modules
Change-Id: I92d479e9802d954f607ba409077bc98581e9e5ca
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-19 11:44:31 +01:00
6b7ebc019c
HBASE-18390 Sleep too long when finding region location failed
2017-07-19 11:34:57 +08:00
0c2915b48e
HBASE-18377 Error handling for FileNotFoundException should consider RemoteException in openReader()
2017-07-17 20:24:29 -07:00
a9352fe956
HBASE-18366 Fix flaky test TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta (Umesh Agashe)
...
Disabled for now. Will be back here when a more fundamental fix.
2017-07-14 22:41:36 +01:00
353627b39d
HBASE-18229: create new Async Split API to embrace AM v2
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-14 22:25:14 +01:00
9e0f450c0c
HBASE-17922 Clean TestRegionServerHostname for hadoop3.
...
Change-Id: I6f1514b1bc301be553912539e6a4192c2ccc782b
Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-07-13 11:44:18 -07:00
c0725ddff1
HBASE-18344 Introduce Append.addColumn as a replacement for Append.add
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-07-13 20:04:57 +08:00
cb5299ae9b
HBASE-18177 FanOutOneBlockAsyncDFSOutputHelper fails to compile against Hadoop 3
...
Because ClientProtocol::create has API changes between Hadoop 2/3
Signed-off-by: zhangduo <zhangduo@apache.org>
2017-07-12 13:40:05 +08:00
22dce22e06
HBASE-18343 (addendum) Track the remaining unimplemented methods for async admin
2017-07-12 09:32:00 +08:00
c0f743e44f
HBASE-18358 Backport HBASE-18099 'FlushSnapshotSubprocedure should wait for concurrent Region#flush() to finish'
2017-07-11 17:26:22 -07:00
d215cb4950
HBASE-18295 The result contains the cells across different rows
2017-07-12 02:27:29 +08:00
1978b78cdf
HBASE-18343 Track the remaining unimplemented methods for async admin
2017-07-11 14:01:56 +08:00
f8e892d7aa
HBASE-18348 The implementation of AsyncTableRegionLocator does not follow the javadoc
2017-07-11 11:43:35 +08:00
43492d2d3b
HBASE-18267 The result from the postAppend is ignored
2017-07-11 10:30:06 +08:00
7d007eac98
HBASE-17705 Procedure execution must fail fast if procedure is not registered (Vladimir Rodionov)
2017-07-10 09:04:56 -07:00
351703455a
HBASE-18307 Share the same EventLoopGroup for NettyRpcServer, NettyRpcClient and AsyncFSWALProvider at RS side
2017-07-10 21:00:44 +08:00
1ddcc07d65
HBASE-18318 Implement updateConfiguration/stopMaster/stopRegionServer/shutdown methods
2017-07-10 13:41:19 +08:00
c48bb67123
HBASE-18316 Implement async admin operations for draining region servers
2017-07-09 19:51:59 +08:00
bc8ebc6f72
HBASE-18241 Change client.Table, client.Admin, Region, Store, and HBaseTestingUtility to not use HTableDescriptor or HColumnDescriptor
2017-07-08 16:54:25 +08:00
7f93729782
HBASE-18317 Implement async admin operations for Normalizer/CleanerChore/CatalogJanitor
2017-07-08 10:55:10 +08:00
b0a5fa0c2a
HBASE-18319 Implement getClusterStatus/getRegionLoad/getCompactionState/getLastMajorCompactionTimestamp methods
2017-07-07 16:21:45 +08:00
4fe7385767
HBASE-18083 Make large/small file clean thread number configurable in HFileCleaner
2017-07-07 14:07:23 +08:00
6786b2b63e
Revert "HBASE-17056 Remove checked in PB generated files Selective add of dependency on"
...
Revert for now. Build unstable and some interesting issues around
CLASSPATH
This reverts commit df93c13fd2
2017-07-06 21:58:32 -07:00
75d2eca8ac
HBASE-17931 Assign system tables to servers with highest version
2017-07-06 17:35:54 +08:00
50bb045723
HBASE-18002 Investigate why bucket cache filling up in file mode in an
...
exisiting file is slower (Ram)
2017-07-06 11:20:00 +05:30
df93c13fd2
HBASE-17056 Remove checked in PB generated files Selective add of dependency on
...
hbase-thirdparty jars. Update to READMEs on how protobuf is done (and update to
refguide) Removed all checked in generated protobuf files. They are generatedon
the fly now as part of mainline build.
2017-07-05 20:57:11 -07:00
c5abb6cabb
Revert "HBASE-14070 - Core HLC"
...
Revert a push too-early
This reverts commit 9fe94c1169
2017-07-05 20:11:05 -07:00
172c662034
HBASE-18325 Disable flakey TestMasterProcedureWalLease
2017-07-05 20:10:43 -07:00
9fe94c1169
HBASE-14070 - Core HLC
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-05 16:51:02 -07:00
b71509151e
HBASE-17201 Edit of HFileBlock comments and javadoc
2017-07-05 13:32:27 -07:00
05e3f394e2
HBASE-18301 Enable TestSimpleRegionNormalizerOnCluster#testRegionNormalizationMergeOnCluster that was disabled by Proc-V2 AM in HBASE-14614 (Stephen Yuan Jiang)
2017-07-05 09:56:30 -07:00
4453472282
HBASE-18312 Ineffective handling of FileNotFoundException in FileLink.tryOpen()
2017-07-05 08:24:37 -07:00
2843214857
HBASE-18010: CellChunkMap integration into CompactingMemStore. Continuation of the previous commit
2017-07-05 12:56:45 +03:00
8ac4308411
HBASE-18010: CellChunkMap integration into CompactingMemStore. CellChunkMap usage is currently switched off by default. New tests are included. Review comments addressed.
2017-07-05 12:35:21 +03:00
e71e5ece88
HBASE-18297 Provide a AsyncAdminBuilder to create new AsyncAdmin instance
2017-07-05 09:18:02 +08:00
63607800cd
HBASE-18310 LoadTestTool unable to write data
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-07-04 13:41:32 -07:00
193a980338
HBASE-15943 Add page displaying JVM process metrics
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-03 21:35:27 -07:00
f2731fc241
HBASE-18264 Update pom plugins
...
Update plugins in main and subprojects
Unified versions to use variable instead of direct values
Affected plugins:
- apache-rat-plugin 0.11 -> 0.12
- asciidoctor-maven-plugin 1.5.2.1 -> 1.5.5
- asciidoctorj-pdf 1.5.0-alpha.6 -> 1.5.0-alpha.15
- build-helper-maven-plugin 1.9.1 -> 3.0.0
- buildnumber-maven-plugin 1.3 -> 1.4
- exec-maven-plugin 1.2.1/1.4.0 -> 1.6.0
- extra-enforcer-rules 1.0-beta-3 -> 1.0-beta-6
- findbugs-maven-plugin 3.0.0 -> 3.0.4
- jamon-maven-plugin 2.4.1 -> 2.4.2
- maven-bundle-plugin 2.5.3 -> 3.3.0
- maven-compiler-plugin 3.2/3.5.1 -> 3.6.1
- maven-eclipse-plugin 2.9 -> 2.10
- maven-shade-plugin 2.4.1 -> 3.0.0
- maven-surefire-plugin 2.18.1 -> 2.20
- maven-surefire-report-plugin 2.7.2 -> 2.20
- scala-maven-plugin 3.2.0 -> 3.2.2
- spotbugs 3.1.0-RC1 -> 3.1.0-RC3
- wagon-ssh 2.2 -> 2.12
- xml-maven-plugin 1.0 -> 1.0.1
- maven-assembly-plugin 2.4 -> 2.6(inherited)
- maven-dependency-plugin 2.4 -> 2.10 (inherited)
- maven-enforcer-plugin 1.3.1 -> 1.4.1 (inherited)
- maven-javadoc-plugin 2.10.3 -> 2.10.4 (inherited)
- maven-resources-plugin 2.7 (inherited)
- maven-site-plugin 3.4 -> 3.5.1 (inherited)
Change-Id: I84539f555be498dff18caed1e3eea1e1aeb2143a
Signed-off-by: Michael Stack <stack@apache.org>
2017-07-03 19:42:46 -07:00
14f0423b58
HBASE-18283 Provide a construct method which accept a thread pool for AsyncAdmin
2017-07-04 09:51:41 +08:00
fc973d0918
HBASE-17995 improve log messages during snapshot tests.
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-30 09:42:14 -05:00
74c5742024
HBASE-18288 Declared dependency on specific javax.ws.rs.
...
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.org>
2017-06-30 08:41:50 -05:00
21653c31d9
HBASE-16585 Rewrite the delegation token tests with Parameterized pattern
2017-06-30 20:40:23 +08:00
92f33ad076
Revert "HBASE-18229: create new Async Split API to embrace AM v2"
...
TestShell is failing.
This reverts commit 5be05e90d4
2017-06-30 03:30:01 -07:00
73c225a071
HBASE-16192 Fix the potential problems in TestAcidGuarantees (Colin Ma)
2017-06-30 03:16:46 -07:00
5be05e90d4
HBASE-18229: create new Async Split API to embrace AM v2
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-29 16:20:18 -07:00
07c1e18a55
HBASE-17982 correct spelling error of 'occured'
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-29 15:09:49 -07:00
9189b88647
HBASE-18292 Fixed flaky test hbase.master.locking.TestLockProcedure#testLocalMasterLockRecovery
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-28 21:32:46 -07:00
038d7e8984
HBASE-18278 Enable and Fix for unit test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-28 16:01:55 -07:00
aef674264e
HBASE-18281 created private static pattern matcher for performance
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-06-28 11:32:10 -07:00
8da6f069c3
HBASE-18164 Fast locality computation in balancer - addendum handles NaN
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-06-27 14:57:53 -05:00
141482512a
Revert "HBASE-18164 Fast locality computation in balancer - addendum handles NaN"
...
This reverts commit 35693f0583
2017-06-27 14:57:53 -05:00
293cb87d52
HBASE-18161 Incremental Load support for Multiple-Table HFileOutputFormat (Densel Santhmayor)
2017-06-27 12:31:55 -07:00
389e142eae
HBASE-18265 Correct the link unuseful in regionServer's region state UI
...
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-06-27 10:38:46 +08:00
35693f0583
HBASE-18164 Fast locality computation in balancer - addendum handles NaN
...
-Added new LocalityCostFunction and LocalityCandidateGenerator that
cache localities of every region/rack combination and mappings of every
region to its most local server and to its most local rack.
-Made LocalityCostFunction incremental so that it only computes locality
based on most recent region moves/swaps, rather than recomputing the
locality of every region in the cluster at every iteration of the
balancer
-Changed locality cost function to reflect the ratio of:
(Current locality) / (Best locality possible given current cluster)
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-06-26 12:38:43 -07:00
2d781aa15c
HBASE-18234 Revisit the async admin api
2017-06-26 17:27:09 +08:00
0e8e176ebd
HBASE-18023 Log multi-* requests for more than threshold number of rows
...
Signed-off-by: Josh Elser <elserj@apache.org>
2017-06-24 15:23:51 -04:00
96aca6b153
HBASE-18263 Resolve NPE in backup Master UI when accessing procedures.jsp
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-06-24 05:20:05 -07:00
d092008766
HBASE-18221 Switch from pread to stream should happen under HStore's
...
reentrant lock (Ram)
2017-06-23 10:32:29 +05:30
7cc458e129
HBASE-18254 ServerCrashProcedure checks and waits for meta initialized, instead should check and wait for meta loaded
...
After enabling test hbase.master.procedure.TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta, this bug is found in ServerCrashProcedure
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-21 21:57:46 -07:00
3489a1b821
HBASE-18235 LoadBalancer.BOGUS_SERVER_NAME should not have a bogus hostname
...
We deliberately use 'localhost' instead of a bogus hostname for
LoadBalancer.BOGUS_SERVER_NAME so the operation will fail fast.
2017-06-21 14:37:47 -07:00
00f657fbeb
HBASE-18252 Resolve BaseLoadBalancer bad practice warnings
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-06-21 10:12:23 -07:00
83be50c2ab
HBASE-18226 Disable reverse DNS lookup at HMaster and use the hostname provided by RegionServer (Duo Xu)
2017-06-20 21:07:45 -07:00
5b485d14cd
HBASE-17752 Shell command to list snapshot sizes WRT quotas
2017-06-20 14:17:00 -04:00
af466bf722
HBASE-18212 reduce log level for unbuffer warning.
...
In Standalone mode with local filesystem HBase logs Warning message:Failed to invoke 'unbuffer' method in class org.apache.hadoop.fs.FSDataInputStream
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
2017-06-20 01:06:47 -05:00
5224064d4d
HBASE-18164 Fast locality computation in balancer
...
-Added new LocalityCostFunction and LocalityCandidateGenerator that
cache localities of every region/rack combination and mappings of every
region to its most local server and to its most local rack.
-Made LocalityCostFunction incremental so that it only computes locality
based on most recent region moves/swaps, rather than recomputing the
locality of every region in the cluster at every iteration of the
balancer
-Changed locality cost function to reflect the ratio of:
(Current locality) / (Best locality possible given current cluster)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Chia-Ping Tsai <chia7712@gmail.com>
2017-06-20 01:06:47 -05:00
ce1ce728c6
HBASE-18180 Possible connection leak while closing BufferedMutator in TableOutputFormat
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-06-18 19:46:47 -07:00
c6e71f159c
HBASE-18170 Refactor ReplicationSourceWALReaderThread
2017-06-19 09:26:45 +08:00
d49208b056
HBASE-18104 AMv2: Enabled aggregation of RPCs
...
Unit test (TestAssignmentManager) uses mock which always aggregates. So added trace level log message and verified manually on a single node cluster.
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-16 23:53:39 -07:00
b02d3d9ed5
HBASE-18227 Fixed unit test hbase.coprocessor.TestCoprocessorMetrics#testRegionObserverAfterRegionClosed
...
Calling closeRegion() directly on remote server is not supported post-AMv2. Calling unassign() on master
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-16 20:19:38 -07:00
c7a64a8313
HBASE-18225 Avoid toString() on an array
2017-06-16 08:45:31 -07:00
c20d9cb1a2
HBASE-18220 - Addendum as per Duo suggestion for readability (Ram)
2017-06-16 12:34:08 +05:30
020f520d17
HBASE-18220 Compaction scanners need not reopen storefile scanners while
...
trying to switch over from pread to stream (Ram)
2017-06-16 11:03:04 +05:30
dd1d81ef5a
HBASE-18004 getRegionLocations needs to be called once in
...
ScannerCallableWithReplicas#call() (Huaxiang Sun)
2017-06-15 13:41:01 -07:00
c2eebfdb61
HBASE-18166 [AMv2] We are splitting already-split files v2 Address Stephen Jiang reivew comments; ADDENDUM TO FIX COMPILE
2017-06-15 11:40:13 -07:00
f64512bee1
HBASE-18166 [AMv2] We are splitting already-split files v2 Address Stephen Jiang reivew comments
2017-06-15 10:26:03 -07:00
8b36da1108
HBASE-18209 Include httpclient / httpcore jars in build artifacts
2017-06-14 20:09:42 -07:00
50e28d62a6
HBASE-18219 Fix typo in constant HConstants.HBASE_CLIENT_MEAT_REPLICA_SCAN_TIMEOUT
2017-06-14 16:02:38 -07:00
384e308e9f
HBASE-18137 Replication gets stuck for empty WALs
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-06-10 10:30:40 -07:00
eb2dc5d2a5
HBASE-18192: Replication drops recovered queues on region server shutdown
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-06-09 19:52:58 -07:00
e5ea457054
HBASE-17748 Include HBase snapshots in space quotas
...
Introduces a new Chore in the Master which computes the size
of the snapshots included in a cluster. The size of these
snapshots are included in the table's which the snapshot was created
from HDFS usage.
Includes some test stabilization, trying to make the tests more
deterministic by ensuring we observe stable values as we know
that those values are mutable. This should help avoid problems
where size reports are delayed and we see an incomplete value.
2017-06-09 18:43:18 -04:00
7b40f4f3ec
HBASE-18092: Removing a peer does not properly clean up the ReplicationSourceManager state and metrics
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-06-09 08:23:04 -07:00
30817b922e
HBASE-18193 Master web UI presents the incorrect number of regions
2017-06-09 14:44:51 +08:00
61839d7143
HBASE-18195 Removed redundant single quote from start message for HMaster and HRegionServer
...
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-08 22:21:00 -07:00
a558d6c57a
HBASE-18141 Regionserver fails to shutdown when abort triggered during RPC call
2017-06-08 17:20:29 -07:00
112bff4ba0
HBASE-18109: Assign system tables first
...
This issue adds comments and a sort so system tables are queued first
(which will ensure they go out first). This should be good enough
along w/ existing scheduling mechanisms to ensure system/meta get
assigned first.
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-08 13:24:28 -07:00
72cb7d97cc
HBASE-18008 Any HColumnDescriptor we give out should be immutable
2017-06-08 23:26:08 +08:00
3f891a66ca
HBASE-18184 Add hbase-hadoop2-compat jar as MapReduce dependency
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-06-07 14:25:55 -07:00
4a1529c89b
HBASE-18158 Two running in-memory compaction threads may lose data for flushing
2017-06-07 17:57:32 +08:00
1d3252eb59
HBASE-17849 PE tool random read is not totally random (Ram)
2017-06-07 11:28:09 +05:30
929c9dab14
HBASE-18181 Move master branch to version 3.0.0-SNAPSHOT post creation of branch-2
2017-06-06 22:04:39 -07:00
0d0c330401
HBASE-17678 FilterList with MUST_PASS_ONE may lead to redundant cells returned
...
Signed-off-by: tedyu <yuzhihong@gmail.com>
2017-06-06 21:08:12 -07:00
2f1923a823
HBASE-15576 Scanning cursor to prevent blocking long time on ResultScanner.next()
2017-06-07 11:32:04 +08:00
80e15aac21
HBASE-16392 Backup delete fault tolerance (Vladimir Rodionov)
2017-06-06 20:29:13 -07:00
da3c023635
HBASE-18145 The flush may cause the corrupt data for reading
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-06-06 18:02:43 -07:00
858bccfcb8
HBASE-18132 Low replication should be checked in period in case of datanode rolling upgrade (Allan Yang)
2017-06-06 17:21:21 -07:00
1950acc67a
HBASE-9393 Hbase does not closing a closed socket resulting in many CLOSE_WAIT
...
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2017-06-06 12:52:46 -07:00
ee0f148c73
HBASE-18038 Rename StoreFile to HStoreFile and add a StoreFile interface for CP
2017-06-06 20:36:38 +08:00
07c38e7165
HBASE-16549 Added new metrics for AMv2 procedures
...
Following AMv2 procedures are modified to override onSubmit(), onFinish() hooks provided by HBASE-17888 to do
metrics calculations when procedures are submitted and finshed:
* AssignProcedure
* UnassignProcedure
* MergeTableRegionProcedure
* SplitTableRegionProcedure
* ServerCrashProcedure
Following metrics is collected for each of the above procedure during lifetime of a process:
* Total number of requests submitted for a type of procedure
* Histogram of runtime in milliseconds for successfully completed procedures
* Total number of failed procedures
As we are moving away from Hadoop's metric2, hbase-metrics-api module is used for newly added metrics.
Modified existing tests to verify count of procedures.
Signed-off-by: Michael Stack <stack@apache.org>
2017-06-05 17:14:14 -07:00
Chia-Ping Tsai
Chia-Ping Tsai
Apekshit Sharma
Umesh Agashe
Vladimir Rodionov
tedyu
Biju Nair
Michael Stack
Mike Drob
Andrew Purtell
zhangduo
Pankaj Kumar
anastas
Guanghao Zhang
Ramkrishna
Ashish Singhi
Vladimir Rodionov
Zach York
Chun-Hao Tang
Lars Francke
Reid Chan
Deon Huang
Jerry He
Guangxu Cheng
Josh Elser
Yi Liang
Esteban Gutierrez
Yu Li
Ashu Pachauri
no_apologies
James Taylor
Sean Busbey
Abhishek Singh Chouhan
Alex Leblang
Malcolm Taylor
Phil Yang
Stephen Yuan Jiang
Balazs Meszaros
Yi Liang
rgidwani
anoopsamjohn
Peter Somogyi
Jan Hentschel
Amit Patel
Samir Ahmic
samirMop
张世彬10204932
Ben-Epstein
Kahlil Oppenheimer
David Harju
QilinCao
Vincent
Ashu Pachauri
Gary Helmling
Alex Araujo
huzheng