Commit Graph

113 Commits

Author SHA1 Message Date
zhangduo 9369cf26eb HBASE-20147 Serial replication will be stuck if we create a table with serial replication but add it to a peer after there are region moves 2018-04-09 15:18:44 +08:00
zhangduo 40ad5aeafe HBASE-20116 addendum fix javadoc and also a simple optimization 2018-04-09 15:18:44 +08:00
huzheng 2b9fed863f HBASE-20116 Optimize the region last pushed sequence id layout on zk 2018-04-09 15:18:44 +08:00
zhangduo 16a4dd6b8f HBASE-20206 WALEntryStream should not switch WAL file silently 2018-04-09 15:18:44 +08:00
zhangduo 644bfe36b2 HBASE-20117 Cleanup the unused replication barriers in meta table 2018-04-09 15:18:44 +08:00
zhangduo cea5199ea1 HBASE-20148 Make serial replication as a option for a peer instead of a table 2018-04-09 15:18:44 +08:00
huzheng 1d11cdb26c HBASE-20050 Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-04-09 15:18:44 +08:00
BELUGA BEHR 039bc73571 HBASE-19488 Move to using Apache commons CollectionUtils
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-04-04 14:16:33 -07:00
Chia-Ping Tsai dd9e46bbf5 HBASE-20212 Make all Public classes have InterfaceAudience category
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-22 18:09:54 +08:00
Michael Stack 3f1c86786c HBASE-20213 [LOGGING] Aligning formatting and logging less (compactions,
in-memory compactions)

Log less. Log using same format as used elsewhere in log.

Align logs in HFileArchiver with how we format elsewhere. Removed
redundant 'region' qualifiers, tried to tighten up the emissions so
easier to read the long lines.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java
 Add a label for each of the chunkcreators we make (I was confused by
two chunk creater stats emissions in log file -- didn't know that one
was for data and the other index).

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java
 Formatting. Log less.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactionStrategy.java
 Make the emissions in here trace-level. When more than a few regions,
log is filled with this stuff.
2018-03-16 13:07:34 -07:00
Guanghao Zhang 55d0d3026a HBASE-19923 Reset peer state and config when refresh replication source failed 2018-03-09 20:55:48 +08:00
zhangduo abaa80f26e HBASE-19873 addendum add missing rule for new tests 2018-03-09 20:55:48 +08:00
zhangduo 1c32241a00 HBASE-19719 Fix checkstyle issues 2018-03-09 20:55:48 +08:00
Guanghao Zhang d36aacdf9e HBASE-19636 All rs should already start work with the new peer change when replication peer procedure is finished
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-09 20:55:48 +08:00
huzheng 5655b3c0ca HBASE-19661 Replace ReplicationStateZKBase with ZKReplicationStorageBase 2018-03-09 20:55:48 +08:00
zhangduo b84fbde175 HBASE-19687 Move the logic in ReplicationZKNodeCleaner to ReplicationChecker and remove ReplicationZKNodeCleanerChore 2018-03-09 20:55:48 +08:00
zhangduo 5d922a5e0c HBASE-19623 Create replication endpoint asynchronously when adding a replication source 2018-03-09 20:55:48 +08:00
zhangduo 525fef572e HBASE-19633 Clean up the replication queues in the postPeerModification stage when removing a peer 2018-03-09 20:55:48 +08:00
huzheng 1e36a84afc HBASE-19622 Reimplement ReplicationPeers with the new replication storage interface 2018-03-09 20:55:48 +08:00
zhangduo d9b5eb3abb HBASE-19617 Remove ReplicationQueues, use ReplicationQueueStorage directly 2018-03-09 20:55:48 +08:00
Guanghao Zhang eae251d203 HBASE-19573 Rewrite ReplicationPeer with the new replication storage interface 2018-03-09 20:55:48 +08:00
zhangduo 15055a421b HBASE-19599 Remove ReplicationQueuesClient, use ReplicationQueueStorage directly 2018-03-09 20:55:48 +08:00
zhangduo 1de08ded58 HBASE-19543 Abstract a replication storage interface to extract the zk specific code 2018-03-09 20:55:48 +08:00
huzheng 3fd417600e HBASE-19525 RS side changes for moving peer modification from zk watcher to procedure 2018-03-09 20:55:48 +08:00
zhangduo 5e410d8140 HBASE-19524 Master side changes for moving peer modification from zk watcher to procedure 2018-03-09 20:55:48 +08:00
Sean Busbey 71cc7869db HBASE-20155 update branch-2 version to 2.1.0-SNAPSHOT
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2018-03-08 08:44:30 -08:00
Sean Busbey 9927c2e14a HBASE-20070 refactor website generation
* rely on git plumbing commands when checking if we've built the site for a particular commit already
* switch to forcing '-e' for bash
* add command line switches for: path to hbase, working directory, and publishing
* only export JAVA/MAVEN HOME if they aren't already set.
* add some docs about assumptions
* Update javadoc plugin to consistently be version 3.0.0
* avoid duplicative site invocations on reactor modules
* update use of cp command so it works both on linux and mac
* manually skip enforcer plugin during build
* still doing install of all jars due to MJAVADOC-490, but then skip rebuilding during aggregate reports.
* avoid the pager on git-diff by teeing to a log file, which also helps later reviewing in the case of big changesets.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>

 Conflicts:
	hbase-backup/pom.xml
	hbase-spark-it/pom.xml
2018-03-02 09:51:43 -06:00
Michael Stack a2de29560f HBASE-20113 Move branch-2 version from 2.0.0-beta-2-SNAPSHOT to 2.0.0-beta-2 2018-03-01 15:46:38 -08:00
Ben Lau 0743bda059 HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated in case of a KeeperException
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2018-02-14 17:23:09 -08:00
zhangduo 66a11f2ac0 HBASE-19904 Break dependency of WAL constructor on Replication 2018-02-03 17:11:30 +08:00
Michael Stack bac4687345 HBASE-19919 Tidying up logging 2018-02-02 22:42:30 -08:00
zhangduo 28d13c54a9 HBASE-19887 Do not overwrite the surefire junit listener property in the pom of sub modules 2018-01-31 20:49:25 +08:00
Michael Stack 90a75fb052 HBASE-19888 Move branch-2 version from 2.0.0-beta-1 to 2.0.0-beta-2-SNAPSHOT 2018-01-29 14:17:54 -08:00
Guanghao Zhang 72702eeb70 HBASE-19783 Change replication peer cluster key/endpoint from a not-null value to null is not allowed
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-12 21:41:57 +08:00
Mike Drob 64cb777a8a HBASE-19552 find-and-replace thirdparty offset 2017-12-28 12:01:25 -06:00
Michael Stack d6d8369655
HBASE-19648 Move branch-2 version from 2.0.0-beta-1-SNAPSHOT to 2.0.0-beta-1 2017-12-27 14:41:19 -08:00
Guanghao Zhang 920005a2dc HBASE-19621 (addendum) Revisit the methods in ReplicationPeerConfigBuilder 2017-12-26 21:56:40 +08:00
Guanghao Zhang 2abf7b508c HBASE-19621 Revisit the methods in ReplicationPeerConfigBuilder 2017-12-26 14:44:07 +08:00
Guanghao Zhang 65159dc256 HBASE-19618 Remove replicationQueuesClient.class/replicationQueues.class config and remove table based ReplicationQueuesClient/ReplicationQueues implementation 2017-12-26 14:44:02 +08:00
Guanghao Zhang b552eb2ff9 HBASE-19576 Introduce builder for ReplicationPeerConfig and make it immutable 2017-12-23 19:04:58 +08:00
Balazs Meszaros 992b5d8630 HBASE-10092 Move up on to log4j2
Changes:
- replaced commons-logging to slf4j everywhere
- log.XXX(Throwable) calls were replaced with log.XXX(t.toString(), t)
- log.XXX(Object) calls were replaced with log.XXX(Objects.toString(obj))
- log.fatal() calls were replaced with log.error(HBaseMarkers.FATAL, ...)
- programmatic log4j configuration was removed from the unit test

This commit does not affect the current logging configurations, because log4j
is still on the classpath. slf4j-log4j12 binds log4j to slf4j.

Signed-off-by: Michael Stack <stack@apache.org>
2017-12-20 22:58:12 -08:00
Guanghao Zhang 60cd494d1c HBASE-19492 Add EXCLUDE_NAMESPACE and EXCLUDE_TABLECFS support to replication peer config 2017-12-19 16:57:38 +08:00
Peter Somogyi 2043c337d8
HBASE-19498 Fix findbugs and error-prone warnings in hbase-client (branch-2)
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-12-15 15:57:01 -08:00
Mike Drob 23a9059cb2 HBASE-18838 Fix hadoop3 check-shaded-invariants 2017-12-15 13:20:54 -06:00
Peter Somogyi 941acc5c05 HBASE-19213 Align check and mutate operations in Table and AsyncTable
- Deprecates old checkAnd*() operations in Table
- Adds Table#CheckAndMutateBuilder and implements it in HTable

Commiter note: When committing the patch, noticed redundant {@inheritDoc} being added in HTable.
Removed new and olds ones.

Signed-off-by: Apekshit Sharma <appy@apache.org>
2017-12-12 20:46:39 -08:00
Josh Elser 4fef4cfc30 HBASE-19267 Remove compiler-plugin mapping executions as it breaks Java8 detection
It seems like the original reason this execution filter was added is no
longer an issue for 2.0. Actually, these entries actually preclude
Eclipse from correctly using the Java8 source/target version that we
have specified (which creates numerous compilation errors in Eclipse)

Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-11-28 14:54:32 -05:00
Guanghao Zhang ca6e7e68f4 HBASE-16868 Add a replicate_all flag to avoid misuse the namespaces and table-cfs config of replication peer
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2017-11-23 15:08:03 +08:00
Peter Somogyi bcd367e293
HBASE-19315 Incorrect snapshot version is used for 2.0.0-beta-1
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-21 10:41:50 -08:00
Guanghao Zhang c978f8ab23 HBASE-19293 Support add a disabled state replication peer directly 2017-11-21 15:37:33 +08:00
Apekshit Sharma e0c4f374b5 HBASE-19114 Split out o.a.h.h.zookeeper from hbase-server and hbase-client
- Moved DrainingServerTracker and RegionServerTracker to hbase-server:o.a.h.h.master.
- Moved SplitOrMergeTracker to oahh.master (because it depends on a PB)
- Moving hbase-client:oahh.zookeeper.*  to hbase-zookeeper module.  After HBASE-19200, hbase-client doesn't need them anymore (except 3 classes).
- Renamed some classes to use a consistent naming for classes - ZK instead of mix of ZK, Zk , ZooKeeper. Couldn't rename following public classes: MiniZooKeeperCluster, ZooKeeperConnectionException. Left RecoverableZooKeeper for lack of better name. (suggestions?)
- Sadly, can't move tests out because they depend on HBaseTestingUtility (which defeats part of the purpose - trimming down hbase-server tests. We need to promote more use of mocks in our tests)
2017-11-17 13:23:28 -08:00
Guanghao Zhang d885e2232d HBASE-19009 implement modifyTable and enable/disableTableReplication for AsyncAdmin 2017-11-16 07:19:34 +08:00
Tamas Penzes 7a69ebc73e HBASE-18601: Update Htrace to 4.2
Updated HTrace version to 4.2
Created TraceUtil class to wrap htrace methods. Uses try with resources.

Signed-off-by: Balazs Meszaros <balazs.meszaros@cloudera.com>
Signed-off-by: Michael Stack <stack@apache.org>
2017-11-13 10:38:36 -08:00
zhangduo 30f55f2316 HBASE-19200 Make hbase-client only depend on ZKAsyncRegistry and ZNodePaths
- Removes zookeeper connection from ClusterConnection
- Deletes class ZooKeeperKeepAliveConnection
- Removes Registry, ZooKeeperRegistry, and RegistryFactory
2017-11-10 10:09:04 -08:00
Michael Stack f13cf56f1c
HBASE-19197 Move version on branch-2 from 2.0.0-alpha4 to 2.0.0-beta-1.SNAPSHOT 2017-11-06 20:46:38 -08:00
Guanghao Zhang 7f1cd12e8c HBASE-14247 Separate the old WALs into different regionserver directories 2017-10-17 19:39:21 +08:00
Sean Busbey 35094bf4d5 HBASE-18933 set version number to 2.0.0-alpha4-SNAPSHOT following release of alpha3 2017-10-04 07:57:49 -05:00
Michael Stack 7660f9e86a HBASE-18819 Set version number to 2.0.0-alpha3 from 2.0.0-alpha3-SNAPSHOT 2017-09-14 12:38:46 -07:00
Sean Busbey d576e5a32d HBASE-17823 Migrate to Apache Yetus Audience Annotations
Includes partial backport of hbase-build-configuration module

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
2017-09-12 23:15:50 -05:00
Michael Stack 7f70b6c8e3 HBASE-18769 Make CompareFilter use generic CompareOperator instead of
internal enum
2017-09-08 10:05:51 -07:00
Apekshit Sharma 56cba5e450 HBASE-14997 Move compareOp and Comparators out of filter to client package
Add a new generic CompareOperator enum and use it instead of the filter
CompareOp enum everywhere BUT inside CompareFilter.
2017-09-07 11:03:22 -07:00
Umesh Agashe bd219c0fb8 HBASE-18674 upgrading to commons-lang3
Signed-off-by: Michael Stack <stack@apache.org>
2017-09-05 09:46:49 -07:00
Michael Stack 43c4bc5761 HBASE-18723 [pom cleanup] Do a pass with dependency:analyze; remove unused and explicity list the dependencies we exploit
Do a pass with dependency:analyze; remove unused and
explicity list the dependencies we exploit.
Remove the parent dependencies set which had junit, mockito,
log4j, and findbugs annotations (had to put junit back
temporarily in subsequent version of this patch TODO). Listing in
parent set meant these libs were dependencies for all modules
which in practice was not the case. Edited all modules so
those that need any from this parent set now do explicit listing.

Ran the dependency:analyze over the project. Acted on most
suggested removals and requests for explicit listing. Some
grey areas remain around transitives that come in with
hadoop -needs better excludes, another project- and that
the dependency:analyze tool is not always accurate in its
reporting.
2017-09-01 08:03:35 -07:00
Apekshit Sharma e2ce252b59 HBASE-17442 Move most of the replication related classes from hbase-client to new hbase-replication package. (Guanghao Zhang).
Change-Id: Ie0e24cc617ab4bf56de8b1747062d1b78a5d4669
2017-08-23 22:18:26 -07:00