Commit Graph

80 Commits

Author SHA1 Message Date
zhangduo a84cdbd579 HBASE-20783 Addendum fix broken TestSyncReplicationStandBy 2018-06-28 18:08:43 +08:00
zhangduo f67763ffa0 HBASE-20424 Allow writing WAL to local and remote cluster concurrently 2018-06-28 18:08:43 +08:00
zhangduo 2d203c4479 HBASE-20434 Also remove remote wals when peer is in DA state 2018-06-28 18:08:43 +08:00
zhangduo d91784e666 HBASE-20370 Also remove the wal file in remote cluster when we finish replicating a file 2018-06-28 18:08:43 +08:00
Guanghao Zhang c7d1085fa2 HBASE-19999 Remove the SYNC_REPLICATION_ENABLED flag 2018-06-28 18:07:44 +08:00
zhangduo 0c97cda2a9 HBASE-19990 Create remote wal directory when transitting to state S 2018-06-28 18:07:44 +08:00
zhangduo a41c549ca4 HBASE-19082 Reject read/write from client but accept write from replication in state S 2018-06-28 18:07:44 +08:00
zhangduo 39dd81a7c6 HBASE-19957 General framework to transit sync replication state 2018-06-28 18:07:44 +08:00
Guanghao Zhang 1481bd9481 HBASE-19864 Use protobuf instead of enum.ordinal to store SyncReplicationState
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-06-28 18:07:44 +08:00
zhangduo d8842dc3d4 HBASE-19857 Complete the procedure for adding a sync replication peer 2018-06-28 18:07:44 +08:00
Guanghao Zhang 2acebac00e HBASE-19781 Add a new cluster state flag for synchronous replication 2018-06-28 18:07:44 +08:00
Sean Busbey f1b536bad4 HBASE-20332 shaded mapreduce module shouldn't include hadoop
* modify the jar checking script to take args; make hadoop stuff optional
* separate out checking the artifacts that have hadoop vs those that don't.
* * Unfortunately means we need two modules for checking things
* * put in a safety check that the support script for checking jar contents is maintained in both modules
* * have to carve out an exception for o.a.hadoop.metrics2. :(
* fix duplicated class warning
* clean up dependencies in hbase-server and some modules that depend on it.
* allow Hadoop to have its own htrace where it needs it
* add a precommit check to make sure we're not using old htrace imports
2018-06-18 11:31:04 -07:00
Guanghao Zhang 57c8671728 HBASE-20678 NPE in ReplicationSourceManager#NodeFailoverWorker 2018-06-05 14:37:52 +08:00
maoling 1b98a96caa HBASE-19761:Fix Checkstyle errors in hbase-zookeeper
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2018-06-02 10:08:15 +02:00
huzheng e9a278adc6 HBASE-20475 Fix the flaky TestReplicationDroppedTables unit test - addendum 2018-04-27 21:38:15 +08:00
Sean Busbey 914de11416 HBASE-20442 clean up incorrect use of commons-collections 3
Signed-off-by: Umesh Agashe <uagashe@cloudera.com>
Signed-off-by: Yu Li <liyu@apache.org>
2018-04-20 07:30:34 -05:00
BELUGA BEHR d866e7c658 HBASE-19488 Move to using Apache commons CollectionUtils
Signed-off-by: Apekshit Sharma <appy@apache.org>
2018-04-04 14:12:19 -07:00
zhangduo 1e56938757 HBASE-20296 Remove last pushed sequence ids when removing tables from a peer 2018-04-01 21:14:31 +08:00
zhangduo 056c3395d9 HBASE-20285 Delete all last pushed sequence ids when removing a peer or removing the serial flag for a peer 2018-03-27 12:20:51 +08:00
huzheng 7a1d00c7a0 HBASE-20138 Find a way to deal with the conflicts when updating replication position 2018-03-26 21:07:22 +08:00
tianjingyun e468b4022f HBASE-20227 Add UT for ReplicationUtils.contains method
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-24 22:12:52 +08:00
zhangduo 64061f896f HBASE-20147 Serial replication will be stuck if we create a table with serial replication but add it to a peer after there are region moves 2018-03-23 14:31:20 +08:00
zhangduo f346a144b9 HBASE-20116 addendum fix javadoc and also a simple optimization 2018-03-22 21:01:45 +08:00
Chia-Ping Tsai a6eeb26cc0 HBASE-20212 Make all Public classes have InterfaceAudience category
Signed-off-by: tedyu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-22 18:10:23 +08:00
huzheng 8ab7b20f48 HBASE-20116 Optimize the region last pushed sequence id layout on zk 2018-03-21 14:24:55 +08:00
zhangduo 67f013430c HBASE-20206 WALEntryStream should not switch WAL file silently 2018-03-19 11:40:11 +08:00
zhangduo 00095a2ef9 Revert "HBASE-19665 Add table based replication peers/queues storage back"
This reverts commit 31978c31bb.

 Conflicts:
	hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/TableReplicationStorageBase.java
2018-03-17 20:25:27 +08:00
Michael Stack bedf849d83 HBASE-20213 [LOGGING] Aligning formatting and logging less (compactions,
in-memory compactions)

Log less. Log using same format as used elsewhere in log.

Align logs in HFileArchiver with how we format elsewhere. Removed
redundant 'region' qualifiers, tried to tighten up the emissions so
easier to read the long lines.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java
 Add a label for each of the chunkcreators we make (I was confused by
two chunk creater stats emissions in log file -- didn't know that one
was for data and the other index).

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java
 Formatting. Log less.

M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactionStrategy.java
 Make the emissions in here trace-level. When more than a few regions,
log is filled with this stuff.
2018-03-16 13:16:49 -07:00
Chia-Ping Tsai 4f2133ee32 HBASE-20119 Introduce a pojo class to carry coprocessor information in order to make TableDescriptorBuilder accept multiple cp at once
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
Signed-off-by: Michael Stack <stack@apache.org>
2018-03-16 01:21:38 +08:00
huzheng 31978c31bb HBASE-19665 Add table based replication peers/queues storage back 2018-03-14 15:42:16 +08:00
zhangduo b7308ee01c HBASE-20117 Cleanup the unused replication barriers in meta table 2018-03-14 12:08:15 +08:00
zhangduo dd6f4525e7 HBASE-20148 Make serial replication as a option for a peer instead of a table 2018-03-10 09:04:44 +08:00
Sean Busbey 2a65066b35 HBASE-20070 refactor website generation
* rely on git plumbing commands when checking if we've built the site for a particular commit already
* switch to forcing '-e' for bash
* add command line switches for: path to hbase, working directory, and publishing
* only export JAVA/MAVEN HOME if they aren't already set.
* add some docs about assumptions
* Update javadoc plugin to consistently be version 3.0.0
* avoid duplicative site invocations on reactor modules
* update use of cp command so it works both on linux and mac
* manually skip enforcer plugin during build
* still doing install of all jars due to MJAVADOC-490, but then skip rebuilding during aggregate reports.
* avoid the pager on git-diff by teeing to a log file, which also helps later reviewing in the case of big changesets.

Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
2018-03-02 09:25:10 -06:00
huzheng 99d3edfc82 HBASE-20050 Reimplement updateReplicationPositions logic in serial replication based on the newly introduced replication storage layer
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-03-02 15:53:14 +08:00
Guanghao Zhang 2b63af376e HBASE-19923 Reset peer state and config when refresh replication source failed 2018-02-09 18:54:03 +08:00
zhangduo fc6d140adf HBASE-19904 Break dependency of WAL constructor on Replication 2018-02-02 14:10:29 +08:00
zhangduo 7c318cead9 HBASE-19887 Do not overwrite the surefire junit listener property in the pom of sub modules 2018-01-31 20:49:18 +08:00
zhangduo 918599ef12 HBASE-19873 Add a CategoryBasedTimeout ClassRule for all UTs 2018-01-29 08:43:56 +08:00
Apekshit Sharma 0b6422658c HBASE-19780 Remove execution phase and goal of checkstyle plugin.
We don't do checkstyle as part of default "mvn install" since it takes significant
amount of time (~90sec) and slows builds and development. We only need it for pre-commits.
2018-01-23 15:08:04 -08:00
zhangduo eb88b69845 HBASE-19719 Fix checkstyle issues 2018-01-09 13:11:01 +08:00
Guanghao Zhang 01f1fc8284 HBASE-19636 All rs should already start work with the new peer change when replication peer procedure is finished
Signed-off-by: zhangduo <zhangduo@apache.org>
2018-01-09 13:11:01 +08:00
huzheng a5cfc51cbe HBASE-19661 Replace ReplicationStateZKBase with ZKReplicationStorageBase 2018-01-09 13:11:01 +08:00
zhangduo 368db315a6 HBASE-19687 Move the logic in ReplicationZKNodeCleaner to ReplicationChecker and remove ReplicationZKNodeCleanerChore 2018-01-09 13:11:01 +08:00
zhangduo a85a215f36 HBASE-19623 Create replication endpoint asynchronously when adding a replication source 2018-01-09 13:11:01 +08:00
zhangduo 19707a85dd HBASE-19633 Clean up the replication queues in the postPeerModification stage when removing a peer 2018-01-09 13:11:01 +08:00
huzheng f89920a60f HBASE-19622 Reimplement ReplicationPeers with the new replication storage interface 2018-01-09 13:11:01 +08:00
zhangduo f4703c6ed3 HBASE-19617 Remove ReplicationQueues, use ReplicationQueueStorage directly 2018-01-09 13:11:01 +08:00
Guanghao Zhang 65eb91f84e HBASE-19573 Rewrite ReplicationPeer with the new replication storage interface 2018-01-09 13:11:01 +08:00
zhangduo c4fa568b47 HBASE-19599 Remove ReplicationQueuesClient, use ReplicationQueueStorage directly 2018-01-09 13:11:01 +08:00
zhangduo 5e6c303528 HBASE-19543 Abstract a replication storage interface to extract the zk specific code 2018-01-09 13:11:01 +08:00