in-memory-compaction logging
Adds logging of CompactingMemStore configuration on construction.
Add logging of detail about Store on creation including memstore type.
Add chapter to refguide on new in-memory compaction feature.
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
Changes for HBASE-20027 seem to cause UI not showing up on default port in standalone mode. For concurrent
unit test execution, individual tests can set hbase.localcluster.assign.random.ports to true or modify
test/resources/hbase-site.xml.
in-memory compactions)
Log less. Log using same format as used elsewhere in log.
Align logs in HFileArchiver with how we format elsewhere. Removed
redundant 'region' qualifiers, tried to tighten up the emissions so
easier to read the long lines.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ChunkCreator.java
Add a label for each of the chunkcreators we make (I was confused by
two chunk creater stats emissions in log file -- didn't know that one
was for data and the other index).
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplit.java
Formatting. Log less.
M hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreCompactionStrategy.java
Make the emissions in here trace-level. When more than a few regions,
log is filled with this stuff.
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/DoNotRetryRegionException.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/exceptions/MergeRegionException.java
Allow passing cause to Constructor.
M hbase-protocol-shaded/src/main/protobuf/MasterProcedure.proto
Add prepare step to move procedure.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
Add check that regions to merge are actually online to the Constructor
so we can fail fast if they are offline
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
Add prepare step. Check regions and context and skip move if not right.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
Add check parent region is online to constructor.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java
Add generic check region is online utility function for use by subclasses.
M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMove.java
Add test that we fail if we try to move an offlined region.
Add Fail-Fast to Procedures by throwing exception out of Procedure
constructor so if move but table is disabled or if master is going
down, etc., we can give notice before the procedure is scheduled.
Will help guard against scheduling Procedures that will have a hard
time succeeding; e.g. a move when table is offline.
Also fixed bug around table state where we presumed ENABLED though no
entry in hbase:meta (we were using this mechanism for whether a table
existed or not).
M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMove.java
Test stolen from HBASE-20131
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableState.java
Add convenience isEnabled/isDisabled
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Promote assert state to throw exception.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java
Add isClusterUp
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Move constructor now throws exception
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MoveRegionProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RestoreSnapshotProcedure.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java
Do environment check at construction and fail-fast if hostile.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AbstractStateMachineTableProcedure.java
Add preflightCheck utility method.
M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
Removed setting time setting table state; broke when using other than
default environment edge masked by presumption that no state meant
active.
* rely on git plumbing commands when checking if we've built the site for a particular commit already
* switch to forcing '-e' for bash
* add command line switches for: path to hbase, working directory, and publishing
* only export JAVA/MAVEN HOME if they aren't already set.
* add some docs about assumptions
* Update javadoc plugin to consistently be version 3.0.0
* avoid duplicative site invocations on reactor modules
* update use of cp command so it works both on linux and mac
* manually skip enforcer plugin during build
* still doing install of all jars due to MJAVADOC-490, but then skip rebuilding during aggregate reports.
* avoid the pager on git-diff by teeing to a log file, which also helps later reviewing in the case of big changesets.
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Misty Stanley-Jones <misty@apache.org>
Conflicts:
hbase-backup/pom.xml
hbase-spark-it/pom.xml
This reverts commit bc080e7500.
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Segment.java
patch reverted changes that happened in parallel without explanation. see jira.
This patch adds mirroring of table state out to zookeeper. HBase-1.x
clients look for table state in zookeeper, not in hbase:meta where
hbase-2.x maintains table state.
The patch also moves and refactors the 'migration' code that was put in
place by HBASE-13032.
D hbase-client/src/main/java/org/apache/hadoop/hbase/CoordinatedStateException.java
Unused.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Move table state migration code from Master startup out to
TableStateManager where it belongs. Also start
MirroringTableStateManager dependent on config.
A hbase-server/src/main/java/org/apache/hadoop/hbase/master/MirroringTableStateManager.java
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
Move migration from zookeeper of table state in here. Also plumb in
mechanism so subclass can get a chance to look at table state as we do
the startup fixup full-table scan of meta.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Bug-fix. Now we create regions in CLOSED state but we fail to check
table state; were presuming table always enabled. Meant on startup
there'd be an unassigned region that never got assigned.
A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMirroringTableStateManager.java
Test migration and mirroring.
If the initial state is not set to CLOSED while creating merged/ children region/s, merged/
children region/s get stored in meta with state OFFLINE(0). If the master gets restarted after
Merge/ Split procedure stores merged/ children region/s, master startup sequence tries to assign
offline region/s. This is followed by re-assignment of merged/ children region from resumed Merge/
Split procedure.
This bug is related to HBASE-19530.
Signed-off-by: Michael Stack <stack@apache.org>
First, we add test resources to CLASSPATH when tests run. W/o it, there
was no logging of hbase-zookeeper test output (not sure why I have to
add this here and not over in hbase-server; research turns up nothing
so far).
M hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKMainServer.java
Improve fail log message.
M hbase-zookeeper/src/test/java/org/apache/hadoop/hbase/zookeeper/TestReadOnlyZKClient.java
M hbase-zookeeper/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKNodeTracker.java
Wait until ZK is connected before progressing. On my slow zk, it could
be a while post construction before zk connected. Using an unconnected
zk caused test to fail.
M hbase-zookeeper/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKMainServer.java
Change session timeout to default 30s from 1s which was way too short.
M hbase-zookeeper/src/test/resources/log4j.properties
Set zk logs to DEBUG level in this module at least.
Adds a ZooKeeperHelper class that has utility to help interacting w/ ZK.
Adds (client-side) validation to ReplicationPeerConfigBuilder and javadoc
to builder methods in addition to the test fix.
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Changes:
- replaced commons-logging to slf4j everywhere
- log.XXX(Throwable) calls were replaced with log.XXX(t.toString(), t)
- log.XXX(Object) calls were replaced with log.XXX(Objects.toString(obj))
- log.fatal() calls were replaced with log.error(HBaseMarkers.FATAL, ...)
- programmatic log4j configuration was removed from the unit test
This commit does not affect the current logging configurations, because log4j
is still on the classpath. slf4j-log4j12 binds log4j to slf4j.
Signed-off-by: Michael Stack <stack@apache.org>
Michael Stack
Yu Li
Toshihiro Suzuki
Jan Hentschel
Chia-Ping Tsai
Umesh Agashe
Andrew Purtell
maoling
Guanghao Zhang
zhangduo
huzheng
Sean Busbey
Mike Drob
Reid Chan
eshcar
Aman Poonia
Duo Zhang
Sergey Soldatov
Guangxu Cheng
tedyu
Josh Elser
Beluga Behr
Niels Basjes
Vasudevan
Peter Somogyi
Apekshit Sharma
Balazs Meszaros
Yun Chi-Shih