HBASE-18784 if available, query underlying outputstream capabilities where we need hflush/hsync.
* pull things that don't rely on HDFS in hbase-server/FSUtils into hbase-common/CommonFSUtils
* refactor setStoragePolicy so that it can move into hbase-common/CommonFSUtils
* refactor WALProcedureStore so that it handles its own FS interactions
* add a reflection-based lookup of stream capabilities
* call said lookup in places where we make WALs to make sure hflush/hsync is available.
HBASE-19289 Add flag to disable stream capability enforcement
HBASE-19841 Tests against hadoop3 fail with StreamLacksCapabilityException
HBASE-19947 ITU should overwrite HTU local FS
HBASE-20354 better docs for impact of proactively checking hsync support.
* Add to the quickstart guide disabling the hsync check, with a
big warning about how we'll lose data if the check is disabled.
* Add to troubleshooting section so folks searching can get a pointer.
HBASE-6028 Start/Stop compactions at region server level
Add switching on/off of compactions.
Switching off compactions will also interrupt any currently ongoing compactions.
Adds a "compaction_switch" to hbase shell. Switching off compactions will
interrupt any currently ongoing compactions. State set from shell will be
lost on restart. To persist the changes across region servers modify
hbase.regionserver.compaction.enabled in hbase-site.xml and restart.
* makes commands always pass any results back to hirb
* print warning if hirb is given the --return-values flag
* add some docs on how to avoid the console clutter that HBASE-15965 sought to address
* add an upgrade section note about this change.
* cleanup where the get_splits command does its printing so there's a building block that doesn't print
* some rubocop suggested tweaks and opt-out for classlength check on table and shell classes.
Signed-off-by: Mike Drob <mdrob@apache.org>
Conflicts:
src/main/asciidoc/_chapters/upgrading.adoc
* backport leaves off refguide change about upgrading.
Conflicts:
hbase-shell/src/main/ruby/hbase/table.rb
hbase-shell/src/main/ruby/shell/commands/get_splits.rb
Removes src.xml used building src tgz via hbase-assembly.
Use git archive instead going forward. Updates developer release candidate
documentation and the make_rc.sh script.
Slight modifications to developer.adoc for branch-1
In some particular deployments, the Replication code believes it has
reached EOF for a WAL prior to succesfully parsing all bytes known to
exist in a cleanly closed file.
Consistently this failure happens due to an InvalidProtobufException
after some number of seeks during our attempts to tail the in-progress
RegionServer WAL. As a work-around, this patch treats cleanly closed
files differently than other execution paths. If an EOF is detected due
to parsing or other errors while there are still unparsed bytes before
the end-of-file trailer, we now reset the WAL to the very beginning and
attempt a clean read-through.
In current testing, a single such reset is sufficient to work around
observed dataloss. However, the above change will retry a given WAL file
indefinitely. On each such attempt, a log message like the below will
be emitted at the WARN level:
Processing end of WAL file '{}'. At position {}, which is too far away
from reported file length {}. Restarting WAL reading (see HBASE-15983
for details).
Additionally, this patch adds some additional log detail at the TRACE
level about file offsets seen while handling recoverable errors. It also
add metrics that measure the use of this recovery mechanism.
Viraj Jasani
Peter Somogyi
Mate Szalay-Beko
Jan Hentschel
syedmurtazahassan
Sean Busbey
Mingliang Liu
Wellington Chevreuil
Andrew Purtell
stack
Guangxu Cheng
Guanghao Zhang
Pankaj Kumar
Jonathan M Hsieh
tedyu
Nick Dimiduk
Junegunn Choi
Enis Soztutar
Elliott Clark
Apekshit(Appy) Sharma