Make it so WALPlayer can replay recovered.edits files.
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALInputFormat.java
Allow for WAL files that do NOT have a startime in their name.
Use the 'generic' WAL-filename parser instead of the one that
used be local here. Implement support for 'startTime' filter.
Previous was just not implemented.
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
Checkstyle.
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java
Use the new general WAL name timestamp parser.
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/WAL.java
Utility for parsing timestamp from WAL filename.
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRecoveredEdits.java
Export attributes about the local recovered.edits test file
so other tests can play with it.
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Add MR counters so operator can see if WALPlayer run actually did
anything. Fix bugs in usage (it enforced two args though usage
describes allowing one arg only). Clean up usage output. In
particular add note on wal file separator as hbase by default uses
the ',' in its WAL file names which could befuddle operator
trying to do simple import.
Signed-off-by: Huaxiang Sun <huaxiangsun@apache.com>
* Admin API getLogEntries() for ring buffer use-cases: so far, provides balancerDecision and slowLogResponse
* Refactor RPC call for similar use-cases
* Single RPC API getLogEntries() for both Master.proto and Admin.proto
Closes#2261
Signed-off-by: Andrew Purtell <apurtell@apache.org>
* remove RM for 1.3
* remove EOM releases from hadoop matrix
* add 1.6 column to hadoop matrix that matches previous 1.5 column
* remove Hadoop rows from hadoop matrix that were only present for now EOM hbase release lines
* remove Hadoop notes that were about Hadoop versions that have been removed.
* update Java version table to only refer to maintained HBase release lines
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
- Update all hbase-shell commands to return help text implicitly
- Add hbase-shell test to ensure all help methods return unindented text
- Set the outermost indentation to zero for the help text of all hbase-shell
commands by using the squiggly heredoc added in Ruby 2.3.0.
- Document return value of hbck_chore_run and add usage examples to truncate commands
- Update docs to reflect current return value of list command
Signed-off-by: stack <stack@apache.org>
2.1 is EOM so the 2.1 Documentation link can be removed.
The content is still available under https://hbase.apache.org/2.1
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Add a line for hadoop-3.2.x. Values are based on the if-statement in
our personality file,
```
if [[ "${PATCH_BRANCH}" = branch-1* ]]; then
yetus_info "Setting Hadoop 3 versions to test based on branch-1.x rules."
hbase_hadoop3_versions=""
elif [[ "${PATCH_BRANCH}" = branch-2.0 ]] || [[ "${PATCH_BRANCH}" = branch-2.1 ]]; then
yetus_info "Setting Hadoop 3 versions to test based on branch-2.0/branch-2.1 rules"
if [[ "${QUICK_HADOOPCHECK}" == "true" ]]; then
hbase_hadoop3_versions="3.0.3 3.1.2"
else
hbase_hadoop3_versions="3.0.3 3.1.1 3.1.2"
fi
else
yetus_info "Setting Hadoop 3 versions to test based on branch-2.2+/master/feature branch rules"
if [[ "${QUICK_HADOOPCHECK}" == "true" ]]; then
hbase_hadoop3_versions="3.1.2 3.2.1"
else
hbase_hadoop3_versions="3.1.1 3.1.2 3.2.0 3.2.1"
fi
fi
```
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
* Add a section calling out our categorical lack of support for
non-LTS Java versions.
* Provide clarification as to the meaning of the 'red x' in our
support matrix.
* Transpose the support matrix to have HBase releases along the x-axis
and Java releases along the y-axis, because there are more Java
releases than HBase release categories.
* Update the JDK11 status in the matrix.
* Add a note qualifying our JDK11 support.
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Trying to make doc changes is pretty awful, with lines running on for
hundreds of columns. At least for this time, clean up with line breaks
where asciidoc will let us.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Simplify our Normalizer story to have just a single, configurable
implementation.
* fold the features of `MergeNormalizer` into
`SimpleRegionNormalizer`, removing the intermediate abstract class.
* configuration keys for merge-only features now share a common
structure.
* add configuration to selectively disable normalizer split/merge
operations.
* `RegionNormalizer` now extends `Configurable` instead of creating a
new instance of `HBaseConfiguration` or snooping one off of other
fields.
* avoid the extra RPCs by using `MasterServices` instead of
`MasterRpcServices`.
* boost test coverage of all the various flags and feature
combinations.
Signed-off-by: Michael Stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: huaxiangsun <huaxiangsun@apache.org>
Add a note to the 2.3 upgrade section regarding the ZooKeeper version
bump and include a link off to ZooKeeper's FAQ.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
The current wording gives an impression the session timeout range
on the ZK side is forced to be in the range [2 * tickTime, 20 * tickTime].
This range is configurable on the ZK service side using minSessionTimeout
and maxSessionTimeout. Clarified that.
Signed-off-by: Jan Hentschel <janh@apache.org
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Simplify the new user experience shipping a configuration that enables
a fresh checkout or tarball distribution to run in standalone mode
without direct user configuration. This change restores the behavior
we had when running on Hadoop 2.8 and earlier.
Patch for master includes an update to the book. This change will be
omitted when backporting to earlier branches.
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
* add design doc for original MOB changes as they were when HBase 2.0 came out
* add design doc for distributed MOB compaction
* remove configuration and commands no longer relevant after distributed MOB compaction
* add in discussion of configuration options
* allow asciimath formulas since we use them in the discussion
closes#1232
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
Set the fork count for first and second parts to be 0.5C. Add a bit of
doc too on this as well as some qualification on our test categories.
Also adds -T0.5C to MAVEN_ARGS in the hbase personality.
The Hadoop AccessControlList allows us to specify admins of the webUI
via a list of users and/or groups. Admins of the WebUI can mutate the
system, potentially seeing sensitive data or modifying the system.
hbase.security.authentication.spnego.admin.users is a comma-separated
list of users who are admins.
hbase.security.authentication.spnego.admin.groups is a comma-separated
list of groups whose membership are admins. Either of these
configuration properties may also contain an asterisk (*) which denotes
"any entity" (e.g user, group).
Previously, when a user was denied from some endpoint that was
designated for admins, they received an HTTP/401. In this case, it is
more correct to return HTTP/403 as they were correctly authenticated,
but they were disallowed from fetching the given resource. This commit
incorporates this change.
hbase.security.authentication.ui.config.protected also exists for users
who have sensitive information stored in the Hadoop service
configuration and want to limit access to this endpoint. By default,
the Hadoop configuration endpoint is not protected and any
authenticated user can access it.
The test is based off of work by Nihal Jain in HBASE-20472.
Co-authored-by: Nihal Jain <nihaljain.cs@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
* Adds a new MapReduce job that builds a report of health of mob files
* Also builds background information on mob system use
* add a basic mob architecture in the ref guide to explain how mob takes the reference hfile value and finds the actual cell contents
* add a troubleshooting mob section to the ref guide to explain how to do a mob reference scan.
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Duo Zhang
Michael Stack
Pankaj
Nick Dimiduk
bsglz
Viraj Jasani
Guanghao Zhang
Sean Busbey
Toshihiro Suzuki
Javier Akira Luca de Tena
neko
Abhey
ethanhur
Elliot
Peter Somogyi
Baiqiang Zhao
niuyulin
Wellington Ramos Chevreuil
Bharath Vissapragada
Yu Wang
huangtianhua
Jan Hentschel
stack
Wei-Chiu Chuang
Andrew Purtell
Sakthi
Michael
maoling
Josh Elser
Geoffrey Jacoby
Beata Sudi
Manu Manjunath
Mingliang Liu
Udai Bhan Kashyap