This PR is a follow-up of HBASE-25181 (#2539), where several issues were
discussed on the PR:
1. Currently we use PBKDF2WithHmacSHA1 key generation algorithm to generate a
secret key for HFile / WalFile encryption, when the user is defining a string
encryption key in the hbase shell. This algorithm is not secure enough and
not allowed in certain environments (e.g. on FIPS compliant clusters). We are
changing it to PBKDF2WithHmacSHA384. It will not break backward-compatibility,
as even the tables created by the shell using the new algorithm will be able
to load (e.g. during bulkload / replication) the HFiles serialized with the
key generated by an old algorithm, as the HFiles themselves already contain
the key necessary for their decryption.
Smaller issues fixed by this commit:
2. Improve the documentation e.g. with the changes introduced by HBASE-25181
and also by some points discussed on the Jira ticket of HBASE-25263.
3. In EncryptionUtil.createEncryptionContext the various encryption config
checks should throw IllegalStateExceptions instead of RuntimeExceptions.
4. Test cases in TestEncryptionTest.java should be broken down into smaller
tests.
5. TestEncryptionDisabled.java should use ExpectedException JUnit rule to
validate exceptions.
closes#2676
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Expose an argument on the `count` command which is passed to the
`setCacheBlocks` method on the Scan which the count command uses.
This is a quick and dirty approach to read all of the blocks for a table
into the block cache.
* Raise an error when the value isn't a boolean or the expected string
Closes#2650
Signed-off-by: Zach York <zyork@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
- upgrade our default jruby to 9.2.13.0
- this major JRuby version update changes the Ruby compatibility from Ruby 2.3 to Ruby 2.5
- use a custom IRB prompt to convey similar information to before
- update the joni and jcoding dependencies to match this version of jruby-complete
closes#2308
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
* removes some cruft from the hbase-shell pom that appears to be from coping the hbase-server pom long ago
* puts the ruby scripts into the hbase-shell jar following the guide from jruby for packaging
* removes hard coding the location of the implementation scripts from our runtime
* removes hard coding the load path for the implementation from the test code (leaves hard coding the test code location)
* provides a work around for a name conflict between our shell and the ruby stdlib shell.
closes#2515
Signed-off-by: Michael Stack <stack@apache.org>
* Admin API getLogEntries() for ring buffer use-cases: so far, provides balancerDecision and slowLogResponse
* Refactor RPC call for similar use-cases
* Single RPC API getLogEntries() for both Master.proto and Admin.proto
Closes#2261
Signed-off-by: Andrew Purtell <apurtell@apache.org>
* HBASE-24806 Small Updates to Functionality of Shell IRB Workspace
- Move exception handler from Shell::Shell#eval_io to new method,
Shell::Shell#exception_handler
- Add unit tests for Shell::Shell#exception_handler
- Change Shell::Shell#eval_io to no longer raise SystemExit when any error is
seen and update unit test
- Update ruby test runner to catch SystemExit and fail to avoid tests that
cause the test runner to incorrectly exit successfully
- Add Hbase::Loader module to find ruby scripts in the $LOAD_PATH and classpath
using JRuby's loader.
- In hbase-shell, install IRB commands before exporting HBase commands. The
HBase commands will override the IRB commands, and no warning will be
printed.
* Remove unused variables from shell_test
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
* HBASE-24874 Fix hbase-shell access to ModifiableTableDescriptor methods
- Fix hbase-shell access in JDK 11 for calls to
TableDescriptorBuilder.toCoprocessorDescriptor and
ModifiableTableDescriptor.toStringTableAttributes.
- Allow coprocessors to be specified using a Ruby hash in the hbase-shell alter
command and replace usage in the help text. The previous String overload of
the alter command will continue to work and is still covered by a unit test,
but will no longer be suggested in the alter command help.
* Update patch
- Add warning over toCoprocessorDescriptor noting the usage by hbase-shell
- Add constants to hbase_constants for coprocessor specification
- Document usage of ModifiableTableDescriptor.toStringTableAttributes
* Convert comment over toCoprocessorDescriptor into docstring
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
Introduce an additional method to our Admin interface that allow an
operator to selectivly run the normalizer. The IPC protocol supports
general table name select via compound filter.
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
- Update all hbase-shell commands to return help text implicitly
- Add hbase-shell test to ensure all help methods return unindented text
- Set the outermost indentation to zero for the help text of all hbase-shell
commands by using the squiggly heredoc added in Ruby 2.3.0.
- Document return value of hbck_chore_run and add usage examples to truncate commands
- Update docs to reflect current return value of list command
Signed-off-by: stack <stack@apache.org>
- In HBase::Table, the instance variable @converters is used to map column
names to converters. This patch fixes how HBase::Table#_get_internal and
HBase::Table#_scan_internal generate the column name key used to access
@converters.
- Refactor parsing of family:qualifier:converter specifications so that the
code is more readable and reusable. As part of this change, I added two
private methods and marked HBase::Table#set_converter as deprecated for
removal in HBase 4.0.0.
- Add unit testing for the fixed bug
Signed-off-by: stack <stack@apache.org>
BukrosSzabolcs
Duo Zhang
Tak Lon (Stephen) Wu
Istvan Toth
mkkumar
xicm
Bryan Beaudreault
Rahul Kumar
Baiqiang Zhao
GeorryHuang
Andrew Purtell
Wellington Ramos Chevreuil
申胜利
Laxman Goswami
xijiawen
Gary Wang
Mate Szalay-Beko
Josh Elser
XinSun
Norbert Kalmar
WenFeiYi
Sean Busbey
Viraj Jasani
bsglz
Bo Cui
Guanghao Zhang
Elliot
Nick Dimiduk
Pankaj