We observed this delete call to be a bottleneck for table with lots of
regions. Patch attempts to parallelize them.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Also fix three bugs:
* We were trying to delete non-empty directory; weren't doing
accounting for meta WALs where meta had moved off the server
(successfully)
* We were deleting split WALs rather than archiving them.
* We were not handling corrupt files.
Deprecations and removal of tests of old system.
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
- In HBase::Table, the instance variable @converters is used to map column
names to converters. This patch fixes how HBase::Table#_get_internal and
HBase::Table#_scan_internal generate the column name key used to access
@converters.
- Refactor parsing of family:qualifier:converter specifications so that the
code is more readable and reusable. As part of this change, I added two
private methods and marked HBase::Table#set_converter as deprecated for
removal in HBase 4.0.0.
- Add unit testing for the fixed bug
Signed-off-by: stack <stack@apache.org>
* HBASE-11686 Shell code should create a binding / irb workspace instead of polluting the root namespace
- Refactor Shell.export_commands to define commands using ruby lambdas.
Additionally, this change stores a reference to shell_inst in scope so that
we no longer need to assume the existance of the variable @shell.
- Add logic to Shell class for constructing an IRB workspace with its own
binding and non-global receiver. This workspace is loaded with all HBase and
IRB commands.
- Create new method on Shell for evaluating input from an arbitrary IO instance
within the created IRB workspace. This is based on work by Hsieh that was
previously in bin/hirb.rb. This method is more generic and more testable.
This single pattern can be used for both executing files and reading from
stdin, therefore reducing complexity.
- Move special 'help' and 'tools' command definitions to shell.rb. These
commands are tightly linked with an instance of the shell, so it is easiest
to have them defined together.
- Remove all global includes of HBaseConstants from ruby test files. Before
this change, tests were loading these constants into the top level, which
could cause tests to pass that should really fail.
- Try to reduce the number of places that constants are included. I think it's
best to reference each ruby constant's full name, but where that would cause
a big diff I instead moved the include to the innermost Module or Class.
- Update docs and comments
- Remove unneccessary includes
- Add shell --top-level-cmds compatibility flag. Since this PR removes all the
HBase symbols from the top-level receiver (ie. main Object), it is possible
(albeit unlikely) that this will break operator scripts. This flag will
export all the commands at the top-level like the shell previously did.
* HBASE-11686 Light refactoring with added unit tests
- Fixes some constants references by admin test 2
- Install HBase commands as singleton methods on recevier instances so that
multiple receivers may exist.
- Rename new flag from --top-level-cmds to --top-level-defs to be more
semantically accurate.
- Create new helper method Shell::Shell#export_all to install @hbase, @shell,
constants, and all hbase commands to a target receiver. As a result, the
HBaseReceiver became very simple and could be moved to shell.rb.
- Add unit tests for Shell::Shell#eval_io and Shell::Shell#export_all
- Add @hbase and @shell to hbase-shell IRB workspace
- Fix robocop issues within patch
* Typo s/is/if/
2.1 is EOM so the 2.1 Documentation link can be removed.
The content is still available under https://hbase.apache.org/2.1
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Sometimes running chaos monkey, I've found that we lose accounting of
region servers. I've taken to a manual process of checking the
reported list against a known reference. It occurs to me that
ChaosMonkey has a known reference, and it can do this accounting for
me.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Running `ServerKillingChaosMonkey` via `RESTApiClusterManager` for any
duration of time slowly leaks region servers. I see failures on the
RESTApi side go unreported on the ChaosMonkey side. It seems like
`RuntimeException`s are being thrown and lost.
`PolicyBasedChaosMonkey` uses a primitive means of thread management
anyway. Update to use a thread pool, thread groups, and an
uncaughtExceptionHandler.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
- Prior to this commit, there were 13 commands that unintentionally return the
number of lines they print (usually one). This commit ensures that they
return the value documented by the help text, or nil if there is not a simple
logical value to return.
- Fixes 6 hbase-shell commands that return String rather than TrueClass or
FalseClass
- Use double-bang to cast truthy values to TrueClass and FalseClass so that
ruby's to_s can reliably print true or false without using ternary operators
- Updates tests for is_disabled, is_enabled, disable_rpc_throttle,
enable_rpc_throttle, disable_exceed_throttle_quota,
enable_exceed_throttle_quota, clear_deadservers, snapshot_cleanup_switch,
snapshot_cleanup_enabled, and balancer to check return values
- Adds new tests for balance_switch, balancer_enabled, normalizer_switch,
normalizer_enabled, catalog_janitor_switch, catalogjanitor_enabled,
cleaner_chore_switch, cleaner_chore_enabled, splitormerge_switch, and
splitormerge_enabled
signed-off-by: stack <stack@apache.org>
- make sure to always clean up excess meta replicas not just when their
number get decreased
- make sure NotServingRegionException is handled properly even when
wrapped
- add test
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Bharath Vissapragada
Sandeep Pal
WenFeiYi
Michael Stack
niuyulin
YeChao Chen
Elliot
Pankaj
Duo Zhang
Toshihiro Suzuki
Wellington Ramos Chevreuil
Viraj Jasani
Guanghao Zhang
Peter Somogyi
surbhi
Nick Dimiduk
Mingliang Liu
bsglz
sguggilam
Sanjeet Nishad
Mohammad Arshad
Lars
huaxiangsun
BukrosSzabolcs
Bo Cui
Geoffrey Jacoby