Commit Graph

525 Commits

Author SHA1 Message Date
Sean Busbey fd0ecadbb9
HBASE-14067 bundle ruby files for hbase shell into a jar.
* removes some cruft from the hbase-shell pom that appears to be from coping the hbase-server pom long ago
* puts the ruby scripts into the hbase-shell jar following the guide from jruby for packaging
* removes hard coding the location of the implementation scripts from our runtime
* removes hard coding the load path for the implementation from the test code (leaves hard coding the test code location)
* provides a work around for a name conflict between our shell and the ruby stdlib shell.

closes #2515

Signed-off-by: Michael Stack <stack@apache.org>
2020-10-13 11:12:07 -05:00
Viraj Jasani 35d59dc570
HBASE-24528 : BalancerDecision queue implementation in HMaster with Admin API
* Admin API getLogEntries() for ring buffer use-cases: so far, provides balancerDecision and slowLogResponse
* Refactor RPC call for similar use-cases
* Single RPC API getLogEntries() for both Master.proto and Admin.proto

Closes #2261

Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-09-16 20:56:54 +05:30
bsglz c5ca191921
HBASE-24875 Remove the force param for unassign since it dose not take effect any more (#2254)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-09-15 22:05:03 -05:00
Bo Cui 1e8db480b3
HBASE-24937 table.rb use LocalDateTime to replace Instant (#2351)
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
2020-09-04 20:20:39 +05:30
bsglz 4021f4577c
HBASE-24890 The command regioninfo is not available (#2263)
* HBASE-24890 The command regioninfo is not available

* add ut for command regioninfo
2020-08-21 20:47:03 -07:00
Guanghao Zhang 54fe81eb56
HBASE-24912 Enlarge MemstoreFlusherChore/CompactionChecker period for unit test (#2285)
Signed-off-by: stack <stack@apache.org>
2020-08-21 12:41:20 +08:00
Elliot 98e35842eb
HBASE-24806 Small Updates to Functionality of Shell IRB Workspace (#2232)
* HBASE-24806 Small Updates to Functionality of Shell IRB Workspace

- Move exception handler from Shell::Shell#eval_io to new method,
  Shell::Shell#exception_handler
- Add unit tests for Shell::Shell#exception_handler
- Change Shell::Shell#eval_io to no longer raise SystemExit when any error is
  seen and update unit test
- Update ruby test runner to catch SystemExit and fail to avoid tests that
  cause the test runner to incorrectly exit successfully
- Add Hbase::Loader module to find ruby scripts in the $LOAD_PATH and classpath
  using JRuby's loader.
- In hbase-shell, install IRB commands before exporting HBase commands. The
  HBase commands will override the IRB commands, and no warning will be
  printed.

* Remove unused variables from shell_test

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-08-18 13:14:34 -07:00
Elliot 6789aca9a0
HBASE-24874 Fix hbase-shell access to ModifiableTableDescriptor methods (#2268)
* HBASE-24874 Fix hbase-shell access to ModifiableTableDescriptor methods

- Fix hbase-shell access in JDK 11 for calls to
  TableDescriptorBuilder.toCoprocessorDescriptor and
  ModifiableTableDescriptor.toStringTableAttributes.
- Allow coprocessors to be specified using a Ruby hash in the hbase-shell alter
  command and replace usage in the help text. The previous String overload of
  the alter command will continue to work and is still covered by a unit test,
  but will no longer be suggested in the alter command help.

* Update patch

- Add warning over toCoprocessorDescriptor noting the usage by hbase-shell
- Add constants to hbase_constants for coprocessor specification
- Document usage of ModifiableTableDescriptor.toStringTableAttributes

* Convert comment over toCoprocessorDescriptor into docstring

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-08-18 12:58:40 -07:00
Nick Dimiduk ecc0c67077 HBASE-24627 Normalize one table at a time
Introduce an additional method to our Admin interface that allow an
operator to selectivly run the normalizer. The IPC protocol supports
general table name select via compound filter.

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-08-18 09:46:29 -07:00
Zheng Wang c81ef7368e
HBASE-24854 Correct the help content of assign and unassign commands in hbase shell
Closes #2241

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-08-13 15:37:01 +05:30
Nick Dimiduk d2afda309d HBASE-24843 Sort the constants in `hbase_constants.rb`
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-08-11 13:29:36 -07:00
bsglz 485e0d2fa4
HBASE-24694 Support flush a single column family of table (#2179)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-08-07 11:28:53 +01:00
bsglz 9b49bd6b66
HBASE-24826 Add some comments for processlist in hbase shell (#2207)
Signed-off-by: Elliot Miller <elliot_miller@apple.com>
Signed-off-by: stack <stack@apple.com>
2020-08-06 19:28:37 -07:00
Elliot 7c4d66a277
HBASE-24803 Unify hbase-shell ::Shell::Commands::Command#help behavior (#2178)
- Update all hbase-shell commands to return help text implicitly
- Add hbase-shell test to ensure all help methods return unindented text
- Set the outermost indentation to zero for the help text of all hbase-shell
  commands by using the squiggly heredoc added in Ruby 2.3.0.
- Document return value of hbck_chore_run and add usage examples to truncate commands
- Update docs to reflect current return value of list command

Signed-off-by: stack <stack@apache.org>
2020-07-31 07:41:49 -07:00
Elliot 7974a1e9bf
HBASE-11676 Scan FORMATTER is not applied for columns using non-printable name in shell (#2161)
- In HBase::Table, the instance variable @converters is used to map column
  names to converters. This patch fixes how HBase::Table#_get_internal and
  HBase::Table#_scan_internal generate the column name key used to access
  @converters.
- Refactor parsing of family:qualifier:converter specifications so that the
  code is more readable and reusable. As part of this change, I added two
  private methods and marked HBase::Table#set_converter as deprecated for
  removal in HBase 4.0.0.
- Add unit testing for the fixed bug

Signed-off-by: stack <stack@apache.org>
2020-07-28 20:43:19 -07:00
Pankaj 4471a644f6
HBASE-24738 [Shell] processlist command fails with ERROR: Unexpected end of file from server when SSL enabled (#2123)
Signed-off-by: Elliot Miller <elliot_miller@apple.com>
2020-07-28 09:10:42 -07:00
Elliot 7eff07d6bf
HBASE-11686 Shell code should create a binding / irb workspace instead of polluting the root namespace (#2141)
* HBASE-11686 Shell code should create a binding / irb workspace instead of polluting the root namespace

- Refactor Shell.export_commands to define commands using ruby lambdas.
  Additionally, this change stores a reference to shell_inst in scope so that
  we no longer need to assume the existance of the variable @shell.
- Add logic to Shell class for constructing an IRB workspace with its own
  binding and non-global receiver. This workspace is loaded with all HBase and
  IRB commands.
- Create new method on Shell for evaluating input from an arbitrary IO instance
  within the created IRB workspace. This is based on work by Hsieh that was
  previously in bin/hirb.rb. This method is more generic and more testable.
  This single pattern can be used for both executing files and reading from
  stdin, therefore reducing complexity.
- Move special 'help' and 'tools' command definitions to shell.rb. These
  commands are tightly linked with an instance of the shell, so it is easiest
  to have them defined together.
- Remove all global includes of HBaseConstants from ruby test files. Before
  this change, tests were loading these constants into the top level, which
  could cause tests to pass that should really fail.
- Try to reduce the number of places that constants are included. I think it's
  best to reference each ruby constant's full name, but where that would cause
  a big diff I instead moved the include to the innermost Module or Class.
- Update docs and comments
- Remove unneccessary includes
- Add shell --top-level-cmds compatibility flag. Since this PR removes all the
  HBase symbols from the top-level receiver (ie. main Object), it is possible
  (albeit unlikely) that this will break operator scripts. This flag will
  export all the commands at the top-level like the shell previously did.

* HBASE-11686 Light refactoring with added unit tests

- Fixes some constants references by admin test 2
- Install HBase commands as singleton methods on recevier instances so that
  multiple receivers may exist.
- Rename new flag from --top-level-cmds to --top-level-defs to be more
  semantically accurate.
- Create new helper method Shell::Shell#export_all to install @hbase, @shell,
  constants, and all hbase commands to a target receiver. As a result, the
  HBaseReceiver became very simple and could be moved to shell.rb.
- Add unit tests for Shell::Shell#eval_io and Shell::Shell#export_all
- Add @hbase and @shell to hbase-shell IRB workspace
- Fix robocop issues within patch

* Typo s/is/if/
2020-07-27 20:56:09 -07:00
Guanghao Zhang 32c7012ad7
HBASE-24743 Reject to add a peer which replicate to itself earlier (#2122)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-07-24 08:08:20 +08:00
Viraj Jasani f35c5eaadd
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses) (#2109)
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-21 20:24:13 +05:30
Elliot 4b3ef815eb
HBASE-24722 Update commands with unintentional return values (#2058)
- Prior to this commit, there were 13 commands that unintentionally return the
  number of lines they print (usually one). This commit ensures that they
  return the value documented by the help text, or nil if there is not a simple
  logical value to return.
- Fixes 6 hbase-shell commands that return String rather than TrueClass or
  FalseClass
- Use double-bang to cast truthy values to TrueClass and FalseClass so that
  ruby's to_s can reliably print true or false without using ternary operators
- Updates tests for is_disabled, is_enabled, disable_rpc_throttle,
  enable_rpc_throttle, disable_exceed_throttle_quota,
  enable_exceed_throttle_quota, clear_deadservers, snapshot_cleanup_switch,
  snapshot_cleanup_enabled, and balancer to check return values
- Adds new tests for balance_switch, balancer_enabled, normalizer_switch,
  normalizer_enabled, catalog_janitor_switch, catalogjanitor_enabled,
  cleaner_chore_switch, cleaner_chore_enabled, splitormerge_switch, and
  splitormerge_enabled

signed-off-by: stack <stack@apache.org>
2020-07-17 11:44:30 -07:00
bsglz 724f0478ed
HBASE-24404 Support flush a single column family of region (#2032)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-10 18:13:50 +01:00
Elliot 83f27b5287
HBASE-20819 Use TableDescriptor to replace HTableDescriptor in hbase-shell module (#1959)
* HBASE-20819 Use TableDescriptor to replace HTableDescriptor in hbase-shell module

* Resolve whitespace issues

* Use removeValue rather than setValue where possible in hbase-shell

* Add String removeValue overload to TableDescriptorBuilder

* Convert remaining usages of HTableDescriptor in security.rb

* Rename hcd (HColumnDescriptor) to cfd (ColumnFamilyDescriptor)

* Add back 3 constants to HBaseConstants in hbase-shell

* Fix bad constant reference

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-06-26 12:39:34 -07:00
XinSun 4b89ee24f3
HBASE-24431 RSGroupInfo add configuration map to store something extra (#1782)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-24 18:46:09 +08:00
Duo Zhang c91829bb41
HBASE-24491 Remove HRegionInfo (#1830)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-06-05 22:19:01 +08:00
bsglz 3963f62453
HBASE-24486 Remove the unused method online in admin.rb (#1824)
Signed-off-by: clarax98007@gmail.com
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-06-02 11:22:29 -07:00
Wellington Ramos Chevreuil e5345b3a7c
HBASE-21406 "status 'replication'" should not show SINK if the cluste… (#1761)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-06-02 09:27:16 +01:00
Viraj Jasani f0f9a20cd7
HBASE-23941 : FilterBy operator support in get_slowlog_responses API (#1793)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-05-30 12:59:59 +05:30
Wellington Ramos Chevreuil 9d9f07be36
HBASE-24413 HBASE-22259 Removed deprecated getTimeStampOfLastShippedOp method from ReplicationLoadSource, but there were still references to this method on ruby admin.rb (#1754)
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off by: Viraj Jasani <vjasani@apache.org>
2020-05-22 10:46:33 +01:00
Duo Zhang 8601416ee8
HBASE-24309 Avoid introducing log4j and slf4j-log4j dependencies for modules other than hbase-assembly (#1640)
Signed-off-by: stack <stack@apache.org>
2020-05-12 12:03:30 +08:00
bsglz 2cafe81e9c
HBASE-24335 Support deleteall with ts but without column in shell mode (#1668)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-05-07 18:18:57 +01:00
Michael Stack 5488124be0 HBASE-24284 [h3/jdk11] REST server won't start Exclude transitive includes of jax-rs 1.x and then explicitly include jax-rs 2.x glassfish impl for REST context when hadoop3. (#1625) 2020-05-05 15:36:01 -07:00
Duo Zhang a2e373dc32 HBASE-24281 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:24:52 +08:00
Duo Zhang a51a1f4d10 Revert "HBASE-24181 Remove some stale hadoop.version properties in the pom for sub modules (#1602)"
This reverts commit af40bff8be.
2020-04-29 18:24:41 +08:00
Duo Zhang af40bff8be
HBASE-24181 Remove some stale hadoop.version properties in the pom for sub modules (#1602)
Signed-off-by: Lijin Bin <binlijin@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-29 18:02:07 +08:00
Duo Zhang 6928674eb8
HBASE-24228 Merge the code in hbase-hadoop2-compat module to hbase-hadoop-compat (#1563)
Signed-off-by: stack <stack@apache.org>
2020-04-29 10:34:53 +08:00
Reid Chan 9cddac01ba
HBASE-24196 [Shell] Add rename rsgroup command in hbase shell (#1551)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-04-21 10:22:33 +08:00
Duo Zhang 1f66806c96
HBASE-24170 Remove hadoop-2.0 profile (#1495)
Signed-off-by: stack <stack@apache.org>
2020-04-16 18:57:40 +08:00
EEPROM 35ea4acf48
HBASE-24176 user_permission '.*' command failed to show all table permissions (#1499)
* user_permission failed when show table permissions with '.*' argument

Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
2020-04-14 10:14:21 +08:00
Jan Hentschel 4ddf55d20a
HBASE-23843 Removed deprecated Scan(byte[]) from Scan
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-04-02 11:44:34 +02:00
Viraj Jasani 0dcbf80583
HBASE-23937 : Support Online LargeLogs similar to SlowLogs APIs (#1346)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-04-01 18:22:22 +05:30
Jan Hentschel fb17ecdd07
HBASE-23842 Removed deprecated Scan(byte[], byte[]) from Scan
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-04-01 13:11:21 +02:00
Michael Stack 3e1cf00c71 HBASE-23993 Use loopback for zk standalone server in minizkcluster (#1291)
hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 Have client and server use loopback instead of 'localhost'

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-03-17 20:26:06 -07:00
stack f66cbe1a40 HBASE-23992 Fix TestAdminShell and TestQuotasShell mistakenly broken by parent commit 2020-03-14 15:18:21 -07:00
Duo Zhang c5aa2bc121
HBASE-23077 move entirely to spotbugs (#1265)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-03-12 10:18:09 +08:00
stack cf121227aa HBASE-23930 Shell should attempt to format `timestamp` attributes as ISO-8601
Make display of Cell timestamp be ISO8601 format instead of pure milliseconds.
2020-03-11 17:15:15 -07:00
stack 0b18a7dc26 HBASE-23929 Shell formatter for for meta table should pretty-print values of info:merge columns 2020-03-11 16:41:30 -07:00
Michael Stack aaf9a148a0 HBASE-23956 Use less resources running tests (#1266)
Add being able to configure netty thread counts. Enable socket reuse
(should not have any impact).

hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BlockingRpcConnection.java
 Rename the threads we create in here so they are NOT named same was
 threads created by Hadoop RPC.

hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/DefaultNettyEventLoopConfig.java
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcClient.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java
 Allow configuring eventloopgroup thread count (so can override for
 tests)

hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/HttpProxyExample.java
 Enable socket resuse.

hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
 Enable socket resuse and config for how many threads to use.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
hbase-server/src/main/java/org/apache/hadoop/hbase/util/ModifyRegionUtils.java
 Thread name edit; drop the redundant 'Thread' suffix.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
 Make closeable and shutdown executor when called.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
 Call close on HFileReplicator

hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationBase.java
 HDFS creates lots of threads. Use less of it so less threads overall.

hbase-server/src/test/resources/hbase-site.xml
hbase-server/src/test/resources/hdfs-site.xml
 Constrain resources when running in test context.

hbase-server/src/test/resources/log4j.properties
 Enable debug on netty to see netty configs in our log

pom.xml
 Add system properties when we launch JVMs to constrain thread counts in
 tests

 Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-11 10:37:50 -07:00
Viraj Jasani 4ef326be15
HBASE-23909 : list_regions raise correct error if it is being splitted/merged/transitioning (#1238)
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-03-06 13:01:17 +05:30
Duo Zhang 7f2d823164 HBASE-23818 Cleanup the remaining RSGroupInfo.getTables call in the code base (#1152)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-03-02 15:43:40 +08:00
Duo Zhang 37e87aeca7 HBASE-23807 Make rsgroup related shell command to use the new admin methods (#1148)
Signed-off-by: stack <stack@apache.org>
2020-03-02 15:43:40 +08:00