18072 Commits

Author SHA1 Message Date
Elliot
7974a1e9bf
HBASE-11676 Scan FORMATTER is not applied for columns using non-printable name in shell (#2161)
- In HBase::Table, the instance variable @converters is used to map column
  names to converters. This patch fixes how HBase::Table#_get_internal and
  HBase::Table#_scan_internal generate the column name key used to access
  @converters.
- Refactor parsing of family:qualifier:converter specifications so that the
  code is more readable and reusable. As part of this change, I added two
  private methods and marked HBase::Table#set_converter as deprecated for
  removal in HBase 4.0.0.
- Add unit testing for the fixed bug

Signed-off-by: stack <stack@apache.org>
2020-07-28 20:43:19 -07:00
Pankaj
4471a644f6
HBASE-24738 [Shell] processlist command fails with ERROR: Unexpected end of file from server when SSL enabled (#2123)
Signed-off-by: Elliot Miller <elliot_miller@apple.com>
2020-07-28 09:10:42 -07:00
Elliot
7eff07d6bf
HBASE-11686 Shell code should create a binding / irb workspace instead of polluting the root namespace (#2141)
* HBASE-11686 Shell code should create a binding / irb workspace instead of polluting the root namespace

- Refactor Shell.export_commands to define commands using ruby lambdas.
  Additionally, this change stores a reference to shell_inst in scope so that
  we no longer need to assume the existance of the variable @shell.
- Add logic to Shell class for constructing an IRB workspace with its own
  binding and non-global receiver. This workspace is loaded with all HBase and
  IRB commands.
- Create new method on Shell for evaluating input from an arbitrary IO instance
  within the created IRB workspace. This is based on work by Hsieh that was
  previously in bin/hirb.rb. This method is more generic and more testable.
  This single pattern can be used for both executing files and reading from
  stdin, therefore reducing complexity.
- Move special 'help' and 'tools' command definitions to shell.rb. These
  commands are tightly linked with an instance of the shell, so it is easiest
  to have them defined together.
- Remove all global includes of HBaseConstants from ruby test files. Before
  this change, tests were loading these constants into the top level, which
  could cause tests to pass that should really fail.
- Try to reduce the number of places that constants are included. I think it's
  best to reference each ruby constant's full name, but where that would cause
  a big diff I instead moved the include to the innermost Module or Class.
- Update docs and comments
- Remove unneccessary includes
- Add shell --top-level-cmds compatibility flag. Since this PR removes all the
  HBase symbols from the top-level receiver (ie. main Object), it is possible
  (albeit unlikely) that this will break operator scripts. This flag will
  export all the commands at the top-level like the shell previously did.

* HBASE-11686 Light refactoring with added unit tests

- Fixes some constants references by admin test 2
- Install HBase commands as singleton methods on recevier instances so that
  multiple receivers may exist.
- Rename new flag from --top-level-cmds to --top-level-defs to be more
  semantically accurate.
- Create new helper method Shell::Shell#export_all to install @hbase, @shell,
  constants, and all hbase commands to a target receiver. As a result, the
  HBaseReceiver became very simple and could be moved to shell.rb.
- Add unit tests for Shell::Shell#eval_io and Shell::Shell#export_all
- Add @hbase and @shell to hbase-shell IRB workspace
- Fix robocop issues within patch

* Typo s/is/if/
2020-07-27 20:56:09 -07:00
Duo Zhang
7e6e7a7051
HBASE-24770 Reimplement the Constraints API and revisit the IA annotations on related classes (#2140)
Signed-off-by: stack <stack@apache.org>
2020-07-28 09:24:55 +08:00
Toshihiro Suzuki
477debdc74
HBASE-24775 [hbtop] StoreFile size should be rounded off (#2144)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-28 08:14:45 +09:00
YeChao Chen
82d09904f2
HBASE-24777 InfoServer support ipv6 host and port
Closes #2147

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-27 20:45:45 +05:30
Wellington Ramos Chevreuil
8c0d7fa5b8
HBASE-24758 Avoid flooding replication source RSes logs when no sinks… (#2118)
Signed-off-by: Josh Elser <elserj@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-27 10:08:13 +01:00
WenFeiYi
975cdf7b88
HBASE-24665 all wal of RegionGroupingProvider together roll (#2021)
Co-authored-by: wen_yi <liu.wenwen@immomo.com>
Signed-off-by: Anoop <anoopsamjohn@apache.org>
Signed-off-by: Ramkrishna <ramkrishna@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-24 22:54:25 +05:30
Viraj Jasani
09e7ccd42d
HBASE-24757 : ReplicationSink should limit row count in batch mutation based on hbase.rpc.rows.warning.threshold
Closes #2127

Signed-off-by: stack <stack@apache.org>
2020-07-24 13:16:02 +05:30
Duo Zhang
0b85729da4
HBASE-24762 Purge protobuf java 2.5.0 dependency (#2128)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-07-24 11:48:35 +08:00
Guanghao Zhang
32c7012ad7
HBASE-24743 Reject to add a peer which replicate to itself earlier (#2122)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-07-24 08:08:20 +08:00
Peter Somogyi
e7963f6486
HBASE-24763 Remove 2.1 Documentation direct link from website (#2129)
2.1 is EOM so the 2.1 Documentation link can be removed.
The content is still available under https://hbase.apache.org/2.1

Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-07-23 22:38:48 +02:00
surbhi
8191fbdd7d HBASE-22146 Removing a namespace-level space quota does not remove policies against contained tables
Closes #1935

Signed-off-by: Josh Elser <elserj@apache.org>
2020-07-21 20:12:46 -04:00
Nick Dimiduk
a6e3db5ba5 HBASE-24662 Update DumpClusterStatusAction to notice changes in region server count
Sometimes running chaos monkey, I've found that we lose accounting of
region servers. I've taken to a manual process of checking the
reported list against a known reference. It occurs to me that
ChaosMonkey has a known reference, and it can do this accounting for
me.

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-21 15:56:22 -07:00
Viraj Jasani
f35c5eaadd
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses) (#2109)
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-21 20:24:13 +05:30
Viraj Jasani
32796ad24e
Revert "HBASE-24743 Reject to add a peer which replicate to itself earlier (#2071)"
This reverts commit 6cf013ddc426a01d3b1e0266b80a6239de630602.

TestReplicationAdmin and TestReplicationShell are broken on branch-2 and master respectively
2020-07-21 20:17:27 +05:30
Mingliang Liu
3d270ba6f9
HBASE-24696 Include JVM information on Web UI under "Software Attributes"
Closes #2087

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-07-21 16:53:58 +05:30
bsglz
fccd03ffa2
HBASE-24555 Correct the description of hbase.hregion.max.filesize (#1895)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-21 11:04:42 +01:00
Peter Somogyi
ea96743dd2
HBASE-24710 Incorrect checksum calculation in saveVersion.sh (#2056)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-07-21 10:40:22 +02:00
Viraj Jasani
f2c087caeb
Revert "HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses)"
This reverts commit 16f306b4a17e0f3ab8e3381bde9b2c4c7f742ef9.

TestAdminShell and TestThriftHBaseServiceHandler are failing consistently
2020-07-21 13:50:27 +05:30
sguggilam
430602aed8
HBASE-24740 Enable journal logging for HBase snapshot operation (#2104)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-07-20 18:11:41 -07:00
Guanghao Zhang
6cf013ddc4
HBASE-24743 Reject to add a peer which replicate to itself earlier (#2071)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-07-21 08:37:56 +08:00
Nick Dimiduk
7ebc617026 HBASE-24658 Update PolicyBasedChaosMonkey to handle uncaught exceptions
Running `ServerKillingChaosMonkey` via `RESTApiClusterManager` for any
duration of time slowly leaks region servers. I see failures on the
RESTApi side go unreported on the ChaosMonkey side. It seems like
`RuntimeException`s are being thrown and lost.

`PolicyBasedChaosMonkey` uses a primitive means of thread management
anyway. Update to use a thread pool, thread groups, and an
uncaughtExceptionHandler.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-20 16:57:11 -07:00
Sanjeet Nishad
a857c6d90c
HBASE-24714 Error message is displayed in UI of table's compaction state if any region of that table is not open
Closes #2047

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-20 21:40:23 +05:30
Zheng Wang
f73b9085ad
HBASE-24748 Add hbase.master.balancer.stochastic.moveCost.offpeak to doc as support dynamically change
Closes #2101

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-20 15:20:56 +05:30
Zheng Wang
9b32943697
HBASE-24709 Support MoveCostFunction use a lower multiplier in offpeak hours
Closes #2099

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-07-19 20:09:12 +05:30
Michael Stack
9b02a26a1d HBASE-24698 Turn OFF Canary WebUI as default (#2037)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-07-17 15:52:52 -07:00
Elliot
4b3ef815eb
HBASE-24722 Update commands with unintentional return values (#2058)
- Prior to this commit, there were 13 commands that unintentionally return the
  number of lines they print (usually one). This commit ensures that they
  return the value documented by the help text, or nil if there is not a simple
  logical value to return.
- Fixes 6 hbase-shell commands that return String rather than TrueClass or
  FalseClass
- Use double-bang to cast truthy values to TrueClass and FalseClass so that
  ruby's to_s can reliably print true or false without using ternary operators
- Updates tests for is_disabled, is_enabled, disable_rpc_throttle,
  enable_rpc_throttle, disable_exceed_throttle_quota,
  enable_exceed_throttle_quota, clear_deadservers, snapshot_cleanup_switch,
  snapshot_cleanup_enabled, and balancer to check return values
- Adds new tests for balance_switch, balancer_enabled, normalizer_switch,
  normalizer_enabled, catalog_janitor_switch, catalogjanitor_enabled,
  cleaner_chore_switch, cleaner_chore_enabled, splitormerge_switch, and
  splitormerge_enabled

signed-off-by: stack <stack@apache.org>
2020-07-17 11:44:30 -07:00
Wellington Ramos Chevreuil
bd42c75cac
HBASE-24747 Log an ERROR if HBaseSaslRpcServer initialisation fails with an uncaught exception (#2079)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-07-17 18:50:03 +01:00
Mohammad Arshad
1781185b7b
HBASE-24675: On Master restart all servers are assigned to default rsgroup
Closes #2053

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-17 22:38:19 +05:30
Zheng Wang
6ae2fdd6b2
HBASE-24746 The sort icons overlap the col name in master UI
Closes #2078

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-17 21:47:17 +05:30
Viraj Jasani
16f306b4a1
HBASE-24718 : Generic NamedQueue framework for multiple use-cases (Refactor SlowLog responses)
Closes #2052

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-17 19:16:22 +05:30
Lars
a4e5ccc798 HBASE-24742 Improve performance of SKIP vs SEEK logic. 2020-07-16 13:17:05 -07:00
huaxiangsun
2505c7760d
HBASE-24581 Skip compaction request/check for replica regions at the early stage. (#1986)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-07-14 15:49:20 -07:00
huaxiangsun
1360bee7f9
HBASE-24705 MetaFixer#fixHoles() does not include the case for read replicas (i.e, replica regions are not created) (#2062)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-14 15:48:19 -07:00
WenFeiYi
37a2994bf8
HBASE-24615 MutableRangeHistogram#updateSnapshotRangeMetrics doesn't calculate the distribution for last bucket
Closes #1962

Signed-off-by: David Manning
Signed-off-by: Rushabh
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-15 00:08:46 +05:30
Nick Dimiduk
ee226de608
HBASE-24566 Add 2.3.0 to the downloads page (#2061)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-14 10:09:35 -07:00
Nick Dimiduk
927df472d9
HBASE-24487 Add 2.3 Documentation to the website (#2060)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-14 09:56:32 -07:00
BukrosSzabolcs
3e709c6f53
HBASE-24720: Meta replicas not cleaned when disabled (#2057)
- make sure to always clean up excess meta replicas not just when their
number get decreased
- make sure NotServingRegionException is handled properly even when
wrapped
- add test

Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2020-07-14 18:24:55 +02:00
Mohammad Arshad
70cfe2525e
HBASE-24721: rename_rsgroup overwriting the existing rsgroup
Closes #2059

Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-14 21:30:23 +05:30
Bo Cui
d132c4b53d
HBASE-24693 regioninfo#isLast() should point to EMPTY_END_ROW (readability improvement)
Closes #2051

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-14 15:41:44 +05:30
bsglz
3bd54217a0
HBASE-24664 Some changing of split region by overall region size rath… (#2011)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-07-13 10:00:48 +01:00
Geoffrey Jacoby
af1cc2fc44
HBASE-23744 - FastPathBalancedQueueRpcExecutor should enforce queue length of 0
Closes #1094

Signed-off-by: Xu Cang <xucang@apache.org>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-13 12:15:10 +05:30
bsglz
724f0478ed
HBASE-24404 Support flush a single column family of region (#2032)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2020-07-10 18:13:50 +01:00
Zheng Wang
b2321b3583
HBASE-24586 Add table level locality in table.jsp
Closes #1926

Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-10 21:59:59 +05:30
Peter Somogyi
6c9534c804
HBASE-24711 Previous RC Release notes are not removed (#2046)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-07-10 18:24:04 +02:00
Viraj Jasani
f0bc33122e
HBASE-24712 : Remove flakes from TestMasterNoCluster#testStopDuringStart (#2045)
Forward-port from 91da188d78
2020-07-10 20:41:03 +05:30
Joseph295
67dfbe0406
HBASE-24663 Add procedure process time statistics UI (#2035)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-10 15:03:52 +08:00
niuyulin
5de1dff8f6
HBASE-24653 Show snapshot owner on Master WebUI (#2016)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-10 15:03:05 +08:00
XinSun
1d0e285a6f
HBASE-24489 Rewrite TestClusterRestartFailover.test since namespace table is gone on on master (#2025)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-07-10 10:03:57 +08:00