Commit Graph

484 Commits

Author SHA1 Message Date
Michael Stack 3e1cf00c71 HBASE-23993 Use loopback for zk standalone server in minizkcluster (#1291)
hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MiniZooKeeperCluster.java
 Have client and server use loopback instead of 'localhost'

Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-03-17 20:26:06 -07:00
stack f66cbe1a40 HBASE-23992 Fix TestAdminShell and TestQuotasShell mistakenly broken by parent commit 2020-03-14 15:18:21 -07:00
Duo Zhang c5aa2bc121
HBASE-23077 move entirely to spotbugs (#1265)
Signed-off-by: Sean Busbey <busbey@apache.org>
2020-03-12 10:18:09 +08:00
stack cf121227aa HBASE-23930 Shell should attempt to format `timestamp` attributes as ISO-8601
Make display of Cell timestamp be ISO8601 format instead of pure milliseconds.
2020-03-11 17:15:15 -07:00
stack 0b18a7dc26 HBASE-23929 Shell formatter for for meta table should pretty-print values of info:merge columns 2020-03-11 16:41:30 -07:00
Michael Stack aaf9a148a0 HBASE-23956 Use less resources running tests (#1266)
Add being able to configure netty thread counts. Enable socket reuse
(should not have any impact).

hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BlockingRpcConnection.java
 Rename the threads we create in here so they are NOT named same was
 threads created by Hadoop RPC.

hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/DefaultNettyEventLoopConfig.java
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcClient.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/AsyncFSWAL.java
 Allow configuring eventloopgroup thread count (so can override for
 tests)

hbase-examples/src/main/java/org/apache/hadoop/hbase/client/example/HttpProxyExample.java
 Enable socket resuse.

hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
 Enable socket resuse and config for how many threads to use.

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
hbase-server/src/main/java/org/apache/hadoop/hbase/util/ModifyRegionUtils.java
 Thread name edit; drop the redundant 'Thread' suffix.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
 Make closeable and shutdown executor when called.

hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
 Call close on HFileReplicator

hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationBase.java
 HDFS creates lots of threads. Use less of it so less threads overall.

hbase-server/src/test/resources/hbase-site.xml
hbase-server/src/test/resources/hdfs-site.xml
 Constrain resources when running in test context.

hbase-server/src/test/resources/log4j.properties
 Enable debug on netty to see netty configs in our log

pom.xml
 Add system properties when we launch JVMs to constrain thread counts in
 tests

 Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-03-11 10:37:50 -07:00
Viraj Jasani 4ef326be15
HBASE-23909 : list_regions raise correct error if it is being splitted/merged/transitioning (#1238)
Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2020-03-06 13:01:17 +05:30
Duo Zhang 7f2d823164 HBASE-23818 Cleanup the remaining RSGroupInfo.getTables call in the code base (#1152)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-03-02 15:43:40 +08:00
Duo Zhang 37e87aeca7 HBASE-23807 Make rsgroup related shell command to use the new admin methods (#1148)
Signed-off-by: stack <stack@apache.org>
2020-03-02 15:43:40 +08:00
Duo Zhang f0a13bb74d HBASE-22676 Move all the code in hbase-rsgroup to hbase-server and remove hbase-rsgroup module (#399)
Signed-off-by: Zheng Hu <openinx@gmail.com>
2020-03-02 15:43:40 +08:00
Viraj Jasani b24ea32b3b
HBASE-22978 : Online slow response log
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-02-28 19:47:59 +05:30
Sakthi 5f61df4792
HBASE-22827 Expose multi-region merge in shell and Admin API (#1138)
mergeRegionsAsync admin API with two regions as parameters deprecated since 2.3.0 and removed from 4.0
merge_region shell command now supports multiple regions merge since 2.3.0 & 3.0.0

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Esteban Gutierrez <esteban@apache.org>
Signed-off-by: Josh Elser <elserj@apache.org>
2020-02-13 14:40:46 -08:00
Duo Zhang 4479cb2313 HBASE-23809 Addendum remove rs group code in AbstractTestShell 2020-02-08 22:46:08 +08:00
Duo Zhang 546828685e
HBASE-23809 The RSGroup shell test is missing (#1142)
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2020-02-08 17:10:00 +08:00
Michael Stack ba3463d9de HBASE-23055 Alter hbase:meta (#1043)
Make hbase:meta region schema dynamic.

Patch has been under development a good while and its focus has changed
a few times so its bloated with fixup from older versions.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
 Shut down access to internals and removed unused methods.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
 Cleanup/refactor section on replica-handling.

M hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
 Get hbase:meta schema from filesystem rather than from hard-coding.
2020-01-21 15:02:40 -08:00
stack c8496bc694 Revert "HBASE-23055 Alter hbase:meta"
This reverts commit 9abdb7b5ae.
2020-01-11 09:17:17 -08:00
stack 9abdb7b5ae HBASE-23055 Alter hbase:meta
Make it so hbase:meta can be altered. TableState for hbase:meta
    was hardcoded ENABLED. Make it dynamic. State is now kept in
    current active Master. It is transient so falls back to default
    if Master crashes. Add to registry a getMetaTableState
    which reads mirrored state from zookeeper (NOT from Master and
    defaults ENABLED if no implementation or error fetching state).
    hbase:meta schema will be bootstrapped from the filesystem.
    Changes to filesystem schema are atomic so we should be ok if
    Master fails mid-edit (TBD). Undoes a bunch of guards that
    prevented our being able to edit hbase:meta.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     Add fetching of hbase:meta table state from registry. Adds cache of
     tablestates w/ a ttl of 1 second (adjustable).

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
      Add querying registry for hbase:meta table state.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
      Add querying of mirrored table state for hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta. Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-01-09 16:25:16 -08:00
stack b0233a294f HBASE-23554 Encoded regionname to regionname utility (#923); ADDENDUM 2019-12-11 11:02:59 -08:00
Michael Stack 8742265d85 HBASE-23554 Encoded regionname to regionname utility (#923)
Adds shell command regioninfo:

  hbase(main):001:0>  regioninfo '0e6aa5c19ae2b2627649dc7708ce27d0'
  {ENCODED => 0e6aa5c19ae2b2627649dc7708ce27d0, NAME => 'TestTable,,1575941375972.0e6aa5c19ae2b2627649dc7708ce27d0.', STARTKEY => '', ENDKEY => '00000000000000000000299441'}
  Took 0.4737 seconds

Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-12-11 08:57:59 -08:00
Karthik Palanisamy a553b78c1c HBASE-23360 [CLI] Fix help command 'set_quota' for removing limits (#896) 2019-12-11 10:06:40 -06:00
Karthik Palanisamy df9cbd4254 HBASE-23336 [CLI] Incorrect row(s) count 'clear_deadservers' (#875)
Signed-off-by: Guangxu Cheng <gxcheng@apache.org>
Signed-off-by: Lijin Bin <binlijin@apache.org>
2019-11-26 19:20:33 +08:00
Mingliang Liu 4cc899d8e3 HBASE-23290 shell processlist command is broken 2019-11-14 16:03:29 -08:00
Karthik Palanisamy 0f910f0c32 HBASE-23176 delete_all_snapshot does not work with regex (#725) 2019-10-17 20:35:56 +08:00
Karthik Palanisamy 065993294f HBASE-23154 list_deadservers return incorrect no of rows (#717)
Signed-off-by: stack <stack@apache.org>
2019-10-12 09:13:50 -07:00
Karthik Palanisamy c0a09ccbbd HBASE-23152 Compaction_switch does not work by RegionServer name (#713)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
 Signed-off-by: stack <stack@apache.org>
2019-10-12 09:08:20 -07:00
Karthik Palanisamy f0b22120a0 HBASE-23144 Compact_rs throw wrong number of arguments
Signed-off-by: Reid Chan <reidchan@apache.org>
2019-10-11 15:08:57 +08:00
Karthik Palanisamy ba12d5b9de HBASE-23123 Merge_region fails from shell (#690)
Signed-off-by: Viraj Jasani <virajjasani007@gmail.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-09 10:39:23 -07:00
Karthik Palanisamy d237106ae6 HBASE-23138 Drop_all table by regex fail (#704)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-10-09 11:50:19 +01:00
Karthik Palanisamy bb83e423cc HBASE-23140 Remove unknown table error (#706)
Signed-off-by: huzheng <openinx@gmail.com>
2019-10-09 16:00:56 +08:00
Karthik Palanisamy e911bb7e21 HBASE-23134 Enable_all and Disable_all table by Regex fail from Shell (#698)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2019-10-08 18:16:47 +08:00
Viraj Jasani 944108c38a HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command (#611)
Signed-off-by: huzheng <openinx@gmail.com>
2019-09-30 17:35:22 +08:00
Peter Somogyi 08b82c5c8c
HBASE-23046 Remove compatibility case from truncate command (#638)
Signed-off-by: Sean Busbey <busbey@apache.org>
2019-09-19 09:52:39 +02:00
Viraj Jasani 1dcc8ee50c HBASE-22760 : Pause/Resume/Query Snapshot Auto Cleanup Activity (#617) 2019-09-13 10:34:01 +05:30
Duo Zhang 124ac4d6c4 Revert "HBASE-22676 Move all the code in hbase-rsgroup to hbase-server and remove hbase-rsgroup module (#399)"
This reverts commit f7bbdde975.
2019-09-10 11:14:23 +08:00
Duo Zhang f7bbdde975 HBASE-22676 Move all the code in hbase-rsgroup to hbase-server and remove hbase-rsgroup module (#399)
Signed-off-by: Zheng Hu <openinx@gmail.com>
2019-09-10 10:47:44 +08:00
meiyi 66ad42c742
HBASE-22879 user_permission command failed to show global permission (#511) 2019-08-21 09:29:31 +08:00
Duo Zhang 46abaef4be HBASE-22875 TestShell and TestAdminShell2 are broken 2019-08-19 11:22:07 -05:00
Viraj Jasani f02f741f99 HBASE-22783 shell with removal of status and load classes
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-06 19:20:22 +02:00
johnhomsea a6f065396a HBASE-22779 fix TestTableShell is broken
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
2019-08-02 12:46:22 +02:00
Viraj Jasani 8cfc46d8d0
HBASE-22735 list_regions show basic info for region currently in transition with error handling
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
2019-07-31 18:48:40 -07:00
Guanghao Zhang 35acf3cb78
HBASE-22737 Add a new admin method and shell cmd to trigger the hbck chore to run (#425)
Signed-off-by: stack <stack@apache.org>
2019-08-01 08:54:47 +08:00
stack 0c80d5b42b HBASE-22723 Have CatalogJanitor report holes and overlaps; i.e. problems it sees when doing its regular scan of hbase:meta
Refactor of CatalogJanitor so it generates a
Report on the state of hbase:meta when it runs. Also
refactor so CJ runs even if RIT (previous it would
punt on running if RIT) so it can generate a 'Report'
on the interval regardless. If RIT, it just doesn't
go on to do the merge/split GC as it used to.

If report finds an issue, dump as a WARN message
to the master log.

Follow-on is to make the Report actionable/available
for the Master to pull when it goes to draw the hbck
UI page (could also consider shipping the Report as
part of ClusterMetrics?)

Adds new, fatter Visitor to CJ, one that generates
Report on each run keeping around more findings as
it runs.

Moved some methods around so class reads better;
previous methods were randomly ordered in the class.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
 Make a few handy methods public.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionInfo.java
 Add utility as defaults on the Inteface; i.e. is this the first region
 in table, is it last, does a passed region come next, or does passed
 region overlap this region (added tests for this new stuff).

M hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
 Bugfix... handle case where buffer passed is null.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
 Lots of change, reorg., but mostly adding consistency checking
 to the visitor used scanning hbase:meta on a period and the
 generation of a Report on what the scan has found traversing
 hbase:meta. Added a main so could try the CatalogJanitor against
 a running cluster.

A hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitorCluster.java
 Fat ugly test for CatalogJanitor consistency checking.

M hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java
 Add tests for new functionality in RI.

M hbase-shell/src/main/ruby/hbase/table.rb
 Bug fix for case where meta has a null regioninfo; scan was aborting.

Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
2019-07-29 16:00:57 -07:00
Jan Hentschel 2edbf0b85a HBASE-22692 Rubocop definition is not used in the /bin directory
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
2019-07-24 11:16:06 +02:00
Viraj Jasani 9615c644f5 HBASE-22648 Snapshot TTL (#371)
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2019-07-22 15:03:44 -07:00
Duo Zhang ae7ffccfb8 HBASE-22661 list_regions command in hbase shell is broken
Signed-off-by: Guanghao Zhang <zghao@apache.org>
2019-07-16 09:58:29 +08:00
zhangduo 47cb6295fa HBASE-22281 Fix failed shell UTs 2019-06-18 08:43:35 +08:00
Jan Hentschel 47a96444da
HBASE-22275 Removed deprecated getRegionInfo in HRegionLocation
Signed-off-by: stack <stack@apache.org>
2019-06-16 16:13:46 +02:00
Andrew Purtell a30b186568
HBASE-22449 https everywhere in Maven metadata (#247) 2019-05-21 12:34:25 -07:00
huzheng e254f5e67f HBASE-22425 Balance shell command broken in HBase-3.0.0 2019-05-17 09:54:57 +08:00
Andrew Purtell a9de9f5f28
HBASE-22377 Provide API to check the existence of a namespace which does not require ADMIN permissions (#225)
Signed-off-by: Xu Cang <xucang@apache.org>
2019-05-14 11:56:16 -07:00