Commit Graph

17093 Commits

Author SHA1 Message Date
Peter Somogyi ac5ac1dcb3 HBASE-23347 Allow custom authentication methods for RPCs; addendum (#1060)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-21 18:48:04 +01:00
Jan Hentschel c16569996b
HBASE-23625 Reduced number of Checkstyle violations in hbase-common
Signed-off-by: stack <stack@apache.org>
2020-01-20 22:24:01 +01:00
Guanghao Zhang 8f292a8d58 HBASE-23156 start-hbase.sh failed with ClassNotFoundException when build with hadoop3 (#1067)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-01-20 13:24:27 +08:00
Jan Hentschel f6a2238889
HBASE-23661 Reduced number of Checkstyle violations in hbase-rest
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-19 18:17:14 +01:00
Duo Zhang 70c69ba765 HBASE-23680 RegionProcedureStore missing cleaning of hfile archive (#1022)
Signed-off-by: stack <stack@apache.org>
2020-01-18 20:52:29 +08:00
Peter Somogyi d9b5b9f3fc HBASE-23653 Expose content of meta table in web ui; addendum (#1061)
Fix error prone problem

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-01-18 09:34:12 +01:00
Nick Dimiduk 2bc67dfda0
HBASE-23700 Upgrade checkstyle and plugin versions (#1056) (#1063)
Bump checkstyle version to 8.28, maven-checkstyle-plugin to 3.1.0.

As per HBASE-23242 and the updated checkstyle docs[1], the LineLength
check should be placed under an instance of Checker.

[1] https://checkstyle.sourceforge.io/config_sizes.html#LineLength

Co-authored-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-17 17:09:16 -08:00
Josh Elser 1ea389a834 HBASE-23701 Try to converge automated checks around Category
We have a compile-time guarantee to either have a category array
of length zero or one because the category annotation is non-repeatable.

Also tries to clean up the checking for unit-test annotations.

Closes #1057

Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-01-17 12:49:50 -05:00
Viraj Jasani 193a9c2e55
HBASE-23569 : Validate that all default chores of HRegionServer are scheduled (ADDENDUM)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-01-16 23:07:51 -08:00
Nick Dimiduk 74bfe023e5
HBASE-23653 Expose content of meta table in web ui (#1021)
Adds a display of the content of 'hbase:meta' to the Master's
table.jsp, when that table is selected. Supports basic pagination,
filtering, &c.

Signed-off-by: stack <stack@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-01-16 08:47:00 -08:00
Josh Elser bef1eb33f4 HBASE-23347 Allow custom authentication methods for RPCs
Decouple the HBase internals such that someone can implement
their own SASL-based authentication mechanism and plug it into
HBase RegionServers/Masters.

Comes with a design doc in dev-support/design-docs and an example in
hbase-examples known as "Shade" which uses a flat-password file
for authenticating users.

Closes #884

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
2020-01-16 11:04:08 -05:00
Josh Elser 0935ba4e16 HBASE-23695 Fail gracefully if no category is present
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

Closes #1052
2020-01-16 10:24:53 -05:00
Duo Zhang 68d3c201dd HBASE-23652 Move the unsupported procedure type check before migrating to RegionProcedureStore (#1018)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-01-16 23:01:58 +08:00
WenFeiYi 9f9fa0d412 HBASE-23677 fix 500/NPE of region.jsp (#1033)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit d60ce17c17)
2020-01-16 14:00:32 +00:00
Wellington Chevreuil bd4eba2b53 Revert "fix 500/NPE of region.jsp (#1033)"
This reverts commit ae95b1f215.
(reverting as commit message lacks jira id. Will rea-apply soon with the jira id)
2020-01-16 13:59:05 +00:00
Wellington Ramos Chevreuil 8b5f467716 HBASE-23694 After RegionProcedureStore completes migration of WALProcedureStore, still running WALProcedureStore.syncThread keeps trying to delete now inexistent log files. (#1048)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-01-16 14:13:53 +08:00
Bharath Vissapragada 31b4fbdee5 HBASE-23665: Split unit tests from TestTableName into a separate test-only class. (#1032)
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
2020-01-15 13:56:51 -08:00
dingbaosheng 9a708a3fd9 HBASE-23674 Too many rit page Numbers show confusion 2020-01-15 13:53:50 -08:00
BukrosSzabolcs 9621bafab6 HBASE-23601 OutputSink.WriterThread exception gets stuck and repeated indefinietly (#1028) 2020-01-15 12:59:58 -08:00
WenFeiYi ae95b1f215 fix 500/NPE of region.jsp (#1033)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
(cherry picked from commit d60ce17c17)
2020-01-15 11:38:16 +00:00
Viraj Jasani 3800f6f131
HBASE-23569 : Validate that all default chores of HMaster are scheduled
Signed-off-by: Andrew Purtell <apurtell@apache.org>
2020-01-14 23:07:17 -08:00
Michael Stack 602f6dd693
HBASE-23687 DEBUG logging cleanup (#1040)
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-01-14 22:07:23 -08:00
Nick Dimiduk 4a35e2f837
HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing (#917) (#1037)
The current process for an operator, after fixing holes in meta, is to
manually disable and enable the whole table. Let's try to avoid
bringing the whole table offline if we can. Have the master attempt to
queue up assignment procedures for any new regions it creates.

Signed-off-by: stack <stack@apache.org>
2020-01-14 09:00:29 -08:00
Josh Elser 4bf7fb8613 HBASE-23679 FileSystem objects leak when cleaned up in cleanupBulkLoad
The cleanupBulkLoad method is only called for the first Region in the
table which was being bulk loaded into. This means that potentially N-1
other RegionServers (where N is the number of RegionServers) will leak
one FileSystem object into the FileSystem cache which will never be
cleaned up. We need to do this clean-up as a part of secureBulkLoadHFiles
otherwise we cannot guarantee that heap usage won't grow unbounded.

Closes #1029

Signed-off-by: Sean Busbey <busbey@apache.org>
2020-01-13 18:42:31 -05:00
Duo Zhang caaaf0810e HBASE-23681 Add UT for procedure store region flusher (#1024)
Signed-off-by: stack <stack@apache.org>
2020-01-12 13:31:11 +08:00
Jan Hentschel 954b602a59
HBASE-23646 Resolved remaining Checkstyle violations in tests of hbase-rest
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-11 21:12:14 +01:00
stack da782e4233 Revert "HBASE-23055 Alter hbase:meta"
This reverts commit d64b0e3612.
2020-01-11 09:21:15 -08:00
Toshihiro Suzuki 1ad28a6f40 HBASE-23165 [hbtop] Some modifications from HBASE-22988 (#987)
Signed-off-by: stack <stack@apache.org>
2020-01-11 21:22:14 +09:00
Peter Somogyi c9fa98a4d6 HBASE-23675 Move to Apache parent POM version 22 2020-01-11 11:19:24 +01:00
stack 6bd6396c97 Revert "HBASE-23668 Master log start filling with "Flush journal status" messages"
Minor addendum fixing log message.
2020-01-10 15:57:46 -08:00
belugabehr 76a6fe6a8b HBASE-23628: Remove Apache Commons Digest Base64 (#977)
Signed-off-by: stack <stack@apache.org>
(cherry picked from commit 280b94434c)
2020-01-10 15:45:59 -06:00
stack 938dd1c2f8 HBASE-23668 Master log start filling with "Flush journal status" messages"
This reverts commit fb9fa04da7.
i.e. reapplication of patch that was preamaturely applied.

Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-01-10 10:27:33 -08:00
Nick Dimiduk fdc9dd63c5
HBASE-23664 Upgrade JUnit to 4.13 (#1009)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-10 07:21:17 -08:00
Guanghao Zhang 75ca8606df HBASE-23659 BaseLoadBalancer#wouldLowerAvailability should consider region replicas (#1001)
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-01-10 15:23:13 +08:00
stack 1217d57492 Revert "HBASE-23668 Master log start filling with "Flush journal status" messages"
This reverts commit 1047246717.
Prematurely pushed
2020-01-09 15:49:29 -08:00
stack 58b0e0f3fc Revert "HBASE-23601: OutputSink.WriterThread exception gets stuck and repeated indefinietly (#956)"
This reverts commit e78ce468d8.
2020-01-09 15:23:36 -08:00
stack 1047246717 HBASE-23668 Master log start filling with "Flush journal status" messages 2020-01-09 14:08:19 -08:00
BukrosSzabolcs e78ce468d8 HBASE-23601: OutputSink.WriterThread exception gets stuck and repeated indefinietly (#956)
* HBASE-23601: OutputSink.WriterThread exception gets stuck and repeated indefinietly

clear exception after logged
try to restart writer threads if needed
2020-01-09 12:22:16 -08:00
stack d64b0e3612 HBASE-23055 Alter hbase:meta
Make it so hbase:meta can be altered. TableState for hbase:meta
    was hardcoded ENABLED. Make it dynamic. State is now kept in
    current active Master. It is transient so falls back to default
    if Master crashes. Add to registry a getMetaTableState
    which reads mirrored state from zookeeper (NOT from Master and
    defaults ENABLED if no implementation or error fetching state).
    hbase:meta schema will be bootstrapped from the filesystem.
    Changes to filesystem schema are atomic so we should be ok if
    Master fails mid-edit (TBD). Undoes a bunch of guards that
    prevented our being able to edit hbase:meta.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     Add fetching of hbase:meta table state from registry. Adds cache of
     tablestates w/ a ttl of 1 second (adjustable).

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
      Add querying registry for hbase:meta table state.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
      Add querying of mirrored table state for hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta. Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.

Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
2020-01-09 09:52:21 -08:00
Beata Sudi d7e7d028bd
HBASE-23654 Adding Apache Trafodion and EsgynDB to 'Powered by Apache HBase' page
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-01-09 15:37:09 +01:00
Guanghao Zhang bd5b0ba55d HBASE-23655 Fix flaky TestRSGroupsKillRS: should wait the SCP to finish (#996)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-01-09 18:39:20 +08:00
Guanghao Zhang e750d2c7af HBASE-23658 Fix flaky TestSnapshotFromMaster (#998)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2020-01-09 17:44:53 +08:00
Peter Somogyi 49ad127fd6 HBASE-23663 Allow dot and hyphen in Profiler's URL (#1002)
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-09 10:32:38 +01:00
binlijin 1591c2025e
Backport "HBASE-23660 hbase:meta's table.jsp ref to wrong rs address" to branch-2 (#1005) 2020-01-09 17:22:34 +08:00
binlijin e242d84b97 HBASE-23651 Region balance throttling can be disabled (#991)
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-01-08 19:03:30 +08:00
Jan Hentschel 4f7de9e27e
HBASE-23645 Fixed remaining Checkstyle violations in hbase-common tests
Signed-off-by: Viraj Jasani <vjasani@apache.org>
2020-01-07 09:07:56 +01:00
binlijin 1415a82d41 HBASE-23636 Disable table may hang when regionserver stop or abort. (#982)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: virajjasani <34790606+virajjasani@users.noreply.github.com>
2020-01-06 15:22:58 +08:00
Viraj Jasani 27660549d1
HBASE-23588 : Cache index & bloom blocks on write (ADDENDUM)
Signed-off-by: Anoop Sam John <anoopsamjohn@apache.org>
2020-01-06 12:07:05 +05:30
Peter Somogyi 3f646d8a81
HBASE-23642 Reintroduce ReplicationUtils.contains as deprecated (#983)
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
2020-01-04 12:14:19 +01:00
Jan Hentschel 085450ba4e
HBASE-23635 Reduced number of Checkstyle violations in hbase-mapreduce
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: stack <stack@apache.org>
2020-01-04 01:09:16 +01:00