This is a demonstration of visualization of regions on the cluster. The visualization is a stacked
bar chart showing total storefile size per table per region server, with the x-axis being server
names, the y-axis being storfile size, and the bars stacked per table. The visualization is
generated entirely on the fly from within the browser, implemented using Vega Lite. So far, Vega
appears to handle rendering this visualization for a cluster of over 700 region servers with
approximately 300,000 regions.
Per [0], include an update to the top-level LICENSE.txt. Also update LICENSE files in all binary
distributions (i.e., jars), by way of LICENSE.vm. Vega uses a BSD 3-clause variant without
advertising clause, and as such is a "Category A" license, per [1].
No changes are made to the NOTICE files, as per the existing example of bundling the minified
JQuery, which is also a Category A license.
[0]: https://infra.apache.org/licensing-howto.html
[1]: https://www.apache.org/legal/resolved.html#category-a
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Publishes a set of JSON endpoints following a RESTful structure, which expose a subset of the
`o.a.h.h.ClusterMetrics` object tree. The URI structure is as follows
/api/v1/admin/cluster_metrics
/api/v1/admin/cluster_metrics/live_servers
/api/v1/admin/cluster_metrics/dead_servers
Signed-off-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Avoid the pattern where a Random object is allocated, used once or twice, and
then left for GC. This pattern triggers warnings from some static analysis tools
because this pattern leads to poor effective randomness. In a few cases we were
legitimately suffering from this issue; in others a change is still good to
reduce noise in analysis results.
Use ThreadLocalRandom where there is no requirement to set the seed to gain
good reuse.
Where useful relax use of SecureRandom to simply Random or ThreadLocalRandom,
which are unlikely to block if the system entropy pool is low, if we don't need
crypographically strong randomness for the use case. The exception to this is
normalization of use of Bytes#random to fill byte arrays with randomness.
Because Bytes#random may be used to generate key material it must be backed by
SecureRandom.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
When starting a jetty http server, one can explicitly exclude certain (unsecure)
SSL cipher suites. This can be especially important, when the HBase cluster
needs to be compliant with security regulations (e.g. FIPS).
Currently it is possible to set the excluded ciphers for the ThriftServer
("hbase.thrift.ssl.exclude.cipher.suites") or for the RestServer
("hbase.rest.ssl.exclude.cipher.suites"), but one can not configure it for the
regular InfoServer started by e.g. the master or region servers.
In this commit I want to introduce a new configuration
"ssl.server.exclude.cipher.list" to configure the excluded cipher suites for the
http server started by the InfoServer. This parameter has the same name and will
work in the same way, as it was already implemented in hadoop (e.g. for hdfs/yarn).
See: HADOOP-12668, HADOOP-14341
Co-authored-by: Mate Szalay-Beko <symat@apache.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
HBASE-25267 Make SSL keystore type configurable in HBase RESTServer
In this patch I want to introduce the hbase.rest.ssl.keystore.type parameter,
enabling us to customize the keystore type for the REST server. If the
parameter is not provided, then we should fall-back to the current behaviour
(which assumes keystore type JKS).
This is similar to how we already configure the InfoServer objects with the
ssl.server.keystore.type parameter to set up HTTPS for the various admin UIs.
Signed-off-by: Wellington Chevreuil <wellington.chevreuil@gmail.com>
Signed-off-by: Balazs Meszaros <meszibalu@apache.org>
Signed-off-by: Sean Busbey <busbey@apache.org>
Move the random free port generate back into hbasecommontestingutility
from hbasetestingutility.
Add a create simple kdc server utility that will start a kdc server and
if a bindexception, create a new one on a new random port in hbase-common.
Add new BoundSocketMaker helpful when trying to manufacture
BindExceptions because of port clash.
Change thrift and http kdc tests to use this new utility (removes
code duplication around kdc server setup).
use the findPort logic even if the BindException is wrapped in an IOException
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <jan.hentschel@ultratendency.com>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
This is causing me issues with parallel test runs.
Also allow setting the surefire reports and temp directories via command line.
Signed-off-by: stack <stack@apache.org>
These classifications come of running at various fork counts.. A test
may complete quick if low fork count but if it is accessing disk, it
will run much slower if fork count is high. This edit accommodates
some of this phenomenon.
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Jan Hentschel <janh@apache.org>
The Hadoop AccessControlList allows us to specify admins of the webUI
via a list of users and/or groups. Admins of the WebUI can mutate the
system, potentially seeing sensitive data or modifying the system.
hbase.security.authentication.spnego.admin.users is a comma-separated
list of users who are admins.
hbase.security.authentication.spnego.admin.groups is a comma-separated
list of groups whose membership are admins. Either of these
configuration properties may also contain an asterisk (*) which denotes
"any entity" (e.g user, group).
Previously, when a user was denied from some endpoint that was
designated for admins, they received an HTTP/401. In this case, it is
more correct to return HTTP/403 as they were correctly authenticated,
but they were disallowed from fetching the given resource. This commit
incorporates this change.
hbase.security.authentication.ui.config.protected also exists for users
who have sensitive information stored in the Hadoop service
configuration and want to limit access to this endpoint. By default,
the Hadoop configuration endpoint is not protected and any
authenticated user can access it.
The test is based off of work by Nihal Jain in HBASE-20472.
Co-authored-by: Nihal Jain <nihaljain.cs@gmail.com>
Signed-off-by: Sean Busbey <busbey@apache.org>
* Add a bit of javadoc around SerialReplicationChecker.
* Miniscule edit to the profiler jsp page and then a bit of doc on how to make it work that might help.
* Add some detail if NPE getting BitSetNode to help w/ debug.
* Change HbckChore to log region names instead of encoded names; helps doing diagnostics; can take region name and query in shell to find out all about the region according to hbase:meta.
* Add some fix-it help inline in the HBCK Report page – how to fix.
* Add counts in procedures page so can see if making progress; move listing of WALs to end of the page.
Nick Dimiduk
Duo Zhang
Andrew Purtell
Bri Augenreich
Andor Molnár
Viraj Jasani
Ruanhui
Bryan Beaudreault
Mate Szalay-Beko
Michael Stack
stack
Richard Antal
申胜利
ramkrish86
Esteban Gutierrez
YeChao Chen
David Manning
shahrs87
Josh Elser
Istvan Toth
Mark Robert Miller
Peter Somogyi
康智冬