Fix test case failures in org.apache.hadoop.hbase.http.log.TestLogLevel under Openjdk 17 because of a missing export of java.security.jgss/sun.security.krb5.
Removed option --illegal-access=permit ignored since Openjdk 17.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Revert "HBASE-24579: Failed SASL authentication does not result in an exception on client side (#1921)"
This reverts commit bd79c4065c.
When Kerberos authentication succeeds, on the server side, after
receiving the final SASL token from the client, we simply wait for
the client to continue by sending the connection header.
After HBASE-24579, on the client side, an additional readStatus()
was added, which mistakenly assumes that after negotiation has
completed a status code will be sent. However when authentication
has succeeded the server will not send one. As a result the client
will hang and only throw an exception when the configured read
timeout is reached, which is 20 seconds by default.
We cannot unilaterally send the expected additional status code
from the server side because older clients will not expect it. The
first call will fail because the client finds unexpected bytes in
the stream ahead of the call response. Fabricating a call response
also does not seem a viable strategy for backwards compatibility.
The HBASE-24579 change needs to be reconsidered given the
difficult backwards compatibility challenges here.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
The configuration settings "hbase.netty.worker.count" and
"hbase.netty.eventloop.rpcserver.thread.count" appear to duplicate each
other.
Also, formalizes another setting found in NettyEventLoopGroupConfig,
"hbase.netty.nativetransport".
Also, native epoll is not limited to amd64. aarch64 supports it too.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
When backporting to branch-2 it was discovered that the order of tear
down operations requires the KDC to be torn down first, although on
master branch the original code did not produce the same errors (a NPE
in KDC#stop). Forward port the difference for maintainability.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Add test coverage for SimpleRpcServer.
Improve the way we test both SimpleRpcServer and NettyRpcServer. Use
LoadTestKVGenerator to generate random values with varying sizes between
1000 bytes and 1M bytes, and also to verify them when reading the values
back.
Add secure test coverage for both SimpleRpcServer and NettyRpcServer.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
* HBASE-27088 IntegrationLoadTestCommonCrawl async load improvements
- Use an async client and work stealing executor for parallelism during loads.
- Remove the verification read retries, these are not that effective during
replication lag anyway.
- Increase max task attempts because S3 might throttle.
- Implement a side task that exercises Increments by extracting urls from
content and updating a cf that tracks referrer counts. These are not
validated at this time. It could be possible to log the increments, sum
them with a reducer, and then verify the total, but this is left as a
future exercise.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
* Sum RPC time for writes (loader) and reads (verifier) and mutation bytes submitted. Expose as job counters.
* Fix an issue with completion chaining
* Pause loading if too many operations are in flight
TestMultiRespectLimits submits a put with an empty value. This appears
to not be what was intended. Instead, use a single byte.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Replace BufferChain#write(channel,int) with a simpler #write(channel)
implementation that does not attempt to "chunk" data to be written. This
method was used exclusively by SimpleRpcServer. The code was unnecessarily
complex and caused short writes when values were large, so was corrected
and simplified. Any difference in performance from this change will be
limited to SimpleRpcServer. Testing under load confirms the fix and does
not show significant regression.
SimpleRpcServer and its related code is now also marked as @Deprecated.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
* Take advantage of the fact that OpenTelemetry can read its configuration from environment
variables and make use of this where possible, only falling back to passing properties into the
process launch configuration when it's necessary. DRY up tracing configuration and make it
easier to manage in a container environment.
* Replace `HBASE_TRACE_OPTS`, which used to act as both a feature flag and a baseline for
configuration shared across processes. Instead, use `HBASE_OTEL_TRACING_ENABLED` as a feature
flag, and let configuration reuse be handled via the environment variables that otel supports
naively.
* Add further explanation for how to write your configuration for our different deployment
modes (standalone, pseudo-distributed, fully distributed) and in different environments.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
* Fix multiple possible buffer leaks
Motivation:
When using ByteBuf you need to be very careful about releasing it as otherwise you might leak data. There were various places in the code-base where such a leak could happen.
Modifications:
- Fix possible buffer leaks
- Ensure we call touch(...) so its easier to debug buffer leaks
Result:
Fix buffer leaks
* Formatting
* Revert some changes as requested
* revert touch
* Also release checksum and header buffers
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Bryan Beaudreault
Duo Zhang
chenglei
Andrew Purtell
Ramón García Fernández
Wellington Ramos Chevreuil
Xiaolin Ha
Minwoo Kang
SiCheng-Zheng
Jinhu Wu
Nick Dimiduk
Norman Maurer
Narasimha Sharma
Ujjawal