Replace BufferChain#write(channel,int) with a simpler #write(channel)
implementation that does not attempt to "chunk" data to be written. This
method was used exclusively by SimpleRpcServer. The code was unnecessarily
complex and caused short writes when values were large, so was corrected
and simplified. Any difference in performance from this change will be
limited to SimpleRpcServer. Testing under load confirms the fix and does
not show significant regression.
SimpleRpcServer and its related code is now also marked as @Deprecated.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcServer.java
* Take advantage of the fact that OpenTelemetry can read its configuration from environment
variables and make use of this where possible, only falling back to passing properties into the
process launch configuration when it's necessary. DRY up tracing configuration and make it
easier to manage in a container environment.
* Replace `HBASE_TRACE_OPTS`, which used to act as both a feature flag and a baseline for
configuration shared across processes. Instead, use `HBASE_OTEL_TRACING_ENABLED` as a feature
flag, and let configuration reuse be handled via the environment variables that otel supports
naively.
* Add further explanation for how to write your configuration for our different deployment
modes (standalone, pseudo-distributed, fully distributed) and in different environments.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
* Fix multiple possible buffer leaks
Motivation:
When using ByteBuf you need to be very careful about releasing it as otherwise you might leak data. There were various places in the code-base where such a leak could happen.
Modifications:
- Fix possible buffer leaks
- Ensure we call touch(...) so its easier to debug buffer leaks
Result:
Fix buffer leaks
* Formatting
* Revert some changes as requested
* revert touch
* Also release checksum and header buffers
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit 2197b3806b)
- the agent jar dropped the `-all` classifier after 1.8.0
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Co-authored-by: Ishika Soni <isoni@isoni-ltmag9r.internal.salesforce.com>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit acfbc3ff7d)
A larger IO buffer for absorbing WALCodec writes can improve the compression
ratio of larger values, because the compressor will be given a larger internal
buffer over which there will be more match opportunities. Does not impact the
ability to read existing written files.
Also, reset the BAOS internal buffer on the way out of compress() so potential
large-ish buffers do not linger on the heap longer than necessary.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Support site configuration of the bytebuf allocator that Netty will use for
NettyRpcServer channels. Property name is 'hbase.netty.rpcserver.allocator'.
Default is no value, which is equivalent to "pooled". Valid values are:
- "pooled": use PooledByteBufAllocator
- "unpooled": use UnpooledByteBufAllocator
- "heap": use HeapByteBufAllocator, which is a PooledByteBufAllocator that
preferentially allocates buffers on heap wherever possible
- <class>: If the value is none of the recognized labels, treat it as a class
name implementing org.apache.hbase.thirdparty.io.netty.buffer.ByteBufAllocator.
This allows the user to add a custom implementation, perhaps for debugging.
Also updates ReflectionUtils with a new helper method.
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcServer.java
hbase-server/src/test/java/org/apache/hadoop/hbase/ipc/TestNettyRpcServer.java
Co-authored-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Yu Li <liyu@apache.org>
(cherry picked from commit 7fc1674933)
Conflicts:
hbase-testing-util/src/main/java/org/apache/hadoop/hbase/testing/TestingHBaseClusterImpl.java
Created OnheapDecodedCell and OffheapDecodedExtendedCell objects with duplicate copy of
ByteBuffer's underlying array instead of original ByteBuffer
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Pankaj Kumar<pankajkumar@apache.org>
This reverts commit f3a48d1910.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Conflicts:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
Andrew Purtell
Bryan Beaudreault
Duo Zhang
Nick Dimiduk
chenglei
Norman Maurer
Ujjawal
wenwj0
tianhang
Ishika
Kerasone
Pankaj
litao
Tak Lon (Stephen) Wu
LiangJun He
XinSun
xiaozhang0319
SiCheng-Zheng
Xiaolin Ha
xicm
Mohammad Arshad
Viraj Jasani