Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HBASE-27805 The chunk created by mslab may cause memory fragement and lead to fullgc

This commit is contained in:
bsglz 2023-04-20 20:48:10 +08:00
parent 398c5ef313
commit 838c7d9170
3 changed files with 9 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
hbase-common/src/main/resources/hbase-default.xml
View File

@ -755,8 +755,12 @@ possible configurations would overwhelm and obscure the important.
</property>
<property>
<name>hbase.hregion.memstore.mslab.chunksize</name>
<value>2097152</value>
<description>The maximum byte size of a chunk in the MemStoreLAB. Unit: bytes</description>
<value>2096128</value>
<description>
The maximum byte size of a chunk in the MemStoreLAB.
Use 2047k as the default because 2m may cause memory fragmentation in some cases.
Unit: bytes
</description>
</property>
<property>
<name>hbase.regionserver.offheap.global.memstore.size</name>

2
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreLAB.java
View File

@ -51,7 +51,7 @@ public interface MemStoreLAB {
String MSLAB_CLASS_NAME = "hbase.regionserver.mslab.class";
String CHUNK_SIZE_KEY = "hbase.hregion.memstore.mslab.chunksize";
int CHUNK_SIZE_DEFAULT = 2048 * 1024;
int CHUNK_SIZE_DEFAULT = 2047 * 1024;
String INDEX_CHUNK_SIZE_PERCENTAGE_KEY = "hbase.hregion.memstore.mslab.indexchunksize.percent";
float INDEX_CHUNK_SIZE_PERCENTAGE_DEFAULT = 0.1f;
String MAX_ALLOC_KEY = "hbase.hregion.memstore.mslab.max.allocation";

3
src/main/asciidoc/_chapters/offheap_read_write.adoc
View File

@ -213,7 +213,8 @@ of off-heap memory that should be used by MSLAB (e.g. `25` would result in 25MB
MaxDirectMemorySize property (see <<hbase.offheapsize>> for more on _HBASE_OFFHEAPSIZE_). The default value of
`hbase.regionserver.offheap.global.memstore.size` is 0 which means MSLAB uses onheap, not offheap, chunks by default.
`hbase.hregion.memstore.mslab.chunksize` controls the size of each off-heap chunk. Default is `2097152` (2MB).
`hbase.hregion.memstore.mslab.chunksize` controls the size of each off-heap chunk. Default is `2096128` (that's 2047KB, why not 2MB?
see link:https://issues.apache.org/jira/browse/HBASE-27805[HBASE-27805] for more detail).
When a Cell is added to a MemStore, the bytes for that Cell are copied into these off-heap buffers (if `hbase.regionserver.offheap.global.memstore.size` is non-zero)
and a Cell POJO will refer to this memory area. This can greatly reduce the on-heap occupancy of the MemStores and reduce the total heap utilization for RegionServers