Apache
/
lucene
mirror of https://github.com/apache/lucene.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

LUCENE-10112: Improve LZ4 Compression performance with direct primitive read/writes (#310) 

Co-authored-by: Tim Brooks <tim@timbrooks.org>
This commit is contained in:
Uwe Schindler 2021-09-20 19:12:38 +02:00 committed by GitHub
parent 57524c6a5e
commit 5871ea7972
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 8 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
lucene/CHANGES.txt
View File

@ -276,6 +276,9 @@ Improvements
file format in input/output classes like DataInput / DataOutput and codecs. file format in input/output classes like DataInput / DataOutput and codecs.
(Uwe Schindler, Robert Muir) (Uwe Schindler, Robert Muir)
* LUCENE-10112: Improve LZ4 Compression performance with direct primitive read/writes.
(Tim Brooks, Uwe Schindler, Robert Muir, Adrien Grand)
Bug fixes Bug fixes
* LUCENE-10070 Skip deleted docs when accumulating facet counts for all docs. (Ankur Goel, Greg Miller) * LUCENE-10070 Skip deleted docs when accumulating facet counts for all docs. (Ankur Goel, Greg Miller)

9
lucene/core/src/java/org/apache/lucene/util/compress/LZ4.java
View File

@ -31,6 +31,7 @@ import java.util.Arrays;
import java.util.Objects; import java.util.Objects;
import org.apache.lucene.store.DataInput; import org.apache.lucene.store.DataInput;
import org.apache.lucene.store.DataOutput; import org.apache.lucene.store.DataOutput;
import org.apache.lucene.util.BitUtil;
import org.apache.lucene.util.packed.PackedInts; import org.apache.lucene.util.packed.PackedInts;
/** /**
@ -63,10 +64,10 @@ public final class LZ4 {
} }
private static int readInt(byte[] buf, int i) { private static int readInt(byte[] buf, int i) {
return ((buf[i] & 0xFF) << 24) // we hardcode LITTLE ENDIAN here as this is most performant on most platforms.
| ((buf[i + 1] & 0xFF) << 16) // According to LZ4's alogrithm the endianness does not matter at all, but we
| ((buf[i + 2] & 0xFF) << 8) // want to prevent indexes to differ just because of platform endianness!
| (buf[i + 3] & 0xFF); return (int) BitUtil.VH_LE_INT.get(buf, i);
} }
private static int commonBytes(byte[] b, int o1, int o2, int limit) { private static int commonBytes(byte[] b, int o1, int o2, int limit) {