Apache
/
lucene
mirror of https://github.com/apache/lucene.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Performance improvement for SegmentMerger.mergeNorms: LUCENE-739 

git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@485469 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Yonik Seeley 2006-12-11 02:38:29 +00:00
parent 0d109d1485
commit 10460b5e87
2 changed files with 21 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
CHANGES.txt
View File

@ -305,6 +305,11 @@ Optimizations
TermDocs for a MultiReader. The old implementation could
recurse up to the number of segments in the index. (Yonik Seeley)
14. LUCENE-739: Improve segment merging performance by reusing
the norm array across different fields and doing bulk writes
of norms of segments with no deleted docs.
(Michael Busch via Yonik Seeley)
Test Cases
1. Added TestTermScorer.java (Grant Ingersoll)
2. Added TestWindowsMMap.java (Benson Margulies via Mike McCandless)

21
src/java/org/apache/lucene/index/SegmentMerger.java
View File

@ -407,6 +407,7 @@ final class SegmentMerger {
}
private void mergeNorms() throws IOException {
byte[] normBuffer = null;
for (int i = 0; i < fieldInfos.size(); i++) {
FieldInfo fi = fieldInfos.fieldInfo(i);
if (fi.isIndexed && !fi.omitNorms) {
@ -415,11 +416,21 @@ final class SegmentMerger {
for (int j = 0; j < readers.size(); j++) {
IndexReader reader = (IndexReader) readers.elementAt(j);
int maxDoc = reader.maxDoc();
byte[] input = new byte[maxDoc];
reader.norms(fi.name, input, 0);
for (int k = 0; k < maxDoc; k++) {
if (!reader.isDeleted(k)) {
output.writeByte(input[k]);
if (normBuffer == null || normBuffer.length < maxDoc) {
// the buffer is too small for the current segment
normBuffer = new byte[maxDoc];
}
reader.norms(fi.name, normBuffer, 0);
if (!reader.hasDeletions()) {
//optimized case for segments without deleted docs
output.writeBytes(normBuffer, maxDoc);
} else {
// this segment has deleted docs, so we have to
// check for every doc if it is deleted or not
for (int k = 0; k < maxDoc; k++) {
if (!reader.isDeleted(k)) {
output.writeByte(normBuffer[k]);
}
}
}
}