Apache
/
lucene
mirror of https://github.com/apache/lucene.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

LUCENE-3222: don't under count RAM required for deleted terms 

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1138069 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Michael McCandless 2011-06-21 16:07:44 +00:00
parent a97c53e7fd
commit a2f08bfb9b
2 changed files with 17 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
lucene/CHANGES.txt
View File

@ -544,6 +544,10 @@ Bug fixes
background optimize when documents are still being deleted
concurrently with the optimize (Mike McCandless)
* LUCENE-3222: The RAM accounting for buffered delete terms was
failing to measure the space required to hold the term's field and
text character data. (Mike McCandless)
API Changes
* LUCENE-3208: Renamed protected IndexSearcher.createWeight() to expert

20
lucene/src/java/org/apache/lucene/index/FrozenBufferedDeletes.java
View File

@ -32,12 +32,12 @@ import org.apache.lucene.index.BufferedDeletesStream.QueryAndLimit;
class FrozenBufferedDeletes {
/* Rough logic: Term is object w/
String field and String text (OBJ_HEADER + 2*POINTER).
Term's text is String (OBJ_HEADER + 4*INT + POINTER +
OBJ_HEADER + text.length*CHAR).
Term's field is String (OBJ_HEADER + 4*INT + POINTER +
OBJ_HEADER + field.length*CHAR). */
final static int BYTES_PER_DEL_TERM = 4*RamUsageEstimator.NUM_BYTES_OBJECT_REF + 4*RamUsageEstimator.NUM_BYTES_OBJECT_HEADER + 8*RamUsageEstimator.NUM_BYTES_INT;
String field and BytesRef text (OBJ_HEADER + 2*POINTER).
String field is (OBJ_HEADER + 4*INT +
POINTER + OBJ_HEADER + CHAR*field.length).
Term's text is BytesRef (OBJ_HEADER + 2*INT + POINTER +
OBJ_HEADER + bytes.length). */
final static int BYTES_PER_DEL_TERM = 4*RamUsageEstimator.NUM_BYTES_OBJECT_REF + 5*RamUsageEstimator.NUM_BYTES_OBJECT_HEADER + 6*RamUsageEstimator.NUM_BYTES_INT;
/* Query we often undercount (say 24 bytes), plus int. */
final static int BYTES_PER_DEL_QUERY = RamUsageEstimator.NUM_BYTES_OBJECT_REF + RamUsageEstimator.NUM_BYTES_INT + 24;
@ -71,7 +71,13 @@ class FrozenBufferedDeletes {
queryLimits[upto] = ent.getValue();
upto++;
}
bytesUsed = terms.length * BYTES_PER_DEL_TERM + queries.length * BYTES_PER_DEL_QUERY;
int termDataBytes = 0;
for(Map.Entry<Term,Integer> ent : deletes.terms.entrySet()) {
final Term term = ent.getKey();
termDataBytes += term.bytes().length;
termDataBytes += term.field().length() * RamUsageEstimator.NUM_BYTES_CHAR;
}
bytesUsed = terms.length * BYTES_PER_DEL_TERM + queries.length * BYTES_PER_DEL_QUERY + termDataBytes;
numTermDeletes = deletes.numTermDeletes.get();
}