LUCENE-2387: don't hang onto Fieldable of last doc indexed in IW, nor to Reader in Tokenizer after close is called

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@932398 13f79535-47bb-0310-9956-ffa450edef68

lucene/CHANGES.txt

@@ -258,6 +258,10 @@ Bug fixes
 * LUCENE-2074: Reduce buffer size of lexer back to default on reset.
   (Ruben Laguna, Shai Erera via Uwe Schindler)
   
+* LUCENE-2387: Don't hang onto Fieldables from the last doc indexed,
+  in IndexWriter, nor the Reader in Tokenizer after close is
+  called.  (Ruben Laguna, Uwe Schindler, Mike McCandless)
+
 New features
 
 * LUCENE-2128: Parallelized fetching document frequencies during weight

lucene/src/java/org/apache/lucene/analysis/Tokenizer.java

@@ -67,7 +67,12 @@ public abstract class Tokenizer extends TokenStream {
   /** By default, closes the input Reader. */
   @Override
   public void close() throws IOException {
-    input.close();
+    if (input != null) {
+      input.close();
+      // LUCENE-2387: don't hold onto Reader after close, so
+      // GC can reclaim
+      input = null;
+    }
   }
   
   /** Return the corrected offset. If {@link #input} is a {@link CharStream} subclass

lucene/src/java/org/apache/lucene/index/DocInverterPerField.java

@@ -199,6 +199,10 @@ final class DocInverterPerField extends DocFieldConsumerPerField {
           fieldState.offset += docState.analyzer.getOffsetGap(field);
         fieldState.boost *= field.getBoost();
       }
+
+      // LUCENE-2387: don't hang onto the field, so GC can
+      // reclaim
+      fields[i] = null;
     }
 
     consumer.finish();