diff --git a/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90TermVectorsFormat.java b/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90TermVectorsFormat.java index 0142f5461e8..e19168ff95d 100644 --- a/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90TermVectorsFormat.java +++ b/lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90TermVectorsFormat.java @@ -56,15 +56,20 @@ import org.apache.lucene.util.packed.PackedInts; *
  • VectorMeta (.tvm) --> <Header>, PackedIntsVersion, ChunkSize, * ChunkIndexMetadata, ChunkCount, DirtyChunkCount, DirtyDocsCount, Footer *
  • Header --> {@link CodecUtil#writeIndexHeader IndexHeader} - *
  • PackedIntsVersion --> {@link PackedInts#VERSION_CURRENT} as a {@link - * DataOutput#writeVInt VInt} - *
  • ChunkSize is the number of bytes of terms to accumulate before flushing, as a {@link - * DataOutput#writeVInt VInt} - *
  • ChunkCount is not known in advance and is the number of chunks necessary to store all - * document of the segment - *
  • DirtyChunkCount --> the number of prematurely flushed chunks in the .tvd file + *
  • PackedIntsVersion, ChunkSize --> {@link DataOutput#writeVInt VInt} + *
  • ChunkCount, DirtyChunkCount, DirtyDocsCount --> {@link DataOutput#writeVLong + * VLong} + *
  • ChunkIndexMetadata --> {@link FieldsIndexWriter} *
  • Footer --> {@link CodecUtil#writeFooter CodecFooter} * + *

    Notes: + *

    *
  • *

    A vector data file (extension .tvd). This file stores terms, frequencies, * positions, offsets and payloads for every document. Upon writing a new segment, it @@ -80,76 +85,78 @@ import org.apache.lucene.util.packed.PackedInts; * FieldNumOffs >, < Flags >, < NumTerms >, < TermLengths >, < * TermFreqs >, < Positions >, < StartOffsets >, < Lengths >, < * PayloadLengths >, < TermAndPayloads > - *

  • DocBase is the ID of the first doc of the chunk as a {@link DataOutput#writeVInt - * VInt} - *
  • ChunkDocs is the number of documents in the chunk *
  • NumFields --> DocNumFieldsChunkDocs - *
  • DocNumFields is the number of fields for each doc, written as a {@link - * DataOutput#writeVInt VInt} if ChunkDocs==1 and as a {@link PackedInts} array - * otherwise - *
  • FieldNums --> FieldNumDeltaTotalDistincFields, a delta-encoded list of - * the sorted unique field numbers present in the chunk - *
  • FieldNumOffs --> FieldNumOffTotalFields, as a {@link PackedInts} array - *
  • FieldNumOff is the offset of the field number in FieldNums - *
  • TotalFields is the total number of fields (sum of the values of NumFields) + *
  • FieldNums --> FieldNumDeltaTotalDistincFields *
  • Flags --> Bit < FieldFlags > - *
  • Bit is a single bit which when true means that fields have the same options for every - * document in the chunk *
  • FieldFlags --> if Bit==1: FlagTotalDistinctFields else * FlagTotalFields + *
  • NumTerms --> FieldNumTermsTotalFields + *
  • TermLengths --> PrefixLengthTotalTerms + * SuffixLengthTotalTerms + *
  • TermFreqs --> TermFreqMinus1TotalTerms + *
  • Positions --> PositionDeltaTotalPositions + *
  • StartOffsets --> (AvgCharsPerTermTotalDistinctFields) + * StartOffsetDeltaTotalOffsets + *
  • Lengths --> LengthMinusTermLengthTotalOffsets + *
  • PayloadLengths --> PayloadLengthTotalPayloads + *
  • TermAndPayloads --> LZ4-compressed representation of < FieldTermsAndPayLoads + * >TotalFields + *
  • FieldTermsAndPayLoads --> Terms (Payloads) + *
  • DocBase, ChunkDocs, DocNumFields (with ChunkDocs==1) --> {@link + * DataOutput#writeVInt VInt} + *
  • AvgCharsPerTerm --> {@link DataOutput#writeInt Int} + *
  • DocNumFields (with ChunkDocs>=1), FieldNumOffs --> {@link PackedInts} array + *
  • FieldNumTerms, PrefixLength, SuffixLength, TermFreqMinus1, PositionDelta, + * StartOffsetDelta, LengthMinusTermLength, PayloadLength --> {@link + * BlockPackedWriter blocks of 64 packed ints} + *
  • Footer --> {@link CodecUtil#writeFooter CodecFooter} + * + *

    Notes: + *

    *
  • *

    An index file (extension .tvx). *

    *