org.apache.lucene.codecs

## Class BlockTreeTermsWriter

- java.lang.Object
- - org.apache.lucene.codecs.FieldsConsumer
  - - org.apache.lucene.codecs.BlockTreeTermsWriter

All Implemented Interfaces:
  Closeable, AutoCloseable

    public class BlockTreeTermsWriter
    extends FieldsConsumer

Block-based terms index and dictionary writer.

Writes terms dict and index, block-encoding (column stride) each term's metadata for each set of terms between two index terms.

Files:

- `.tim`: Term Dictionary
- `.tip`: Term Index

### Term Dictionary

The .tim file contains the list of terms in each field along with per-term statistics (such as docfreq) and per-term metadata (typically pointers to the postings list for that term in the inverted index).

The .tim is arranged in blocks: with blocks containing a variable number of entries (by default 25-48), where each entry is either a term or a reference to a sub-block.

NOTE: The term dictionary can plug into different postings implementations: the postings writer/reader are actually responsible for encoding and decoding the Postings Metadata and Term Metadata sections.

- TermsDict (.tim) --> Header, Postings Metadata, BlockNumBlocks, FieldSummary, DirOffset
- Block --> SuffixBlock, StatsBlock, MetadataBlock
- SuffixBlock --> EntryCount, SuffixLength, ByteSuffixLength
- StatsBlock --> StatsLength, <DocFreq, TotalTermFreq>EntryCount
- MetadataBlock --> MetaLength, <Term Metadata>EntryCount
- FieldSummary --> NumFields, <FieldNumber, NumTerms, RootCodeLength, ByteRootCodeLength, SumDocFreq, DocCount>NumFields
- Header --> `CodecHeader`
- DirOffset --> `Uint64`
- EntryCount,SuffixLength,StatsLength,DocFreq,MetaLength,NumFields, FieldNumber,RootCodeLength,DocCount --> `VInt`
- TotalTermFreq,NumTerms,SumTotalTermFreq,SumDocFreq --> `VLong`

Notes:

- Header is a `CodecHeader` storing the version information for the BlockTree implementation.
- DirOffset is a pointer to the FieldSummary section.
- DocFreq is the count of documents which contain the term.
- TotalTermFreq is the total number of occurrences of the term. This is encoded as the difference between the total number of occurrences and the DocFreq.
- FieldNumber is the fields number from `FieldInfos`. (.fnm)
- NumTerms is the number of unique terms for the field.
- RootCode points to the root block for the field.
- SumDocFreq is the total number of postings, the number of term-document pairs across the entire field.
- DocCount is the number of documents that have at least one posting for this field.
- PostingsMetadata and TermMetadata are plugged into by the specific postings implementation: these contain arbitrary per-file data (such as parameters or versioning information) and per-term data (such as pointers to inverted files).

### Term Index

The .tip file contains an index into the term dictionary, so that it can be accessed randomly. The index is also used to determine when a given term cannot exist on disk (in the .tim file), saving a disk seek.

- TermsIndex (.tip) --> Header, FSTIndexNumFields <IndexStartFP>NumFields, DirOffset
- Header --> `CodecHeader`
- DirOffset --> `Uint64`
- IndexStartFP --> `VLong`
- FSTIndex --> `FST<byte[]>`

Notes:

- The .tip file contains a separate FST for each field. The FST maps a term prefix to the on-disk block that holds all terms starting with that prefix. Each field's IndexStartFP points to its FST.
- DirOffset is a pointer to the start of the IndexStartFPs for all fields
- It's possible that an on-disk block would contain too many terms (more than the allowed maximum (default: 48)). When this happens, the block is sub-divided into new blocks (called "floor blocks"), and then the output in the FST for the block's prefix encodes the leading byte of each sub-block, and its file pointer.

See Also:
  `BlockTreeTermsReader`
WARNING: This API is experimental and might change in incompatible ways in the next release.

### Field Summary

Fields
`static int` `DEFAULT_MAX_BLOCK_SIZE` Suggested default value for the `maxItemsInBlock` parameter to `BlockTreeTermsWriter(SegmentWriteState,PostingsWriterBase,int,int)`.
`static int` `DEFAULT_MIN_BLOCK_SIZE` Suggested default value for the `minItemsInBlock` parameter to `BlockTreeTermsWriter(SegmentWriteState,PostingsWriterBase,int,int)`.
`static int` `TERMS_INDEX_VERSION_APPEND_ONLY` Append-only
`static int` `TERMS_INDEX_VERSION_CURRENT` Current index format.
`static int` `TERMS_INDEX_VERSION_START` Initial index format.
`static int` `TERMS_VERSION_APPEND_ONLY` Append-only
`static int` `TERMS_VERSION_CURRENT` Current terms format.
`static int` `TERMS_VERSION_START` Initial terms format.

### Constructor Summary

Constructors
Constructor and Description
`BlockTreeTermsWriter(SegmentWriteState state, PostingsWriterBase postingsWriter, int minItemsInBlock, int maxItemsInBlock)` Create a new writer.

### Method Summary

Methods
`TermsConsumer` `addField(FieldInfo field)` Add a new field
`void` `close()` Called when we are done adding everything.
`protected void` `writeHeader(IndexOutput out)` Writes the terms file header.
`protected void` `writeIndexHeader(IndexOutput out)` Writes the index file header.
`protected void` `writeIndexTrailer(IndexOutput indexOut, long dirStart)` Writes the index file trailer.
`protected void` `writeTrailer(IndexOutput out, long dirStart)` Writes the terms file trailer.

### Methods inherited from class org.apache.lucene.codecs.FieldsConsumer

`merge`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### DEFAULT_MIN_BLOCK_SIZE

    public static final int DEFAULT_MIN_BLOCK_SIZE

Suggested default value for the `minItemsInBlock` parameter to `BlockTreeTermsWriter(SegmentWriteState,PostingsWriterBase,int,int)`.

See Also:
  Constant Field Values

#### DEFAULT_MAX_BLOCK_SIZE

    public static final int DEFAULT_MAX_BLOCK_SIZE

Suggested default value for the `maxItemsInBlock` parameter to `BlockTreeTermsWriter(SegmentWriteState,PostingsWriterBase,int,int)`.

See Also:
  Constant Field Values

#### TERMS_VERSION_START

    public static final int TERMS_VERSION_START

Initial terms format.

See Also:
  Constant Field Values

#### TERMS_VERSION_APPEND_ONLY

    public static final int TERMS_VERSION_APPEND_ONLY

Append-only

See Also:
  Constant Field Values

#### TERMS_VERSION_CURRENT

    public static final int TERMS_VERSION_CURRENT

Current terms format.

See Also:
  Constant Field Values

#### TERMS_INDEX_VERSION_START

    public static final int TERMS_INDEX_VERSION_START

Initial index format.

See Also:
  Constant Field Values

#### TERMS_INDEX_VERSION_APPEND_ONLY

    public static final int TERMS_INDEX_VERSION_APPEND_ONLY

Append-only

See Also:
  Constant Field Values

#### TERMS_INDEX_VERSION_CURRENT

    public static final int TERMS_INDEX_VERSION_CURRENT

Current index format.

See Also:
  Constant Field Values

### Constructor Detail

#### BlockTreeTermsWriter

    public BlockTreeTermsWriter(SegmentWriteState state,
                        PostingsWriterBase postingsWriter,
                        int minItemsInBlock,
                        int maxItemsInBlock)
                         throws IOException

Create a new writer. The number of items (terms or sub-blocks) per block will aim to be between minItemsPerBlock and maxItemsPerBlock, though in some cases the blocks may be smaller than the min.

Throws:
  `IOException`

### Method Detail

#### writeHeader

    protected void writeHeader(IndexOutput out)
                        throws IOException

Writes the terms file header.

Throws:
  `IOException`

#### writeIndexHeader

    protected void writeIndexHeader(IndexOutput out)
                             throws IOException

Writes the index file header.

Throws:
  `IOException`

#### writeTrailer

    protected void writeTrailer(IndexOutput out,
                    long dirStart)
                         throws IOException

Writes the terms file trailer.

Throws:
  `IOException`

#### writeIndexTrailer

    protected void writeIndexTrailer(IndexOutput indexOut,
                         long dirStart)
                              throws IOException

Writes the index file trailer.

Throws:
  `IOException`

#### addField

    public TermsConsumer addField(FieldInfo field)
                           throws IOException

Description copied from class: `FieldsConsumer`
Add a new field

Specified by:
  `addField` in class `FieldsConsumer`
Throws:
  `IOException`

#### close

    public void close()
               throws IOException

Description copied from class: `FieldsConsumer`
Called when we are done adding everything.

Specified by:
  `close` in interface `Closeable`
Specified by:
  `close` in interface `AutoCloseable`
Specified by:
  `close` in class `FieldsConsumer`
Throws:
  `IOException`