org.apache.lucene.search.similarities

## Class BM25Similarity

- java.lang.Object
- - org.apache.lucene.search.similarities.Similarity
  - - org.apache.lucene.search.similarities.BM25Similarity

    public class BM25Similarity
    extends Similarity

BM25 Similarity. Introduced in Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. Okapi at TREC-3. In Proceedings of the Third Text REtrieval Conference (TREC 1994). Gaithersburg, USA, November 1994.

WARNING: This API is experimental and might change in incompatible ways in the next release.

### Nested Class Summary

### Nested classes/interfaces inherited from class org.apache.lucene.search.similarities.Similarity

`Similarity.ExactSimScorer, Similarity.SimWeight, Similarity.SloppySimScorer`

### Field Summary

Fields
`protected boolean` `discountOverlaps` True if overlap tokens (tokens with a position of increment of zero) are discounted from the document's length.

### Constructor Summary

Constructors
Constructor and Description
`BM25Similarity()` BM25 with these default values: `k1 = 1.2`, `b = 0.75`.
`BM25Similarity(float k1, float b)` BM25 with the supplied parameter values.

### Method Summary

Methods
`protected float` `avgFieldLength(CollectionStatistics collectionStats)` The default implementation computes the average as `sumTotalTermFreq / maxDoc`, or returns `1` if the index does not store sumTotalTermFreq (Lucene 3.x indexes or any field that omits frequency information).
`long` `computeNorm(FieldInvertState state)` Computes the normalization value for a field, given the accumulated state of term processing for this field (see `FieldInvertState`).
`Similarity.SimWeight` `computeWeight(float queryBoost, CollectionStatistics collectionStats, TermStatistics... termStats)` Compute any collection-level weight (e.g.
`protected float` `decodeNormValue(byte b)` The default implementation returns `1 / f2` where `f` is `SmallFloat.byte315ToFloat(byte)`.
`protected byte` `encodeNormValue(float boost, int fieldLength)` The default implementation encodes `boost / sqrt(length)` with `SmallFloat.floatToByte315(float)`.
`Similarity.ExactSimScorer` `exactSimScorer(Similarity.SimWeight stats, AtomicReaderContext context)` Creates a new `Similarity.ExactSimScorer` to score matching documents from a segment of the inverted index.
`float` `getB()` Returns the `b` parameter
`boolean` `getDiscountOverlaps()` Returns true if overlap tokens are discounted from the document's length.
`float` `getK1()` Returns the `k1` parameter
`protected float` `idf(long docFreq, long numDocs)` Implemented as `log(1 + (numDocs - docFreq + 0.5)/(docFreq + 0.5))`.
`Explanation` `idfExplain(CollectionStatistics collectionStats, TermStatistics termStats)` Computes a score factor for a simple term and returns an explanation for that score factor.
`Explanation` `idfExplain(CollectionStatistics collectionStats, TermStatistics[] termStats)` Computes a score factor for a phrase.
`protected float` `scorePayload(int doc, int start, int end, BytesRef payload)` The default implementation returns `1`
`void` `setDiscountOverlaps(boolean v)` Sets whether overlap tokens (Tokens with 0 position increment) are ignored when computing norm.
`protected float` `sloppyFreq(int distance)` Implemented as `1 / (distance + 1)`.
`Similarity.SloppySimScorer` `sloppySimScorer(Similarity.SimWeight stats, AtomicReaderContext context)` Creates a new `Similarity.SloppySimScorer` to score matching documents from a segment of the inverted index.
`String` `toString()`

### Methods inherited from class org.apache.lucene.search.similarities.Similarity

`coord, queryNorm`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

### Field Detail

#### discountOverlaps

    protected boolean discountOverlaps

True if overlap tokens (tokens with a position of increment of zero) are discounted from the document's length.

### Constructor Detail

#### BM25Similarity

    public BM25Similarity(float k1,
                  float b)

BM25 with the supplied parameter values.

Parameters:
  `k1` - Controls non-linear term frequency normalization (saturation).
  `b` - Controls to what degree document length normalizes tf values.

#### BM25Similarity

    public BM25Similarity()

BM25 with these default values:

- `k1 = 1.2`,
- `b = 0.75`.

### Method Detail

#### idf

    protected float idf(long docFreq,
            long numDocs)

Implemented as `log(1 + (numDocs - docFreq + 0.5)/(docFreq + 0.5))`.

#### sloppyFreq

    protected float sloppyFreq(int distance)

Implemented as `1 / (distance + 1)`.

#### scorePayload

    protected float scorePayload(int doc,
                     int start,
                     int end,
                     BytesRef payload)

The default implementation returns `1`

#### avgFieldLength

    protected float avgFieldLength(CollectionStatistics collectionStats)

The default implementation computes the average as `sumTotalTermFreq / maxDoc`, or returns `1` if the index does not store sumTotalTermFreq (Lucene 3.x indexes or any field that omits frequency information).

#### encodeNormValue

    protected byte encodeNormValue(float boost,
                       int fieldLength)

The default implementation encodes `boost / sqrt(length)` with `SmallFloat.floatToByte315(float)`. This is compatible with Lucene's default implementation. If you change this, then you should change `decodeNormValue(byte)` to match.

#### decodeNormValue

    protected float decodeNormValue(byte b)

The default implementation returns `1 / f2` where `f` is `SmallFloat.byte315ToFloat(byte)`.

#### setDiscountOverlaps

    public void setDiscountOverlaps(boolean v)

Sets whether overlap tokens (Tokens with 0 position increment) are ignored when computing norm. By default this is true, meaning overlap tokens do not count when computing norms.

#### getDiscountOverlaps

    public boolean getDiscountOverlaps()

Returns true if overlap tokens are discounted from the document's length.

See Also:
  `setDiscountOverlaps(boolean)`

#### computeNorm

    public final long computeNorm(FieldInvertState state)

Description copied from class: `Similarity`
Computes the normalization value for a field, given the accumulated state of term processing for this field (see `FieldInvertState`).

Matches in longer fields are less precise, so implementations of this method usually set smaller values when `state.getLength()` is large, and larger values when `state.getLength()` is small.

Specified by:
  `computeNorm` in class `Similarity`
Parameters:
  `state` - current processing state for this field
Returns:
  computed norm value

#### idfExplain

    public Explanation idfExplain(CollectionStatistics collectionStats,
                         TermStatistics termStats)

Computes a score factor for a simple term and returns an explanation for that score factor.

The default implementation uses:

     idf(docFreq, searcher.maxDoc());

Note that `CollectionStatistics.maxDoc()` is used instead of `IndexReader#numDocs()` because also `TermStatistics.docFreq()` is used, and when the latter is inaccurate, so is `CollectionStatistics.maxDoc()`, and in the same direction. In addition, `CollectionStatistics.maxDoc()` is more efficient to compute

Parameters:
  `collectionStats` - collection-level statistics
  `termStats` - term-level statistics for the term
Returns:
  an Explain object that includes both an idf score factor and an explanation for the term.

#### idfExplain

    public Explanation idfExplain(CollectionStatistics collectionStats,
                         TermStatistics[] termStats)

Computes a score factor for a phrase.

The default implementation sums the idf factor for each term in the phrase.

Parameters:
  `collectionStats` - collection-level statistics
  `termStats` - term-level statistics for the terms in the phrase
Returns:
  an Explain object that includes both an idf score factor for the phrase and an explanation for each term.

#### computeWeight

    public final Similarity.SimWeight computeWeight(float queryBoost,
                                     CollectionStatistics collectionStats,
                                     TermStatistics... termStats)

Description copied from class: `Similarity`
Compute any collection-level weight (e.g. IDF, average document length, etc) needed for scoring a query.

Specified by:
  `computeWeight` in class `Similarity`
Parameters:
  `queryBoost` - the query-time boost.
  `collectionStats` - collection-level statistics, such as the number of tokens in the collection.
  `termStats` - term-level statistics, such as the document frequency of a term across the collection.
Returns:
  SimWeight object with the information this Similarity needs to score a query.

#### exactSimScorer

    public final Similarity.ExactSimScorer exactSimScorer(Similarity.SimWeight stats,
                                           AtomicReaderContext context)
                                                   throws IOException

Description copied from class: `Similarity`
Creates a new `Similarity.ExactSimScorer` to score matching documents from a segment of the inverted index.

Specified by:
  `exactSimScorer` in class `Similarity`
Parameters:
  `stats` - collection information from `Similarity.computeWeight(float, CollectionStatistics, TermStatistics...)`
  `context` - segment of the inverted index to be scored.
Returns:
  ExactSimScorer for scoring documents across `context`
Throws:
  `IOException` - if there is a low-level I/O error

#### sloppySimScorer

    public final Similarity.SloppySimScorer sloppySimScorer(Similarity.SimWeight stats,
                                             AtomicReaderContext context)
                                                     throws IOException

Description copied from class: `Similarity`
Creates a new `Similarity.SloppySimScorer` to score matching documents from a segment of the inverted index.

Specified by:
  `sloppySimScorer` in class `Similarity`
Parameters:
  `stats` - collection information from `Similarity.computeWeight(float, CollectionStatistics, TermStatistics...)`
  `context` - segment of the inverted index to be scored.
Returns:
  SloppySimScorer for scoring documents across `context`
Throws:
  `IOException` - if there is a low-level I/O error

#### toString

    public String toString()

Overrides:
  `toString` in class `Object`

#### getK1

    public float getK1()

Returns the `k1` parameter

See Also:
  `BM25Similarity(float, float)`

#### getB

    public float getB()

Returns the `b` parameter

See Also:
  `BM25Similarity(float, float)`