org.apache.lucene.analysis.compound

## Class CompoundWordTokenFilterBase

- java.lang.Object
- - org.apache.lucene.util.AttributeSource
  - - org.apache.lucene.analysis.TokenStream
    - - org.apache.lucene.analysis.TokenFilter
      - - org.apache.lucene.analysis.compound.CompoundWordTokenFilterBase

All Implemented Interfaces:
  Closeable, AutoCloseable

Direct Known Subclasses:
  DictionaryCompoundWordTokenFilter, HyphenationCompoundWordTokenFilter

    public abstract class CompoundWordTokenFilterBase
    extends TokenFilter

Base class for decomposition token filters.

You must specify the required `Version` compatibility when creating CompoundWordTokenFilterBase:

- As of 3.1, CompoundWordTokenFilterBase correctly handles Unicode 4.0 supplementary characters in strings and char arrays provided as compound word dictionaries.

### Nested Class Summary

Nested Classes
`protected class` `CompoundWordTokenFilterBase.CompoundToken` Helper class to hold decompounded token information

### Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

`AttributeSource.AttributeFactory, AttributeSource.State`

### Field Summary

Fields
`static int` `DEFAULT_MAX_SUBWORD_SIZE` The default for maximal length of subwords that get propagated to the output of this filter
`static int` `DEFAULT_MIN_SUBWORD_SIZE` The default for minimal length of subwords that get propagated to the output of this filter
`static int` `DEFAULT_MIN_WORD_SIZE` The default for minimal word length that gets decomposed
`protected CharArraySet` `dictionary`
`protected int` `maxSubwordSize`
`protected int` `minSubwordSize`
`protected int` `minWordSize`
`protected OffsetAttribute` `offsetAtt`
`protected boolean` `onlyLongestMatch`
`protected CharTermAttribute` `termAtt`
`protected LinkedList<CompoundWordTokenFilterBase.CompoundToken>` `tokens`

### Fields inherited from class org.apache.lucene.analysis.TokenFilter

`input`

### Constructor Summary

Constructors
`protected` `CompoundWordTokenFilterBase(Version matchVersion, TokenStream input, CharArraySet dictionary)`
`protected` `CompoundWordTokenFilterBase(Version matchVersion, TokenStream input, CharArraySet dictionary, boolean onlyLongestMatch)`
`protected` `CompoundWordTokenFilterBase(Version matchVersion, TokenStream input, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch)`

### Method Summary

Methods
`protected abstract void` `decompose()` Decomposes the current `termAtt` and places `CompoundWordTokenFilterBase.CompoundToken` instances in the `tokens` list.
`boolean` `incrementToken()`
`void` `reset()`

### Methods inherited from class org.apache.lucene.analysis.TokenFilter

`close, end`

### Methods inherited from class org.apache.lucene.util.AttributeSource

`addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState`

### Methods inherited from class java.lang.Object

`clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### DEFAULT_MIN_WORD_SIZE

    public static final int DEFAULT_MIN_WORD_SIZE

The default for minimal word length that gets decomposed

See Also:
  Constant Field Values

#### DEFAULT_MIN_SUBWORD_SIZE

    public static final int DEFAULT_MIN_SUBWORD_SIZE

The default for minimal length of subwords that get propagated to the output of this filter

See Also:
  Constant Field Values

#### DEFAULT_MAX_SUBWORD_SIZE

    public static final int DEFAULT_MAX_SUBWORD_SIZE

The default for maximal length of subwords that get propagated to the output of this filter

See Also:
  Constant Field Values

#### dictionary

    protected final CharArraySet dictionary

#### tokens

    protected final LinkedList<CompoundWordTokenFilterBase.CompoundToken> tokens

#### minWordSize

    protected final int minWordSize

#### minSubwordSize

    protected final int minSubwordSize

#### maxSubwordSize

    protected final int maxSubwordSize

#### onlyLongestMatch

    protected final boolean onlyLongestMatch

#### termAtt

    protected final CharTermAttribute termAtt

#### offsetAtt

    protected final OffsetAttribute offsetAtt

### Constructor Detail

#### CompoundWordTokenFilterBase

    protected CompoundWordTokenFilterBase(Version matchVersion,
                               TokenStream input,
                               CharArraySet dictionary,
                               boolean onlyLongestMatch)

#### CompoundWordTokenFilterBase

    protected CompoundWordTokenFilterBase(Version matchVersion,
                               TokenStream input,
                               CharArraySet dictionary)

#### CompoundWordTokenFilterBase

    protected CompoundWordTokenFilterBase(Version matchVersion,
                               TokenStream input,
                               CharArraySet dictionary,
                               int minWordSize,
                               int minSubwordSize,
                               int maxSubwordSize,
                               boolean onlyLongestMatch)

### Method Detail

#### incrementToken

    public final boolean incrementToken()
                                 throws IOException

Specified by:
  `incrementToken` in class `TokenStream`
Throws:
  `IOException`

#### decompose

    protected abstract void decompose()

Decomposes the current `termAtt` and places `CompoundWordTokenFilterBase.CompoundToken` instances in the `tokens` list. The original token may not be placed in the list, as it is automatically passed through this filter.

#### reset

    public void reset()
               throws IOException

Overrides:
  `reset` in class `TokenFilter`
Throws:
  `IOException`