org.apache.lucene.analysis.core

## Class WhitespaceTokenizer

- java.lang.Object
- - org.apache.lucene.util.AttributeSource
  - - org.apache.lucene.analysis.TokenStream
    - - org.apache.lucene.analysis.Tokenizer
      - - org.apache.lucene.analysis.util.CharTokenizer
        - - org.apache.lucene.analysis.core.WhitespaceTokenizer

All Implemented Interfaces:
  Closeable, AutoCloseable

    public final class WhitespaceTokenizer
    extends CharTokenizer

A WhitespaceTokenizer is a tokenizer that divides text at whitespace. Adjacent sequences of non-Whitespace characters form tokens.

You must specify the required `Version` compatibility when creating `WhitespaceTokenizer`:

- As of 3.1, `CharTokenizer` uses an int based API to normalize and detect token characters. See `CharTokenizer.isTokenChar(int)` and `CharTokenizer.normalize(int)` for details.

### Nested Class Summary

### Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

`AttributeSource.AttributeFactory, AttributeSource.State`

### Field Summary

### Fields inherited from class org.apache.lucene.analysis.Tokenizer

`input`

### Constructor Summary

Constructors
Constructor and Description
`WhitespaceTokenizer(Version matchVersion, AttributeSource.AttributeFactory factory, Reader in)` Construct a new WhitespaceTokenizer using a given `AttributeSource.AttributeFactory`.
`WhitespaceTokenizer(Version matchVersion, AttributeSource source, Reader in)` Construct a new WhitespaceTokenizer using a given `AttributeSource`.
`WhitespaceTokenizer(Version matchVersion, Reader in)` Construct a new WhitespaceTokenizer.

### Method Summary

Methods
`protected boolean` `isTokenChar(int c)` Collects only characters which do not satisfy `Character.isWhitespace(int)`.

### Methods inherited from class org.apache.lucene.analysis.util.CharTokenizer

`end, incrementToken, normalize, reset`

### Methods inherited from class org.apache.lucene.analysis.Tokenizer

`close, correctOffset, setReader`

### Methods inherited from class org.apache.lucene.util.AttributeSource

`addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState`

### Methods inherited from class java.lang.Object

`clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### WhitespaceTokenizer

    public WhitespaceTokenizer(Version matchVersion,
                       Reader in)

Construct a new WhitespaceTokenizer. * @param matchVersion Lucene version to match See above

Parameters:
  `in` - the input to split up into tokens

#### WhitespaceTokenizer

    public WhitespaceTokenizer(Version matchVersion,
                       AttributeSource source,
                       Reader in)

Construct a new WhitespaceTokenizer using a given `AttributeSource`.

Parameters:
  `matchVersion` - Lucene version to match See above
  `source` - the attribute source to use for this `Tokenizer`
  `in` - the input to split up into tokens

#### WhitespaceTokenizer

    public WhitespaceTokenizer(Version matchVersion,
                       AttributeSource.AttributeFactory factory,
                       Reader in)

Construct a new WhitespaceTokenizer using a given `AttributeSource.AttributeFactory`.

Parameters:
  `matchVersion` - Lucene version to match See above
  `factory` - the attribute factory to use for this `Tokenizer`
  `in` - the input to split up into tokens

### Method Detail

#### isTokenChar

    protected boolean isTokenChar(int c)

Collects only characters which do not satisfy `Character.isWhitespace(int)`.

Specified by:
  `isTokenChar` in class `CharTokenizer`