## org.exist.storage.analysis
Class SimpleTokenizer

    java.lang.Object
      org.exist.storage.analysis.SimpleTokenizer

All Implemented Interfaces:
  Tokenizer

public class SimpleTokenizer
extends java.lang.Object
implements Tokenizer

This is the default class used by the fulltext indexer for tokenizing a string into words. Known token types are defined by class Token.

Author:
  Wolfgang Meier

Constructor Summary
`SimpleTokenizer()`
`SimpleTokenizer(boolean stem)`

Method Summary
`int` `getLength()`
`java.lang.String` `getText()`
`static void` `main(java.lang.String[] args)`
`TextToken` `nextToken()`
`TextToken` `nextToken(boolean wildcards)`
`void` `setStemming(boolean stem)`
`void` `setText(java.lang.CharSequence text)`

Methods inherited from class java.lang.Object
`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Constructor Detail

### SimpleTokenizer

    public SimpleTokenizer()

### SimpleTokenizer

    public SimpleTokenizer(boolean stem)

Method Detail

### setStemming

    public void setStemming(boolean stem)

  Specified by:
    `setStemming` in interface `Tokenizer`

### getLength

    public int getLength()

### getText

    public java.lang.String getText()

### nextToken

    public TextToken nextToken()

  Specified by:
    `nextToken` in interface `Tokenizer`

### nextToken

    public TextToken nextToken(boolean wildcards)

  Specified by:
    `nextToken` in interface `Tokenizer`

### setText

    public void setText(java.lang.CharSequence text)

  Specified by:
    `setText` in interface `Tokenizer`

### main

    public static void main(java.lang.String[] args)

Copyright (C) Wolfgang Meier. All rights reserved.