org.apache.lucene.analysis.icu.segmentation

## Class DefaultICUTokenizerConfig

- java.lang.Object
- - org.apache.lucene.analysis.icu.segmentation.ICUTokenizerConfig
  - - org.apache.lucene.analysis.icu.segmentation.DefaultICUTokenizerConfig

    public class DefaultICUTokenizerConfig
    extends ICUTokenizerConfig

Default `ICUTokenizerConfig` that is generally applicable to many languages.

Generally tokenizes Unicode text according to UAX#29 (`BreakIterator.getWordInstance(ULocale.ROOT)`), but with the following tailorings:

- Thai text is broken into words with a `DictionaryBasedBreakIterator`
- Lao, Myanmar, and Khmer text is broken into syllables based on custom BreakIterator rules.
- Hebrew text has custom tailorings to handle special cases involving punctuation.

WARNING: This API is experimental and might change in incompatible ways in the next release.

### Field Summary

Fields
`static String` `WORD_HANGUL` Token type for words containing Korean hangul
`static String` `WORD_HIRAGANA` Token type for words containing Japanese hiragana
`static String` `WORD_IDEO` Token type for words containing ideographic characters
`static String` `WORD_KATAKANA` Token type for words containing Japanese katakana
`static String` `WORD_LETTER` Token type for words that contain letters
`static String` `WORD_NUMBER` Token type for words that appear to be numbers

### Constructor Summary

Constructors
Constructor and Description
`DefaultICUTokenizerConfig()` Creates a new config.

### Method Summary

Methods
`com.ibm.icu.text.BreakIterator` `getBreakIterator(int script)` Return a breakiterator capable of processing a given script.
`String` `getType(int script, int ruleStatus)` Return a token type value for a given script and BreakIterator rule status.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### WORD_IDEO

    public static final String WORD_IDEO

Token type for words containing ideographic characters

#### WORD_HIRAGANA

    public static final String WORD_HIRAGANA

Token type for words containing Japanese hiragana

#### WORD_KATAKANA

    public static final String WORD_KATAKANA

Token type for words containing Japanese katakana

#### WORD_HANGUL

    public static final String WORD_HANGUL

Token type for words containing Korean hangul

#### WORD_LETTER

    public static final String WORD_LETTER

Token type for words that contain letters

#### WORD_NUMBER

    public static final String WORD_NUMBER

Token type for words that appear to be numbers

### Constructor Detail

#### DefaultICUTokenizerConfig

    public DefaultICUTokenizerConfig()

Creates a new config. This object is lightweight, but the first time the class is referenced, breakiterators will be initialized.

### Method Detail

#### getBreakIterator

    public com.ibm.icu.text.BreakIterator getBreakIterator(int script)

Description copied from class: `ICUTokenizerConfig`
Return a breakiterator capable of processing a given script.

Specified by:
  `getBreakIterator` in class `ICUTokenizerConfig`

#### getType

    public String getType(int script,
                 int ruleStatus)

Description copied from class: `ICUTokenizerConfig`
Return a token type value for a given script and BreakIterator rule status.

Specified by:
  `getType` in class `ICUTokenizerConfig`