## org.apache.lucene.analysis.cn.smart.hhmm
Class SegTokenFilter

    java.lang.Object
      org.apache.lucene.analysis.cn.smart.hhmm.SegTokenFilter

    public class SegTokenFilter

extends Object

Filters a `SegToken` by converting full-width latin to half-width, then lowercasing latin. Additionally, all punctuation is converted into `Utility.COMMON_DELIMITER`

WARNING: This API is experimental and might change in incompatible ways in the next release.

Constructor Summary
`SegTokenFilter()`

Method Summary
`SegToken` `filter(SegToken token)` Filter an input `SegToken`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Constructor Detail

### SegTokenFilter

    public SegTokenFilter()

Method Detail

### filter

    public SegToken filter(SegToken token)

  Filter an input `SegToken`

  Full-width latin will be converted to half-width, then all latin will be lowercased. All punctuation is converted into `Utility.COMMON_DELIMITER`

  Parameters:
    `token` - input `SegToken`
  Returns:
    normalized `SegToken`