## org.apache.lucene.analysis
Class BaseTokenStreamTestCase

    java.lang.Object
      org.junit.Assert
          org.apache.lucene.util.LuceneTestCase
              org.apache.lucene.analysis.BaseTokenStreamTestCase

    public abstract class BaseTokenStreamTestCase

extends LuceneTestCase

Base class for all Lucene unit tests that use TokenStreams.

When writing unit tests for analysis components, its highly recommended to use the helper methods here (especially in conjunction with `MockAnalyzer` or `MockTokenizer`), as they contain many assertions and checks to catch bugs.

See Also:
  `MockAnalyzer`, `MockTokenizer`

Nested Class Summary
`static interface` `BaseTokenStreamTestCase.CheckClearAttributesAttribute` Attribute that records if it was cleared or not.
`static class` `BaseTokenStreamTestCase.CheckClearAttributesAttributeImpl` Attribute that records if it was cleared or not.

Nested classes/interfaces inherited from class org.apache.lucene.util.LuceneTestCase
`LuceneTestCase.Nightly`

Field Summary

Fields inherited from class org.apache.lucene.util.LuceneTestCase
`classRules, random, RANDOM_MULTIPLIER, ruleChain, stores, TEMP_DIR, TEST_CLEAN_THREADS, TEST_DIRECTORY, TEST_ITER, TEST_ITER_MIN, TEST_LINE_DOCS_FILE, TEST_LOCALE, TEST_NIGHTLY, TEST_SEED, TEST_TIMEZONE, TEST_VERSION_CURRENT, testsFailed, VERBOSE`

Constructor Summary
`BaseTokenStreamTestCase()`

Method Summary
`static void` `assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a, String input, String[] output)`
`static void` `assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a, String input, String[] output, int[] posIncrements)`
`static void` `assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets)`
`static void` `assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets, int[] posIncrements)`
`static void` `assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements)`
`static void` `assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements, int[] posLengths)`
`static void` `assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements, int[] posLengths, boolean offsetsAreCorrect)`
`static void` `assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a, String input, String[] output, String[] types)`
`static void` `assertAnalyzesToPositions(org.apache.lucene.analysis.Analyzer a, String input, String[] output, int[] posIncrements, int[] posLengths)`
`static void` `assertAnalyzesToReuse(org.apache.lucene.analysis.Analyzer a, String input, String[] output)`
`static void` `assertAnalyzesToReuse(org.apache.lucene.analysis.Analyzer a, String input, String[] output, int[] posIncrements)`
`static void` `assertAnalyzesToReuse(org.apache.lucene.analysis.Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets)`
`static void` `assertAnalyzesToReuse(org.apache.lucene.analysis.Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets, int[] posIncrements)`
`static void` `assertAnalyzesToReuse(org.apache.lucene.analysis.Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements)`
`static void` `assertAnalyzesToReuse(org.apache.lucene.analysis.Analyzer a, String input, String[] output, String[] types)`
`static void` `assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts, String[] output)`
`static void` `assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts, String[] output, int[] posIncrements)`
`static void` `assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets)`
`static void` `assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, int[] posIncrements)`
`static void` `assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, int[] posIncrements, int[] posLengths, Integer finalOffset)`
`static void` `assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, int[] posIncrements, Integer finalOffset)`
`static void` `assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, Integer finalOffset)`
`static void` `assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements)`
`static void` `assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements, int[] posLengths, Integer finalOffset)`
`static void` `assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements, int[] posLengths, Integer finalOffset, boolean offsetsAreCorrect)`
`static void` `assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements, Integer finalOffset)`
`static void` `assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts, String[] output, String[] types)`
`static void` `checkAnalysisConsistency(Random random, org.apache.lucene.analysis.Analyzer a, boolean useCharFilter, String text)`
`static void` `checkAnalysisConsistency(Random random, org.apache.lucene.analysis.Analyzer a, boolean useCharFilter, String text, boolean offsetsAreCorrect)`
`static void` `checkOneTerm(org.apache.lucene.analysis.Analyzer a, String input, String expected)`
`static void` `checkOneTermReuse(org.apache.lucene.analysis.Analyzer a, String input, String expected)`
`static void` `checkRandomData(Random random, org.apache.lucene.analysis.Analyzer a, int iterations)` utility method for blasting tokenstreams with data to make sure they don't do anything crazy
`static void` `checkRandomData(Random random, org.apache.lucene.analysis.Analyzer a, int iterations, boolean simple)` utility method for blasting tokenstreams with data to make sure they don't do anything crazy
`static void` `checkRandomData(Random random, org.apache.lucene.analysis.Analyzer a, int iterations, int maxWordLength)` utility method for blasting tokenstreams with data to make sure they don't do anything crazy
`static void` `checkRandomData(Random random, org.apache.lucene.analysis.Analyzer a, int iterations, int maxWordLength, boolean simple)`
`static void` `checkRandomData(Random random, org.apache.lucene.analysis.Analyzer a, int iterations, int maxWordLength, boolean simple, boolean offsetsAreCorrect)`
`static String` `escape(String s)`
`protected String` `toDot(org.apache.lucene.analysis.Analyzer a, String inputText)`
`protected void` `toDotFile(org.apache.lucene.analysis.Analyzer a, String inputText, String localFileName)`

Methods inherited from class org.apache.lucene.util.LuceneTestCase

`afterClassLuceneTestCaseJ4, alwaysIgnoredTestMethod, assertEquals, assertEquals, assertEquals, assertEquals, assertSaneFieldCaches, asSet, assumeFalse, assumeNoException, assumeTrue, atLeast, atLeast, beforeClassLuceneTestCaseJ4, dumpArray, dumpIterator, getDataFile, getName, getTestClass, getTestLabel, isTestThread, localeForName, newDirectory, newDirectory, newDirectory, newDirectory, newField, newField, newField, newField, newField, newField, newFSDirectory, newFSDirectory, newIndexWriterConfig, newIndexWriterConfig, newLogMergePolicy, newLogMergePolicy, newLogMergePolicy, newLogMergePolicy, newLogMergePolicy, newSearcher, newSearcher, newTieredMergePolicy, newTieredMergePolicy, purgeFieldCache, randomDirectory, randomLocale, randomTimeZone, rarely, rarely, reportAdditionalFailureInfo, reportPartialFailureInfo, setMergeFactor, setUp, setUseCompoundFile, tearDown, usually, usually`

Methods inherited from class org.junit.Assert

`assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertFalse, assertFalse, assertNotNull, assertNotNull, assertNotSame, assertNotSame, assertNull, assertNull, assertSame, assertSame, assertThat, assertThat, assertTrue, assertTrue, fail, fail`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Constructor Detail

### BaseTokenStreamTestCase

    public BaseTokenStreamTestCase()

Method Detail

### assertTokenStreamContents

    public static void assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts,
                                                 String[] output,
                                                 int[] startOffsets,
                                                 int[] endOffsets,
                                                 String[] types,
                                                 int[] posIncrements,
                                                 int[] posLengths,
                                                 Integer finalOffset,
                                                 boolean offsetsAreCorrect)
                                          throws IOException

  Throws:
    `IOException`

### assertTokenStreamContents

    public static void assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts,
                                                 String[] output,
                                                 int[] startOffsets,
                                                 int[] endOffsets,
                                                 String[] types,
                                                 int[] posIncrements,
                                                 int[] posLengths,
                                                 Integer finalOffset)
                                          throws IOException

  Throws:
    `IOException`

### assertTokenStreamContents

    public static void assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts,
                                                 String[] output,
                                                 int[] startOffsets,
                                                 int[] endOffsets,
                                                 String[] types,
                                                 int[] posIncrements,
                                                 Integer finalOffset)
                                          throws IOException

  Throws:
    `IOException`

### assertTokenStreamContents

    public static void assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts,
                                                 String[] output,
                                                 int[] startOffsets,
                                                 int[] endOffsets,
                                                 String[] types,
                                                 int[] posIncrements)
                                          throws IOException

  Throws:
    `IOException`

### assertTokenStreamContents

    public static void assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts,
                                                 String[] output)
                                          throws IOException

  Throws:
    `IOException`

### assertTokenStreamContents

    public static void assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts,
                                                 String[] output,
                                                 String[] types)
                                          throws IOException

  Throws:
    `IOException`

### assertTokenStreamContents

    public static void assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts,
                                                 String[] output,
                                                 int[] posIncrements)
                                          throws IOException

  Throws:
    `IOException`

### assertTokenStreamContents

    public static void assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts,
                                                 String[] output,
                                                 int[] startOffsets,
                                                 int[] endOffsets)
                                          throws IOException

  Throws:
    `IOException`

### assertTokenStreamContents

    public static void assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts,
                                                 String[] output,
                                                 int[] startOffsets,
                                                 int[] endOffsets,
                                                 Integer finalOffset)
                                          throws IOException

  Throws:
    `IOException`

### assertTokenStreamContents

    public static void assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts,
                                                 String[] output,
                                                 int[] startOffsets,
                                                 int[] endOffsets,
                                                 int[] posIncrements)
                                          throws IOException

  Throws:
    `IOException`

### assertTokenStreamContents

    public static void assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts,
                                                 String[] output,
                                                 int[] startOffsets,
                                                 int[] endOffsets,
                                                 int[] posIncrements,
                                                 Integer finalOffset)
                                          throws IOException

  Throws:
    `IOException`

### assertTokenStreamContents

    public static void assertTokenStreamContents(org.apache.lucene.analysis.TokenStream ts,
                                                 String[] output,
                                                 int[] startOffsets,
                                                 int[] endOffsets,
                                                 int[] posIncrements,
                                                 int[] posLengths,
                                                 Integer finalOffset)
                                          throws IOException

  Throws:
    `IOException`

### assertAnalyzesTo

    public static void assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a,
                                        String input,
                                        String[] output,
                                        int[] startOffsets,
                                        int[] endOffsets,
                                        String[] types,
                                        int[] posIncrements)
                                 throws IOException

  Throws:
    `IOException`

### assertAnalyzesTo

    public static void assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a,
                                        String input,
                                        String[] output,
                                        int[] startOffsets,
                                        int[] endOffsets,
                                        String[] types,
                                        int[] posIncrements,
                                        int[] posLengths)
                                 throws IOException

  Throws:
    `IOException`

### assertAnalyzesTo

    public static void assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a,
                                        String input,
                                        String[] output,
                                        int[] startOffsets,
                                        int[] endOffsets,
                                        String[] types,
                                        int[] posIncrements,
                                        int[] posLengths,
                                        boolean offsetsAreCorrect)
                                 throws IOException

  Throws:
    `IOException`

### assertAnalyzesTo

    public static void assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a,
                                        String input,
                                        String[] output)
                                 throws IOException

  Throws:
    `IOException`

### assertAnalyzesTo

    public static void assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a,
                                        String input,
                                        String[] output,
                                        String[] types)
                                 throws IOException

  Throws:
    `IOException`

### assertAnalyzesTo

    public static void assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a,
                                        String input,
                                        String[] output,
                                        int[] posIncrements)
                                 throws IOException

  Throws:
    `IOException`

### assertAnalyzesToPositions

    public static void assertAnalyzesToPositions(org.apache.lucene.analysis.Analyzer a,
                                                 String input,
                                                 String[] output,
                                                 int[] posIncrements,
                                                 int[] posLengths)
                                          throws IOException

  Throws:
    `IOException`

### assertAnalyzesTo

    public static void assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a,
                                        String input,
                                        String[] output,
                                        int[] startOffsets,
                                        int[] endOffsets)
                                 throws IOException

  Throws:
    `IOException`

### assertAnalyzesTo

    public static void assertAnalyzesTo(org.apache.lucene.analysis.Analyzer a,
                                        String input,
                                        String[] output,
                                        int[] startOffsets,
                                        int[] endOffsets,
                                        int[] posIncrements)
                                 throws IOException

  Throws:
    `IOException`

### assertAnalyzesToReuse

    public static void assertAnalyzesToReuse(org.apache.lucene.analysis.Analyzer a,
                                             String input,
                                             String[] output,
                                             int[] startOffsets,
                                             int[] endOffsets,
                                             String[] types,
                                             int[] posIncrements)
                                      throws IOException

  Throws:
    `IOException`

### assertAnalyzesToReuse

    public static void assertAnalyzesToReuse(org.apache.lucene.analysis.Analyzer a,
                                             String input,
                                             String[] output)
                                      throws IOException

  Throws:
    `IOException`

### assertAnalyzesToReuse

    public static void assertAnalyzesToReuse(org.apache.lucene.analysis.Analyzer a,
                                             String input,
                                             String[] output,
                                             String[] types)
                                      throws IOException

  Throws:
    `IOException`

### assertAnalyzesToReuse

    public static void assertAnalyzesToReuse(org.apache.lucene.analysis.Analyzer a,
                                             String input,
                                             String[] output,
                                             int[] posIncrements)
                                      throws IOException

  Throws:
    `IOException`

### assertAnalyzesToReuse

    public static void assertAnalyzesToReuse(org.apache.lucene.analysis.Analyzer a,
                                             String input,
                                             String[] output,
                                             int[] startOffsets,
                                             int[] endOffsets)
                                      throws IOException

  Throws:
    `IOException`

### assertAnalyzesToReuse

    public static void assertAnalyzesToReuse(org.apache.lucene.analysis.Analyzer a,
                                             String input,
                                             String[] output,
                                             int[] startOffsets,
                                             int[] endOffsets,
                                             int[] posIncrements)
                                      throws IOException

  Throws:
    `IOException`

### checkOneTerm

    public static void checkOneTerm(org.apache.lucene.analysis.Analyzer a,
                                    String input,
                                    String expected)
                             throws IOException

  Throws:
    `IOException`

### checkOneTermReuse

    public static void checkOneTermReuse(org.apache.lucene.analysis.Analyzer a,
                                         String input,
                                         String expected)
                                  throws IOException

  Throws:
    `IOException`

### checkRandomData

    public static void checkRandomData(Random random,
                                       org.apache.lucene.analysis.Analyzer a,
                                       int iterations)
                                throws IOException

  utility method for blasting tokenstreams with data to make sure they don't do anything crazy

  Throws:
    `IOException`

### checkRandomData

    public static void checkRandomData(Random random,
                                       org.apache.lucene.analysis.Analyzer a,
                                       int iterations,
                                       int maxWordLength)
                                throws IOException

  utility method for blasting tokenstreams with data to make sure they don't do anything crazy

  Throws:
    `IOException`

### checkRandomData

    public static void checkRandomData(Random random,
                                       org.apache.lucene.analysis.Analyzer a,
                                       int iterations,
                                       boolean simple)
                                throws IOException

  utility method for blasting tokenstreams with data to make sure they don't do anything crazy

  Parameters:
    `simple` - true if only ascii strings will be used (try to avoid)
  Throws:
    `IOException`

### checkRandomData

    public static void checkRandomData(Random random,
                                       org.apache.lucene.analysis.Analyzer a,
                                       int iterations,
                                       int maxWordLength,
                                       boolean simple)
                                throws IOException

  Throws:
    `IOException`

### checkRandomData

    public static void checkRandomData(Random random,
                                       org.apache.lucene.analysis.Analyzer a,
                                       int iterations,
                                       int maxWordLength,
                                       boolean simple,
                                       boolean offsetsAreCorrect)
                                throws IOException

  Throws:
    `IOException`

### escape

    public static String escape(String s)

### checkAnalysisConsistency

    public static void checkAnalysisConsistency(Random random,
                                                org.apache.lucene.analysis.Analyzer a,
                                                boolean useCharFilter,
                                                String text)
                                         throws IOException

  Throws:
    `IOException`

### checkAnalysisConsistency

    public static void checkAnalysisConsistency(Random random,
                                                org.apache.lucene.analysis.Analyzer a,
                                                boolean useCharFilter,
                                                String text,
                                                boolean offsetsAreCorrect)
                                         throws IOException

  Throws:
    `IOException`

### toDot

    protected String toDot(org.apache.lucene.analysis.Analyzer a,
                           String inputText)
                    throws IOException

  Throws:
    `IOException`

### toDotFile

    protected void toDotFile(org.apache.lucene.analysis.Analyzer a,
                             String inputText,
                             String localFileName)
                      throws IOException

  Throws:
    `IOException`