Lingua::ZH::WordSegmenter(3)

NAME
Lingua::ZH::WordSegmenter - Simplified Chinese Word Segmentation

VERSION
Version 0.01

SYNOPSIS
    use Lingua::ZH::WordSegmenter;

    my $segmenter = Lingua::ZH::WordSegmenter->new();
    print encode('gbk', $segmenter->seg($_) );

Description
This is a perl version of simplified Chinese word segmentation.

The algorithm for this segmenter is to search the longest word at each point from both left and right directions, and choose the one with higher frequency product.

The original program is from the CPAN module Lingua::ZH::WordSegment (http://search.cpan.org/~chenyr/) I did the follwing changes: 1) make the interface object oriented; 2) make the internal string into utf8; 3) using sogou's dictionary (http://www.sogou.com/labs/dl/w.html) as the default dictionary.

METHODS

$segmenter = Lingua::ZH::WordSegmenter->new(%opinions)
    This method constructs a new "Lingua::ZH::WordSegmenter" object and returns it. Key/value pair arguments may be provided to set up the initial state. The following options correspond to attribute methods described below:

       KEY            PURPOSE                       DEFAULT
       -----------    -------------                 --------------------
       dic            filename of the dic           sogou dic
       dic_encoding   encoding of the dic           "gbk"
       seperator      string to seperate wrods      " "
       verbose        show the segment process      0

$segmenter->seg($input,[$encoding])
    Segment a input string, you can specify the encoding by the optional parameter.

    The return result is encoded in utf8 format.

SEE ALSO
Lingua::ZH::WordSegment

AUTHOR
Zhang Jun, "<jzhang533 at gmail.com>"

COPYRIGHT & LICENSE
Copyright 2007 Zhang Jun, all rights reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.