buftok.rb

Path: lib/em/buftok.rb
Last Update: Sat Apr 20 14:56:51 +0000 2013

BufferedTokenizer takes a delimiter upon instantiation, or acts line-based by default. It allows input to be spoon-fed from some outside source which receives arbitrary length datagrams which may-or-may-not contain the token by which entities are delimited.

By default, new BufferedTokenizers will operate on lines delimited by "\n" by default or allow you to specify any delimiter token you so choose, which will then be used by String#split to tokenize the input data

@example Using BufferedTokernizer to parse lines out of incoming data

  module LineBufferedConnection
    def receive_data(data)
      (@buffer ||= BufferedTokenizer.new).extract(data).each do |line|
        receive_line(line)
      end
    end
  end

@author Tony Arcieri @author Martin Emde

[Validate]