class Syntax::Token
A single token extracted by a tokenizer. It is simply the lexeme itself, decorated with a 'group' attribute to identify the type of the lexeme.
Attributes
group[R]
the type of the lexeme that was extracted.
instruction[R]
the instruction associated with this token (:none, :region_open, or :region_close)
Public Class Methods
new( text, group, instruction = :none )
click to toggle source
Create a new Token representing the given text, and belonging to the given group.
Calls superclass method
# File lib/syntax/common.rb, line 19 def initialize( text, group, instruction = :none ) super text @group = group @instruction = instruction end