module ANTLR3::TokenSource

TokenSource is a simple mixin module that demands an implementation of the method next_token. In return, it defines methods next and each, which provide basic iterator methods for token generators. Furthermore, it includes Enumerable to provide the standard Ruby iteration methods to token generators, like lexers.

Public Instance Methods

each() { |token| ... } click to toggle source
# File lib/antlr3/token.rb, line 319
def each
  block_given? or return enum_for( :each )
  while token = next_token and token.type != EOF
    yield( token )
  end
  return self
end
next() click to toggle source
# File lib/antlr3/token.rb, line 313
def next
  token = next_token()
  raise StopIteration if token.nil? || token.type == EOF
  return token
end
to_stream( options = {} ) { |t, stream| ... } click to toggle source
# File lib/antlr3/token.rb, line 327
def to_stream( options = {} )
  if block_given?
    CommonTokenStream.new( self, options ) { | t, stream | yield( t, stream ) }
  else
    CommonTokenStream.new( self, options )
  end
end