module ANTLR3::Debug::TokenStream

A module that wraps token stream methods with debugging event code. A debuggable parser will extend its input stream with this module if the stream is not already a Debug::TokenStream.

Attributes

debug_listener[RW]
last_marker[R]

Public Class Methods

wrap( stream, debug_listener = nil ) click to toggle source
# File lib/antlr3/debug.rb, line 295
def self.wrap( stream, debug_listener = nil )
  stream.extend( self )
  stream.instance_eval do
    @initial_stream_state = true
    @debug_listener = debug_listener
    @last_marker = nil
  end
  return( stream )
end

Public Instance Methods

consume() click to toggle source
Calls superclass method
# File lib/antlr3/debug.rb, line 307
def consume
  @initial_stream_state and consume_initial_hidden_tokens
  a = index + 1 # the next position IF there are no hidden tokens in between
  t = super
  b = index     # the actual position after consuming
  @debug_listener.consume_token( t ) if @debug_listener
  
  # if b > a, report the consumption of hidden tokens
  for i in a...b
    @debug_listener.consume_hidden_token at( i )
  end
end
look( steps = 1 ) click to toggle source

Stream Methods ######################################

Calls superclass method
# File lib/antlr3/debug.rb, line 346
def look( steps = 1 )
  @initial_stream_state and consume_initial_hidden_tokens
  token = super( steps )
  @debug_listener.look( steps, token )
  return token
end
mark() click to toggle source
Calls superclass method
# File lib/antlr3/debug.rb, line 357
def mark
  @last_marker = super
  @debug_listener.mark( @last_marker )
  return @last_marker
end
peek( steps = 1 ) click to toggle source
# File lib/antlr3/debug.rb, line 353
def peek( steps = 1 )
  look( steps ).type
end
rewind( marker = nil, release = true ) click to toggle source
Calls superclass method
# File lib/antlr3/debug.rb, line 363
def rewind( marker = nil, release = true )
  @debug_listener.rewind( marker )
  super
end

Private Instance Methods

consume_initial_hidden_tokens() click to toggle source

after a token stream fills up its buffer by exhausting its token source, it may skip to an initial position beyond the first actual token, if there are hidden tokens at the beginning of the stream.

This private method is used to figure out if any hidden tokens were skipped initially, and then report their consumption to the debug listener

# File lib/antlr3/debug.rb, line 332
def consume_initial_hidden_tokens
  first_on_channel_token_index = self.index
  first_on_channel_token_index.times do |index|
    @debug_listener.consume_hidden_token at( index )
  end
  @initial_stream_state = false
end