module ANTLR3::FilterMode

If a lexer grammar specifies the filter = true</t> option, the generated Lexer code will include this module. It modifies the standard <tt>next_token to catch RecognitionErrors and skip ahead in the input until the token! method can match a token without raising a RecognitionError.

See www.antlr.org/wiki/display/ANTLR3/Lexical+filters for more info on lexer filter mode.

Public Instance Methods

already_parsed_rule?( rule ) click to toggle source
Calls superclass method
# File lib/antlr3/modes/filter.rb, line 54
def already_parsed_rule?( rule )
  @state.backtracking > 1 ? super( rule ) : false
end
memoize( rule, start_index, success ) click to toggle source
Calls superclass method
# File lib/antlr3/modes/filter.rb, line 50
def memoize( rule, start_index, success )
  super( rule, start_index, success ) if @state.backtracking > 1
end
next_token() click to toggle source
# File lib/antlr3/modes/filter.rb, line 19
def next_token
  # if at end-of-file, return the EOF token
  @input.peek == ANTLR3::EOF and return ANTLR3::EOF_TOKEN
  
  @state.token                = nil
  @state.channel              = ANTLR3::DEFAULT_CHANNEL
  @state.token_start_position = @input.index
  @state.token_start_column   = @input.column
  @state.token_start_line     = @input.line
  @state.text                 = nil
  @state.backtracking         = 1
  
  m = @input.mark
  token!
  @input.release( m )
  emit
  return @state.token
rescue ANTLR3::BacktrackingFailed
  # token! backtracks with synpred at backtracking==2
  # and we set the synpredgate to allow actions at level 1.
  @input.rewind( m )
  @input.consume # advance one char and try again
  retry
rescue ANTLR3::Error::RecognitionError => re
  # shouldn't happen in backtracking mode, but...
  report_error( re )
  recover( re )
ensure
  @state.backtracking = 0
end