class Syntax::Default
A default tokenizer for handling syntaxes that are not explicitly handled elsewhere. It simply yields the given text as a single token.
Public Instance Methods
tokenize( text ) { |token( text, :normal )| ... }
click to toggle source
Yield the given text as a single token.
# File lib/syntax.rb, line 10 def tokenize( text ) yield Token.new( text, :normal ) end