module Parslet

A simple parser generator library. Typical usage would look like this: 

require 'parslet'

class MyParser < Parslet::Parser
  rule(:a) { str('a').repeat }
  root(:a)        
end

pp MyParser.new.parse('aaaa')   # => 'aaaa'@0
pp MyParser.new.parse('bbbb')   # => Parslet::Atoms::ParseFailed:
                                #    Don't know what to do with bbbb at line 1 char 1.

The simple DSL allows you to define grammars in PEG-style. This kind of grammar construction does away with the ambiguities that usually comes with parsers; instead, it allows you to construct grammars that are easier to debug, since less magic is involved.

Parslet is typically used in stages:

The first stage is traditionally intermingled with the second stage; output from the second stage is usually called the 'Abstract Syntax Tree' or AST.

The stages are completely decoupled; You can change your grammar around and use the second stage to isolate the rest of your code from the changes you've effected.

Further reading ↑

All parslet atoms are subclasses of {Parslet::Atoms::Base}. You might want to look at all of those: {Parslet::Atoms::Re}, {Parslet::Atoms::Str}, {Parslet::Atoms::Repetition}, {Parslet::Atoms::Sequence}, {Parslet::Atoms::Alternative}.

When things go wrong ↑

A parse that fails will raise {Parslet::ParseFailed}. This exception contains all the details of what went wrong, including a detailed error trace that can be printed out as an ascii tree. ({Parslet::Cause})

Public Class Methods

any() click to toggle source

Returns an atom matching any character. It acts like the '.' (dot) character in regular expressions.

any.parse('a')    # => 'a'

@return [Parslet::Atoms::Re] a parslet atom

# File lib/parslet.rb, line 168
def any
  Atoms::Re.new('.')
end
dynamic(&block) click to toggle source

Designates a piece of the parser as being dynamic. Dynamic parsers can either return a parser at runtime, which will be applied on the input, or return a result from a parse.

Dynamic parse pieces are never cached and can introduce performance abnormalitites - use sparingly where other constructs fail.

Example: 

# Parses either 'a' or 'b', depending on the weather
dynamic { rand() < 0.5 ? str('a') : str('b') }

# File lib/parslet.rb, line 199
def dynamic(&block)
  Parslet::Atoms::Dynamic.new(block)
end
exp(str) click to toggle source

A special kind of atom that allows embedding whole treetop expressions into parslet construction. 

# the same as str('a') >> str('b').maybe
exp(%Q("a" "b"?))

@param str [String] a treetop expression @return [Parslet::Atoms::Base] the corresponding parslet parser

# File lib/parslet.rb, line 253
def exp(str)
  Parslet::Expression.new(str).to_parslet
end
included(base) click to toggle source

Extends classes that include Parslet with the module {Parslet::ClassMethods}.

# File lib/parslet.rb, line 52
def self.included(base)
  base.extend(ClassMethods)
end
infix_expression(element, *operations, &reducer) click to toggle source

Returns a parslet atom that parses infix expressions. Operations are specified as a list of <atom, precedence, associativity> tuples, where atom is simply the parslet atom that matches an operator, precedence is a number and associativity is either :left or :right.

Higher precedence indicates that the operation should bind tighter than other operations with lower precedence. In common algebra, '+' has lower precedence than '*'. So you would have a precedence of 1 for '+' and a precedence of 2 for '*'. Only the order relation between these two counts, so any number would work.

Associativity is what decides what interpretation to take for strings that are ambiguous like '1 + 2 + 3'. If '+' is specified as left associative, the expression would be interpreted as '(1 + 2) + 3'. If right associativity is chosen, it would be interpreted as '1 + (2 + 3)'. Note that the hash trees output reflect that choice as well.

An optional block can be provided in order to manipulate the generated tree. The block will be called on each operator and passed 3 arguments: the left operand, the operator, and the right operand.

Examples:

infix_expression(integer, [add_op, 1, :left])
# would parse things like '1 + 2'

infix_expression(integer, [add_op, 1, :left]) { |l,o,r| { :plus => [l, r] } }
# would parse '1 + 2 + 3' as:
# { :plus => [1, { :plus => [2, 3] }] }

@param element [Parslet::Atoms::Base] elements that take the NUMBER position

in the expression

@param operations [Array<(Parslet::Atoms::Base, Integer, {:left, :right})>]

@see Parslet::Atoms::Infix

# File lib/parslet.rb, line 239
def infix_expression(element, *operations, &reducer)
  Parslet::Atoms::Infix.new(element, operations, &reducer)
end
match(str=nil) click to toggle source

Returns an atom matching a character class. All regular expressions can be used, as long as they match only a single character at a time. 

match('[ab]')     # will match either 'a' or 'b'
match('[\n\s]')   # will match newlines and spaces

There is also another (convenience) form of this method: 

match['a-z']      # synonymous to match('[a-z]')
match['\n']       # synonymous to match('[\n]')

@overload match(str)

@param str [String] character class to match (regexp syntax)
@return [Parslet::Atoms::Re] a parslet atom
# File lib/parslet.rb, line 142
def match(str=nil)
  return DelayedMatchConstructor.new unless str
  
  return Atoms::Re.new(str)
end
scope(&block) click to toggle source

Introduces a new capture scope. This means that all old captures stay accessible, but new values stored will only be available during the block given and the old values will be restored after the block.

Example: 

# :a will be available until the end of the block. Afterwards,
# :a from the outer scope will be available again, if such a thing
# exists.
scope { str('a').capture(:a) }

# File lib/parslet.rb, line 183
def scope(&block)
  Parslet::Atoms::Scope.new(block)
end
sequence(symbol) click to toggle source

Returns a placeholder for a tree transformation that will only match a sequence of elements. The symbol you specify will be the key for the matched sequence in the returned dictionary.

# This would match a body element that contains several declarations.
{ :body => sequence(:declarations) }

The above example would match :body => ['a', 'b'], but not :body => 'a'.

see {Parslet::Transform}

# File lib/parslet.rb, line 270
def sequence(symbol)
  Pattern::SequenceBind.new(symbol)
end
simple(symbol) click to toggle source

Returns a placeholder for a tree transformation that will only match simple elements. This matches everything that #sequence doesn't match.

# Matches a single header.
{ :header => simple(:header) }

see {Parslet::Transform}

# File lib/parslet.rb, line 284
def simple(symbol)
  Pattern::SimpleBind.new(symbol)
end
str(str) click to toggle source

Returns an atom matching the str given:

str('class')      # will match 'class'

@param str [String] string to match verbatim @return [Parslet::Atoms::Str] a parslet atom

# File lib/parslet.rb, line 156
def str(str)
  Atoms::Str.new(str)
end
subtree(symbol) click to toggle source

Returns a placeholder for tree transformation patterns that will match any kind of subtree. 

{ :expression => subtree(:exp) }

# File lib/parslet.rb, line 294
def subtree(symbol)
  Pattern::SubtreeBind.new(symbol)
end

Private Instance Methods

any() click to toggle source

Returns an atom matching any character. It acts like the '.' (dot) character in regular expressions.

any.parse('a')    # => 'a'

@return [Parslet::Atoms::Re] a parslet atom

# File lib/parslet.rb, line 168
def any
  Atoms::Re.new('.')
end
dynamic(&block) click to toggle source

Designates a piece of the parser as being dynamic. Dynamic parsers can either return a parser at runtime, which will be applied on the input, or return a result from a parse.

Dynamic parse pieces are never cached and can introduce performance abnormalitites - use sparingly where other constructs fail.

Example: 

# Parses either 'a' or 'b', depending on the weather
dynamic { rand() < 0.5 ? str('a') : str('b') }

# File lib/parslet.rb, line 199
def dynamic(&block)
  Parslet::Atoms::Dynamic.new(block)
end
exp(str) click to toggle source

A special kind of atom that allows embedding whole treetop expressions into parslet construction. 

# the same as str('a') >> str('b').maybe
exp(%Q("a" "b"?))

@param str [String] a treetop expression @return [Parslet::Atoms::Base] the corresponding parslet parser

# File lib/parslet.rb, line 253
def exp(str)
  Parslet::Expression.new(str).to_parslet
end
infix_expression(element, *operations, &reducer) click to toggle source

Returns a parslet atom that parses infix expressions. Operations are specified as a list of <atom, precedence, associativity> tuples, where atom is simply the parslet atom that matches an operator, precedence is a number and associativity is either :left or :right.

Higher precedence indicates that the operation should bind tighter than other operations with lower precedence. In common algebra, '+' has lower precedence than '*'. So you would have a precedence of 1 for '+' and a precedence of 2 for '*'. Only the order relation between these two counts, so any number would work.

Associativity is what decides what interpretation to take for strings that are ambiguous like '1 + 2 + 3'. If '+' is specified as left associative, the expression would be interpreted as '(1 + 2) + 3'. If right associativity is chosen, it would be interpreted as '1 + (2 + 3)'. Note that the hash trees output reflect that choice as well.

An optional block can be provided in order to manipulate the generated tree. The block will be called on each operator and passed 3 arguments: the left operand, the operator, and the right operand.

Examples:

infix_expression(integer, [add_op, 1, :left])
# would parse things like '1 + 2'

infix_expression(integer, [add_op, 1, :left]) { |l,o,r| { :plus => [l, r] } }
# would parse '1 + 2 + 3' as:
# { :plus => [1, { :plus => [2, 3] }] }

@param element [Parslet::Atoms::Base] elements that take the NUMBER position

in the expression

@param operations [Array<(Parslet::Atoms::Base, Integer, {:left, :right})>]

@see Parslet::Atoms::Infix

# File lib/parslet.rb, line 239
def infix_expression(element, *operations, &reducer)
  Parslet::Atoms::Infix.new(element, operations, &reducer)
end
match(str=nil) click to toggle source

Returns an atom matching a character class. All regular expressions can be used, as long as they match only a single character at a time. 

match('[ab]')     # will match either 'a' or 'b'
match('[\n\s]')   # will match newlines and spaces

There is also another (convenience) form of this method: 

match['a-z']      # synonymous to match('[a-z]')
match['\n']       # synonymous to match('[\n]')

@overload match(str)

@param str [String] character class to match (regexp syntax)
@return [Parslet::Atoms::Re] a parslet atom
# File lib/parslet.rb, line 142
def match(str=nil)
  return DelayedMatchConstructor.new unless str
  
  return Atoms::Re.new(str)
end
scope(&block) click to toggle source

Introduces a new capture scope. This means that all old captures stay accessible, but new values stored will only be available during the block given and the old values will be restored after the block.

Example: 

# :a will be available until the end of the block. Afterwards,
# :a from the outer scope will be available again, if such a thing
# exists.
scope { str('a').capture(:a) }

# File lib/parslet.rb, line 183
def scope(&block)
  Parslet::Atoms::Scope.new(block)
end
sequence(symbol) click to toggle source

Returns a placeholder for a tree transformation that will only match a sequence of elements. The symbol you specify will be the key for the matched sequence in the returned dictionary.

# This would match a body element that contains several declarations.
{ :body => sequence(:declarations) }

The above example would match :body => ['a', 'b'], but not :body => 'a'.

see {Parslet::Transform}

# File lib/parslet.rb, line 270
def sequence(symbol)
  Pattern::SequenceBind.new(symbol)
end
simple(symbol) click to toggle source

Returns a placeholder for a tree transformation that will only match simple elements. This matches everything that #sequence doesn't match.

# Matches a single header.
{ :header => simple(:header) }

see {Parslet::Transform}

# File lib/parslet.rb, line 284
def simple(symbol)
  Pattern::SimpleBind.new(symbol)
end
str(str) click to toggle source

Returns an atom matching the str given:

str('class')      # will match 'class'

@param str [String] string to match verbatim @return [Parslet::Atoms::Str] a parslet atom

# File lib/parslet.rb, line 156
def str(str)
  Atoms::Str.new(str)
end
subtree(symbol) click to toggle source

Returns a placeholder for tree transformation patterns that will match any kind of subtree. 

{ :expression => subtree(:exp) }

# File lib/parslet.rb, line 294
def subtree(symbol)
  Pattern::SubtreeBind.new(symbol)
end