class YARD::DocstringParser
Parses text and creates a {Docstring} object to represent documentation for a {CodeObjects::Base}. To create a new docstring, you should initialize the parser and call {#parse} followed by {#to_docstring}.
Subclassing Notes¶ ↑
The DocstringParser
can be subclassed and substituted during parsing by setting the {Docstring.default_parser} attribute with the name of the subclass. This allows developers to change the way docstrings are parsed, allowing for completely different docstring syntaxes.
@example Creating a Docstring
with a DocstringParser
DocstringParser.new.parse("text here").to_docstring
@example Creating a Custom DocstringParser
# Parses docstrings backwards! class ReverseDocstringParser def parse_content(content) super(content.reverse) end end # Set the parser as default when parsing YARD::Docstring.default_parser = ReverseDocstringParser
@see parse_content
@since 0.8.0
Constants
- META_MATCH
The regular expression to match the tag syntax
Attributes
@return [Array<Tags::Directive>] a list of directives identified
by the parser. This list will not be passed on to the Docstring object.
@return [Handlers::Base, nil] the handler parsing this
docstring. May be nil if this docstring parser is not initialized through
@return [Tags::Library] the tag library being used to
identify registered tags in the docstring.
@return [CodeObjects::Base, nil] the object associated with
the docstring being parsed. May be nil if the docstring is not attached to any object.
@return [String] the complete input string to the parser.
@return [CodeObjects::Base, nil] the object referenced by
the docstring being parsed. May be nil if the docstring doesn't refer to any object.
@return [OpenStruct] any arbitrary state to be passed between
tags during parsing. Mainly used by directives to coordinate behaviour (so that directives can be aware of other directives used in a docstring).
@return [String] the parsed text portion of the docstring,
with tags removed.
Public Class Methods
Creates a callback that is called after a docstring is successfully parsed. Use this method to perform sanity checks on a docstring’s tag data, or add any extra tags automatically to a docstring.
@yield [parser] a block to be called after a docstring is parsed @yieldparam [DocstringParser] parser the docstring parser object
with all directives and tags created.
@yieldreturn [void] @return [void]
# File lib/yard/docstring_parser.rb, line 266 def self.after_parse(&block) after_parse_callbacks << block end
@return [Array<Proc>] the {after_parse} callback proc objects
# File lib/yard/docstring_parser.rb, line 271 def self.after_parse_callbacks @after_parse_callbacks ||= [] end
Creates a new parser to parse docstring data
@param [Tags::Library] library a tag library for recognizing
tags.
# File lib/yard/docstring_parser.rb, line 81 def initialize(library = Tags::Library.instance) @text = "" @raw_text = "" @tags = [] @directives = [] @library = library @object = nil @reference = nil @handler = nil @state = OpenStruct.new end
Public Instance Methods
Creates a new directive using the registered {#library} @return [Tags::Directive] the directive object that is created
# File lib/yard/docstring_parser.rb, line 232 def create_directive(tag_name, tag_buf) if library.has_directive?(tag_name) dir = library.directive_create(tag_name, tag_buf, self) if dir.is_a?(Tags::Directive) @directives << dir dir end else log.warn "Unknown directive @!#{tag_name}" + (object ? " in file `#{object.file}` near line #{object.line}" : "") nil end rescue Tags::TagFormatError log.warn "Invalid directive format for @!#{tag_name}" + (object ? " in file `#{object.file}` near line #{object.line}" : "") nil end
Creates a {Tags::RefTag}
# File lib/yard/docstring_parser.rb, line 226 def create_ref_tag(tag_name, name, object_name) @tags << Tags::RefTagList.new(tag_name, P(object, object_name), name) end
Creates a tag from the {Tags::DefaultFactory tag factory}.
To add an already created tag object, append it to {#tags}.
@param [String] tag_name the tag name @param [String] tag_buf the text attached to the tag with newlines removed. @return [Tags::Tag, Tags::RefTag
] a tag
# File lib/yard/docstring_parser.rb, line 209 def create_tag(tag_name, tag_buf = '') if tag_buf =~ /\A\s*(?:(\S+)\s+)?\(\s*see\s+(\S+)\s*\)\s*\Z/ return create_ref_tag(tag_name, $1, $2) end if library.has_tag?(tag_name) @tags += [library.tag_create(tag_name, tag_buf)].flatten else log.warn "Unknown tag @#{tag_name}" + (object ? " in file `#{object.file}` near line #{object.line}" : "") end rescue Tags::TagFormatError log.warn "Invalid tag format for @#{tag_name}" + (object ? " in file `#{object.file}` near line #{object.line}" : "") end
Parses all content and returns itself.
@param [String] content the docstring text to parse @param [CodeObjects::Base] object the object that the docstring
is attached to. Will be passed to directives to act on this object.
@param [Handlers::Base, nil] handler the handler object that is
parsing this object. May be nil if this parser is not being called from a {Parser::SourceParser} context.
@return [self] the parser object. To get the docstring,
call {#to_docstring}.
@see to_docstring
# File lib/yard/docstring_parser.rb, line 113 def parse(content, object = nil, handler = nil) @object = object @handler = handler @reference, @raw_text = detect_reference(content) text = parse_content(@raw_text) @text = text.strip call_directives_after_parse post_process self end
Parses a given block of text.
@param [String] content the content to parse @note Subclasses can override this method to perform custom
parsing of content data.
# File lib/yard/docstring_parser.rb, line 129 def parse_content(content) content = content.split(/\r?\n/) if content.is_a?(String) return '' if !content || content.empty? docstring = String.new("") indent = content.first[/^\s*/].length last_indent = 0 orig_indent = 0 directive = false last_line = "" tag_name = nil tag_buf = [] (content + ['']).each_with_index do |line, index| indent = line[/^\s*/].length empty = (line =~ /^\s*$/ ? true : false) done = content.size == index if tag_name && (((indent < orig_indent && !empty) || done || (indent == 0 && !empty)) || (indent <= last_indent && line =~ META_MATCH)) buf = tag_buf.join("\n") if directive || tag_is_directive?(tag_name) directive = create_directive(tag_name, buf) if directive docstring << parse_content(directive.expanded_text).chomp end else create_tag(tag_name, buf) end tag_name = nil tag_buf = [] directive = false orig_indent = 0 end # Found a meta tag if line =~ META_MATCH directive = $1 tag_name = $2 tag_buf = [($3 || '')] elsif tag_name && indent >= orig_indent && !empty orig_indent = indent if orig_indent == 0 # Extra data added to the tag on the next line last_empty = last_line =~ /^[ \t]*$/ ? true : false tag_buf << '' if last_empty tag_buf << line.gsub(/^[ \t]{#{orig_indent}}/, '') elsif !tag_name # Regular docstring text docstring << line docstring << "\n" end last_indent = indent last_line = line end docstring end
Call post processing callbacks on parser. This is called implicitly by parser. Use this when manually configuring a {Docstring} object.
@return [void]
# File lib/yard/docstring_parser.rb, line 196 def post_process call_after_parse_callbacks end
Backward compatibility to detect old tags that should be specified as directives in 0.8 and onward.
# File lib/yard/docstring_parser.rb, line 252 def tag_is_directive?(tag_name) list = %w(attribute endgroup group macro method scope visibility) list.include?(tag_name) end
@return [Docstring] translates parsed text into
a Docstring object.
# File lib/yard/docstring_parser.rb, line 95 def to_docstring Docstring.new!(text, tags, object, raw_text, reference) end
Private Instance Methods
Calls all {after_parse} callbacks
# File lib/yard/docstring_parser.rb, line 324 def call_after_parse_callbacks self.class.after_parse_callbacks.each do |cb| cb.call(self) end end
Calls the {Tags::Directive#after_parse} callback on all the created directives.
# File lib/yard/docstring_parser.rb, line 319 def call_directives_after_parse directives.each(&:after_parse) end
# File lib/yard/docstring_parser.rb, line 305 def detect_reference(content) if content =~ /\A\s*\(see (\S+)\s*\)(?:\s|$)/ path = $1 extra = $' [CodeObjects::Proxy.new(namespace, path), extra] else [nil, content] end end
# File lib/yard/docstring_parser.rb, line 301 def namespace object && object.namespace end