module Robotstxt::CommonMethods
Protected Instance Methods
objectify_uri(uri)
click to toggle source
Convert a URI or a String into a URI
# File lib/robotstxt/common.rb, line 9 def objectify_uri(uri) if uri.is_a? String # URI.parse will explode when given a character that it thinks # shouldn't appear in uris. We thus escape them before passing the # string into the function. Unfortunately URI.escape does not respect # all characters that have meaning in HTTP (esp. #), so we are forced # to state exactly which characters we would like to escape. uri = URI.escape(uri, %r{[^!$#%&'()*+,\-./0-9:;=?@A-Z_a-z~]}) uri = URI.parse(uri) else uri end end