class ProxyFetcher::Providers::ProxyList

ProxyList provider class.

Public Instance Methods

provider_url() click to toggle source

Provider URL to fetch proxy list

# File lib/proxy_fetcher/providers/proxy_list.rb, line 10
def provider_url
  "https://proxy-list.org/english/index.php"
end
to_proxy(html_node) click to toggle source

Converts HTML node (entry of N tags) to ProxyFetcher::Proxy object.

@param html_node [Object]

HTML node from the <code>ProxyFetcher::Document</code> DOM model.

@return [ProxyFetcher::Proxy]

Proxy object
# File lib/proxy_fetcher/providers/proxy_list.rb, line 27
def to_proxy(html_node)
  ProxyFetcher::Proxy.new.tap do |proxy|
    uri = parse_proxy_uri(html_node)
    proxy.addr = uri.host
    proxy.port = uri.port

    proxy.type = html_node.content_at("li[2]")
    proxy.anonymity = html_node.content_at("li[4]")
    proxy.country = html_node.find("li[5]//span[@class='country']").attr("title")
  end
end
xpath() click to toggle source
# File lib/proxy_fetcher/providers/proxy_list.rb, line 14
def xpath
  '//div[@class="table-wrap"]/div[@class="table"]/ul'
end

Private Instance Methods

parse_proxy_uri(html_node) click to toggle source

Parses HTML node to extract URI object with proxy host and port.

@param html_node [Object]

HTML node from the <code>ProxyFetcher::Document</code> DOM model.

@return [URI]

URI object
# File lib/proxy_fetcher/providers/proxy_list.rb, line 49
def parse_proxy_uri(html_node)
  full_addr = ::Base64.decode64(html_node.at_css("li script").html.match(/'(.+)'/)[1])
  URI.parse("http://#{full_addr}")
end