class ProxyFetcher::Providers::ProxyscrapeHTTP
FreeProxyList
provider class.
Public Instance Methods
load_document(url, filters = {})
click to toggle source
Loads provider HTML and parses it with internal document object.
@param url [String]
URL to fetch
@param filters [Hash]
filters for proxy provider
@return [Array]
Collection of extracted proxies with ports
# File lib/proxy_fetcher/providers/proxyscrape_http.rb, line 25 def load_document(url, filters = {}) html = load_html(url, filters) CSV.parse(html, col_sep: "\t").map(&:first) end
load_proxy_list(filters = {})
click to toggle source
Fetches HTML content by sending HTTP request to the provider URL and parses the txt document to return all the proxy entries (ip addresses and ports).
@return [Array]
Collection of extracted proxies with ports
# File lib/proxy_fetcher/providers/proxyscrape_http.rb, line 38 def load_proxy_list(filters = {}) load_document(provider_url, filters) end
provider_url()
click to toggle source
Provider URL to fetch proxy list
# File lib/proxy_fetcher/providers/proxyscrape_http.rb, line 10 def provider_url "https://api.proxyscrape.com/v2/?request=getproxies&protocol=http" end
to_proxy(node)
click to toggle source
Converts String to ProxyFetcher::Proxy
object.
@param node [String]
String
@return [ProxyFetcher::Proxy]
Proxy object
# File lib/proxy_fetcher/providers/proxyscrape_http.rb, line 50 def to_proxy(node) addr, port = node.split(":") ProxyFetcher::Proxy.new.tap do |proxy| proxy.addr = addr proxy.port = Integer(port) proxy.country = "Unknown" proxy.anonymity = "Unknown" proxy.type = ProxyFetcher::Proxy::HTTP end end