class ProxyFetcher::Configuration
ProxyFetcher
configuration. Stores all the options for dealing with HTTP requests, adapters, custom classes.
Constants
- DEFAULT_ADAPTER
HTML parser adapter name.
Default is Nokogiri, but can be changed in
ProxyFetcher.config
.- DEFAULT_USER_AGENT
User-Agent string that will be used by the
ProxyFetcher
HTTP client (to send requests via proxy) and to fetch proxy lists from the sources.Default is Google Chrome 60, but can be changed in
ProxyFetcher.config
.
Attributes
@!attribute [r] adapter
@return [#to_s] HTML parser adapter
@!attribute client_timeout
@return [Integer] HTTP request timeout (connect / open) for [ProxyFetcher::Client]
@!attribute [r] http_client
@return [Object] HTTP client class
@!attribute [r] logger
@return [Logger] Logger object
@!attribute pool_size
@return [Integer] proxy validator pool size (max number of threads)
@!attribute [r] providers
@return [Array<String, Symbol>] proxy providers list to be used
@!attribute provider_proxies_load_timeout
@return [Integer] HTTP request timeout (connect / open) for loading of proxies list by provider
@!attribute [r] providers
@return [Array<String, Symbol>] proxy providers list to be used
@!attribute proxy_validation_timeout
@return [Integer] HTTP request timeout (connect / open) for proxy validation with [ProxyFetcher::ProxyValidator]
@!attribute [r] proxy_validator
@return [Object] proxy validator class
@!attribute client_timeout
@return [Integer] HTTP request timeout (connect / open) for [ProxyFetcher::Client]
@!attribute client_timeout
@return [Integer] HTTP request timeout (connect / open) for [ProxyFetcher::Client]
@!attribute user_agent
@return [String] User-Agent string
Public Class Methods
Initialize ProxyFetcher
configuration with default options.
@return [ProxyFetcher::Configuration]
ProxyFetcher gem configuration object
# File lib/proxy_fetcher/configuration.rb, line 111 def initialize reset! end
Registry for handling proxy providers.
@return [ProxyFetcher::ProvidersRegistry]
providers registry
# File lib/proxy_fetcher/configuration.rb, line 79 def providers_registry @providers_registry ||= ProvidersRegistry.new end
Register new proxy provider. Requires provider name and class that will process proxy list.
@param name [String, Symbol]
name of the provider
@param klass [Class]
Class that will fetch and process proxy list
# File lib/proxy_fetcher/configuration.rb, line 92 def register_provider(name, klass) providers_registry.register(name, klass) end
Returns registered providers names.
@return [Array<String, Symbol>]
registered providers names
# File lib/proxy_fetcher/configuration.rb, line 101 def registered_providers providers_registry.providers.keys end
Public Instance Methods
# File lib/proxy_fetcher/configuration.rb, line 130 def adapter=(value) remove_instance_variable(:@adapter_class) if defined?(@adapter_class) @adapter = value end
# File lib/proxy_fetcher/configuration.rb, line 135 def adapter_class self.class.instance_variable_get(:@__adapter_lock__).synchronize do return @adapter_class if defined?(@adapter_class) @adapter_class = ProxyFetcher::Document::Adapters.lookup(adapter) @adapter_class.setup! @adapter_class end end
Setups HTTP client class that will be used to fetch proxy lists. Validates class for the required methods to be defined.
@param klass [Class]
HTTP client class
# File lib/proxy_fetcher/configuration.rb, line 163 def http_client=(klass) @http_client = setup_custom_class(klass, required_methods: :fetch) end
Setups collection of providers that will be used to fetch proxies.
@param value [String, Symbol, Array<String, Symbol>]
provider names
# File lib/proxy_fetcher/configuration.rb, line 150 def providers=(value) @providers = Array(value) end
Setups class that will be used to validate proxy lists. Validates class for the required methods to be defined.
@param klass [Class]
Proxy validator class
# File lib/proxy_fetcher/configuration.rb, line 173 def proxy_validator=(klass) @proxy_validator = setup_custom_class(klass, required_methods: :connectable?) end
Sets default configuration options
# File lib/proxy_fetcher/configuration.rb, line 116 def reset! @logger = Logger.new($stdout) @user_agent = DEFAULT_USER_AGENT @pool_size = 10 @client_timeout = 3 @provider_proxies_load_timeout = 30 @proxy_validation_timeout = 3 @http_client = HTTPClient @proxy_validator = ProxyValidator self.providers = self.class.registered_providers end
Private Instance Methods
Checks if custom class has some required class methods
# File lib/proxy_fetcher/configuration.rb, line 180 def setup_custom_class(klass, required_methods: []) unless klass.respond_to?(*required_methods) raise ProxyFetcher::Exceptions::WrongCustomClass.new(klass, required_methods) end klass end