class Vault::PersistentHTTP
Constants
- VERSION
-
The version of PersistentHTTP you are using
Attributes
An SSL certificate authority. Setting this will set verify_mode
to VERIFY_PEER.
A directory of SSL certificates to be used as certificate authorities. Setting this will set verify_mode
to VERIFY_PEER.
This client’s OpenSSL::X509::Certificate
An SSL certificate store. Setting this will override the default certificate store. See verify_mode
for more information.
This client’s OpenSSL::X509::Certificate
The ciphers allowed for SSL connections
Sends debug_output
to this IO via Net::HTTP#set_debug_output.
Never use this method in production code, it causes a serious security hole.
Headers that are added to every request using Net::HTTP#add_field
Maps host:port to an HTTP version. This allows us to enable version specific features.
Maximum time an unused connection can remain idle before being automatically closed.
The value sent in the Keep-Alive header. Defaults
to 30. Not needed for HTTP/1.1 servers.
This may not work correctly for HTTP/1.0 servers
This method may be removed in a future version as RFC 2616 does not require this header.
This client’s SSL private key
Maximum number of requests on a connection before it is considered expired and automatically closed.
Minimum SSL version to use.
A name for this connection. Allows you to keep your connections apart from everybody else’s.
List of host suffixes which will not be proxied
Seconds to wait until a connection is opened. See Net::HTTP#open_timeout
Headers that are added to every request using Net::HTTP#[]=
This client’s SSL private key
The URL through which requests will be proxied
Seconds to wait until reading one block. See Net::HTTP#read_timeout
Enable retries of non-idempotent requests that change data (e.g. POST requests) when the server has disconnected.
This will in the worst case lead to multiple requests with the same data, but it may be useful for some applications. Take care when enabling this option to ensure it is safe to POST or perform other non-idempotent requests to the server.
By default SSL sessions are reused to avoid extra SSL handshakes. Set this to false if you have problems communicating with an HTTPS server like:
SSL_connect [...] read finished A: unexpected message (OpenSSL::SSL::SSLError)
An array of options for Socket#setsockopt.
By default the TCP_NODELAY option is set on sockets.
To set additional options append them to this array:
http.socket_options << [Socket::SOL_SOCKET, Socket::SO_KEEPALIVE, 1]
SSL session lifetime
Sets the depth of SSL certificate verification
HTTPS verify mode. Defaults
to OpenSSL::SSL::VERIFY_PEER which verifies the server certificate.
If no ca_file
, ca_path
or cert_store
is set the default system certificate store is used.
You can use verify_mode
to override any default values.
Public Class Methods
Source
# File lib/vault/persistent.rb, line 248 def self.detect_idle_timeout uri, max = 10 uri = URI uri unless URI::Generic === uri uri += '/' req = Net::HTTP::Head.new uri.request_uri http = new 'net-http-persistent detect_idle_timeout' http.connection_for uri do |connection| sleep_time = 0 http = connection.http loop do response = http.request req $stderr.puts "HEAD #{uri} => #{response.code}" if $DEBUG unless Net::HTTPOK === response then raise Error, "bad response code #{response.code} detecting idle timeout" end break if sleep_time >= max sleep_time += 1 $stderr.puts "sleeping #{sleep_time}" if $DEBUG sleep sleep_time end end rescue # ignore StandardErrors, we've probably found the idle timeout. ensure return sleep_time unless $! end
Use this method to detect the idle timeout of the host at uri
. The value returned can be used to configure idle_timeout
. max
controls the maximum idle timeout to detect.
After
Idle timeout detection is performed by creating a connection then performing a HEAD request in a loop until the connection terminates waiting one additional second per loop.
NOTE: This may not work on ruby > 1.9.
Source
# File lib/vault/persistent.rb, line 503 def initialize name=nil, proxy=nil, pool_size=Vault::Defaults::DEFAULT_POOL_SIZE, pool_timeout=Vault::Defaults::DEFAULT_POOL_TIMEOUT @name = name @debug_output = nil @proxy_uri = nil @no_proxy = [] @headers = {} @override_headers = {} @http_versions = {} @keep_alive = 30 @open_timeout = nil @read_timeout = nil @idle_timeout = 5 @max_requests = nil @socket_options = [] @ssl_generation = 0 # incremented when SSL session variables change @socket_options << [Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1] if Socket.const_defined? :TCP_NODELAY @pool = PersistentHTTP::Pool.new size: pool_size, timeout: pool_timeout do |http_args| PersistentHTTP::Connection.new Net::HTTP, http_args, @ssl_generation end @certificate = nil @ca_file = nil @ca_path = nil @ciphers = nil @min_version = nil @private_key = nil @ssl_timeout = nil @verify_callback = nil @verify_depth = nil @verify_mode = nil @cert_store = nil @generation = 0 # incremented when proxy URI changes if HAVE_OPENSSL then @verify_mode = OpenSSL::SSL::VERIFY_PEER @reuse_ssl_sessions = OpenSSL::SSL.const_defined? :Session end @retry_change_requests = false self.proxy = proxy if proxy end
Creates a new PersistentHTTP.
Set name
to keep your connections apart from everybody else’s. Not required currently, but highly recommended. Your library name should be good enough. This parameter will be required in a future version.
proxy
may be set to a URI::HTTP or :ENV to pick up proxy options from the environment. See proxy_from_env
for details.
In order to use a URI for the proxy you may need to do some extra work beyond URI parsing if the proxy requires a password:
proxy = URI 'http://proxy.example' proxy.user = 'AzureDiamond' proxy.password = 'hunter2'
Set pool_size
to limit the maximum number of connections allowed. Defaults
to 1/4 the number of allowed file handles. You can have no more than this many threads with active HTTP transactions.
Public Instance Methods
Source
# File lib/vault/persistent.rb, line 566 def ca_file= file @ca_file = file reconnect_ssl end
Sets the SSL certificate authority file.
Source
# File lib/vault/persistent.rb, line 575 def ca_path= path @ca_path = path reconnect_ssl end
Sets the SSL certificate authority path.
Source
# File lib/vault/persistent.rb, line 735 def can_retry? req @retry_change_requests && !idempotent?(req) end
Is the request req
idempotent or is retry_change_requests
allowed.
Source
# File lib/vault/persistent.rb, line 585 def cert_store= store @cert_store = store reconnect_ssl end
Overrides the default SSL certificate store used for verifying connections.
Source
# File lib/vault/persistent.rb, line 554 def certificate= certificate @certificate = certificate reconnect_ssl end
Sets this client’s OpenSSL::X509::Certificate
Source
# File lib/vault/persistent.rb, line 594 def ciphers= ciphers @ciphers = ciphers reconnect_ssl end
The ciphers allowed for SSL connections
Source
# File lib/vault/persistent.rb, line 603 def connection_for uri use_ssl = uri.scheme.downcase == 'https' net_http_args = [uri.hostname, uri.port] net_http_args.concat @proxy_args if @proxy_uri and not proxy_bypass? uri.hostname, uri.port connection = @pool.checkout net_http_args http = connection.http connection.ressl @ssl_generation if connection.ssl_generation != @ssl_generation if not http.started? then ssl http if use_ssl start http elsif expired? connection then reset connection end http.read_timeout = @read_timeout if @read_timeout http.keep_alive_timeout = @idle_timeout if @idle_timeout return yield connection rescue Errno::ECONNREFUSED address = http.proxy_address || http.address port = http.proxy_port || http.port raise Error, "connection refused: #{address}:#{port}" rescue Errno::EHOSTDOWN address = http.proxy_address || http.address port = http.proxy_port || http.port raise Error, "host down: #{address}:#{port}" ensure # Only perform checkin if we successfully checked a connection out if connection @pool.checkin net_http_args end end
Creates a new connection for uri
Source
# File lib/vault/persistent.rb, line 650 def error_message connection connection.requests -= 1 # fixup age = Time.now - connection.last_use "after #{connection.requests} requests on #{connection.http.object_id}, " \ "last used #{age} seconds ago" end
Returns an error message containing the number of requests performed on this connection
Source
# File lib/vault/persistent.rb, line 662 def escape str CGI.escape str if str end
URI::escape wrapper
Source
# File lib/vault/persistent.rb, line 678 def expired? connection return true if @max_requests && connection.requests >= @max_requests return false unless @idle_timeout return true if @idle_timeout.zero? Time.now - connection.last_use > @idle_timeout end
Returns true if the connection should be reset due to an idle timeout, or maximum request count, false otherwise.
Source
# File lib/vault/persistent.rb, line 707 def finish connection connection.finish connection.http.instance_variable_set :@ssl_session, nil unless @reuse_ssl_sessions end
Finishes the Net::HTTP connection
Source
# File lib/vault/persistent.rb, line 717 def http_version uri @http_versions["#{uri.hostname}:#{uri.port}"] end
Returns the HTTP protocol version for uri
Source
# File lib/vault/persistent.rb, line 724 def idempotent? req case req when Net::HTTP::Delete, Net::HTTP::Get, Net::HTTP::Head, Net::HTTP::Options, Net::HTTP::Put, Net::HTTP::Trace then true end end
Is req
idempotent according to RFC 2616?
Source
# File lib/vault/persistent.rb, line 1111 def min_version= min_version @min_version = min_version reconnect_ssl end
Minimum SSL version to use
Source
# File lib/vault/persistent.rb, line 742 def normalize_uri uri (uri =~ /^https?:/) ? uri : "http://#{uri}" end
Adds “http://” to the String uri
if it is missing.
Source
# File lib/vault/persistent.rb, line 757 def pipeline uri, requests, &block # :yields: responses connection_for uri do |connection| connection.http.pipeline requests, &block end end
Pipelines requests
to the HTTP server at uri
yielding responses if a block is given. Returns all responses recieved.
See Net::HTTP::Pipeline for further details.
Only if net-http-pipeline
was required before net-http-persistent
pipeline
will be present.
Source
# File lib/vault/persistent.rb, line 766 def private_key= key @private_key = key reconnect_ssl end
Sets this client’s SSL private key
Source
# File lib/vault/persistent.rb, line 789 def proxy= proxy @proxy_uri = case proxy when :ENV then proxy_from_env when URI::HTTP then proxy when nil then # ignore else raise ArgumentError, 'proxy must be :ENV or a URI::HTTP' end @no_proxy.clear if @proxy_uri then @proxy_args = [ @proxy_uri.hostname, @proxy_uri.port, unescape(@proxy_uri.user), unescape(@proxy_uri.password), ] @proxy_connection_id = [nil, *@proxy_args].join ':' if @proxy_uri.query then @no_proxy = CGI.parse(@proxy_uri.query)['no_proxy'].join(',').downcase.split(',').map { |x| x.strip }.reject { |x| x.empty? } end end reconnect reconnect_ssl end
Sets the proxy server. The proxy
may be the URI of the proxy server, the symbol :ENV
which will read the proxy from the environment or nil to disable use of a proxy. See proxy_from_env
for details on setting the proxy from the environment.
If the proxy URI is set after requests have been made, the next request will shut-down and re-open all connections.
The no_proxy
query parameter can be used to specify hosts which shouldn’t be reached via proxy; if set it should be a comma separated list of hostname suffixes, optionally with :port
appended, for example example.com,some.host:8080
.
Source
# File lib/vault/persistent.rb, line 863 def proxy_bypass? host, port host = host.downcase host_port = [host, port].join ':' @no_proxy.each do |name| return true if host[-name.length, name.length] == name or host_port[-name.length, name.length] == name end false end
Returns true when proxy should by bypassed for host.
Source
# File lib/vault/persistent.rb, line 836 def proxy_from_env env_proxy = ENV['http_proxy'] || ENV['HTTP_PROXY'] return nil if env_proxy.nil? or env_proxy.empty? uri = URI normalize_uri env_proxy env_no_proxy = ENV['no_proxy'] || ENV['NO_PROXY'] # '*' is special case for always bypass return nil if env_no_proxy == '*' if env_no_proxy then uri.query = "no_proxy=#{escape(env_no_proxy)}" end unless uri.user or uri.password then uri.user = escape ENV['http_proxy_user'] || ENV['HTTP_PROXY_USER'] uri.password = escape ENV['http_proxy_pass'] || ENV['HTTP_PROXY_PASS'] end uri end
Creates a URI for an HTTP proxy server from ENV variables.
If HTTP_PROXY
is set a proxy will be returned.
If HTTP_PROXY_USER
or HTTP_PROXY_PASS
are set the URI is given the indicated user and password unless HTTP_PROXY contains either of these in the URI.
The NO_PROXY
ENV variable can be used to specify hosts which shouldn’t be reached via proxy; if set it should be a comma separated list of hostname suffixes, optionally with :port
appended, for example example.com,some.host:8080
. When set to *
no proxy will be returned.
For Windows users, lowercase ENV variables are preferred over uppercase ENV variables.
Source
# File lib/vault/persistent.rb, line 878 def reconnect @generation += 1 end
Forces reconnection of HTTP connections.
Source
# File lib/vault/persistent.rb, line 885 def reconnect_ssl @ssl_generation += 1 end
Forces reconnection of SSL connections.
Source
# File lib/vault/persistent.rb, line 920 def request uri, req = nil, &block retried = false bad_response = false uri = URI uri req = request_setup req || uri response = nil connection_for uri do |connection| http = connection.http begin connection.requests += 1 response = http.request req, &block if req.connection_close? or (response.http_version <= '1.0' and not response.connection_keep_alive?) or response.connection_close? then finish connection end rescue Net::HTTPBadResponse => e message = error_message connection finish connection raise Error, "too many bad responses #{message}" if bad_response or not can_retry? req bad_response = true retry rescue *RETRIED_EXCEPTIONS => e request_failed e, req, connection if retried or not can_retry? req reset connection retried = true retry rescue Errno::EINVAL, Errno::ETIMEDOUT => e # not retried on ruby 2 request_failed e, req, connection if retried or not can_retry? req reset connection retried = true retry rescue Exception => e finish connection raise ensure connection.last_use = Time.now end end @http_versions["#{uri.hostname}:#{uri.port}"] ||= response.http_version response end
Makes a request on uri
. If req
is nil a Net::HTTP::Get is performed against uri
.
If a block is passed request
behaves like Net::HTTP#request (the body of the response will not have been read).
req
must be a Net::HTTPRequest subclass (see Net::HTTP for a list).
If there is an error and the request is idempotent according to RFC 2616 it will be retried automatically.
Source
# File lib/vault/persistent.rb, line 892 def reset connection http = connection.http finish connection start http rescue Errno::ECONNREFUSED e = Error.new "connection refused: #{http.address}:#{http.port}" e.set_backtrace $@ raise e rescue Errno::EHOSTDOWN e = Error.new "host down: #{http.address}:#{http.port}" e.set_backtrace $@ raise e end
Finishes then restarts the Net::HTTP connection
Source
# File lib/vault/persistent.rb, line 1033 def shutdown @pool.available.shutdown do |http| http.finish end end
Shuts down all connections
NOTE: Calling shutdown for can be dangerous!
If any thread is still using a connection it may cause an error! Call shutdown
when you are completely done making requests!
Source
# File lib/vault/persistent.rb, line 1042 def ssl connection connection.use_ssl = true connection.ciphers = @ciphers if @ciphers if @min_version if connection.respond_to? :min_version= connection.min_version = @min_version else connection.ssl_version = @min_version end end connection.ssl_timeout = @ssl_timeout if @ssl_timeout connection.verify_depth = @verify_depth connection.verify_mode = @verify_mode if OpenSSL::SSL::VERIFY_PEER == OpenSSL::SSL::VERIFY_NONE and not Object.const_defined?(:I_KNOW_THAT_OPENSSL_VERIFY_PEER_EQUALS_VERIFY_NONE_IS_WRONG) then warn <<-WARNING !!!SECURITY WARNING!!! The SSL HTTP connection to: #{connection.address}:#{connection.port} !!!MAY NOT BE VERIFIED!!! On your platform your OpenSSL implementation is broken. There is no difference between the values of VERIFY_NONE and VERIFY_PEER. This means that attempting to verify the security of SSL connections may not work. This exposes you to man-in-the-middle exploits, snooping on the contents of your connection and other dangers to the security of your data. To disable this warning define the following constant at top-level in your application: I_KNOW_THAT_OPENSSL_VERIFY_PEER_EQUALS_VERIFY_NONE_IS_WRONG = nil WARNING end connection.ca_file = @ca_file if @ca_file connection.ca_path = @ca_path if @ca_path if @ca_file or @ca_path then connection.verify_callback = @verify_callback if @verify_callback end if @certificate and @private_key then connection.cert = @certificate connection.key = @private_key end connection.cert_store = if @cert_store then @cert_store else store = OpenSSL::X509::Store.new store.set_default_paths store end end
Enables SSL on connection
Source
# File lib/vault/persistent.rb, line 1120 def ssl_timeout= ssl_timeout @ssl_timeout = ssl_timeout reconnect_ssl end
SSL session lifetime
Source
# File lib/vault/persistent.rb, line 689 def start http http.set_debug_output @debug_output if @debug_output http.open_timeout = @open_timeout if @open_timeout http.start socket = http.instance_variable_get :@socket if socket then # for fakeweb @socket_options.each do |option| socket.io.setsockopt(*option) end end end
Starts the Net::HTTP connection
Source
# File lib/vault/persistent.rb, line 669 def unescape str CGI.unescape str if str end
URI::unescape wrapper
Source
# File lib/vault/persistent.rb, line 1151 def verify_callback= callback @verify_callback = callback reconnect_ssl end
SSL verification callback.
Source
# File lib/vault/persistent.rb, line 1129 def verify_depth= verify_depth @verify_depth = verify_depth reconnect_ssl end
Sets the depth of SSL certificate verification
Source
# File lib/vault/persistent.rb, line 1142 def verify_mode= verify_mode @verify_mode = verify_mode reconnect_ssl end
Sets the HTTPS verify mode. Defaults
to OpenSSL::SSL::VERIFY_PEER.
Setting this to VERIFY_NONE is a VERY BAD IDEA and should NEVER be used. Securely transfer the correct certificate and update the default certificate store or set the ca file instead.