class Krikri::AsyncUriGetter
Helper class for fetching multiple URLs concurrently.
@example to fetch 5 URLs in 5 threads
urls = ['http://example.com/one', 'http://example.com/two', 'http://example.com/three', 'http://example.com/four', 'http://example.com/five'] .map { |url| URI.parse(url) } getter = Krikri::AsyncUriGetter.new requests = urls.map do |url| getter.add_request(uri: url, opts: { follow_redirects: true }) end
At this point, 5 threads are launched to fetch the list of URLs. We can wait for them all to finish if we want to make sure we don't continue until all threads have terminated: `requests.map(&:join)`
Or simply access the responses and have our current thread block until they're available:
requests.each do |request| request.with_response do |response| if response.status == 200 puts "Response body: #{response.body}" else puts "Got return status: #{response.status}" end end end
Constants
- MAX_REDIRECTS
- Request
Public Class Methods
Create a new asynchronous URL fetcher.
@param opts [Hash] a hash of the supported options, which are: @option opts [Boolean] :follow_redirects Whether to follow HTTP 3xx
redirects.
@option opts [Integer] :max_redirects Number of redirects to follow before
giving up. (default: 10)
@option opts [Boolean] :inline_exceptions If true, pass exceptions as a
5xx response with the exception string in the body. (default: false)
# File lib/krikri/async_uri_getter.rb, line 56 def initialize(opts: {}) @default_opts = { max_redirects: MAX_REDIRECTS }.merge(opts) end
Public Instance Methods
Run a request (in a new thread) and return a promise-like object for the response.
@param uri [URI] the URI to be fetched @param headers [Hash<String, String>] HTTP headers to include with the
request
@param opts [Hash] options to override the ones provided when
AsyncUriGetter was initialized. All supported options from `#initialize` are available here as well.
# File lib/krikri/async_uri_getter.rb, line 70 def add_request(uri: nil, headers: {}, opts: {}) fail ArgumentError, "uri must be a URI; got: #{uri}" unless uri.is_a?(URI) Request.new(uri, headers, @default_opts.merge(opts)) end