class TrickBag::Enumerables::BufferedEnumerable

Provides the plumbing for an enumerator that, in order to serve objects in each(), fetches them in chunks.

This class knows nothing about how to fetch anything; that behavior is provided by either subclassing this class, or calling .create_with_callables and passing a callable that knows how to do that.

Also supported is an optional fetch notification, a method or callable that will be called whenever a fetch is done. This can be useful to update counters, provide user feedback (e.g. a progress bar), etc.

This is useful, for example, in network requests, when multiple requests can be sent one immediately after another, and the responses can be collected as a group, for improved performance.

The fetch method and fetcher callable modify the instance's data array directly, to avoid the need to allow the callable to modify the data array reference, needlessly copying arrays, and to eliminate the need for garbage collecting many array objects (though the latter is rarely important).

Attributes

chunk_size[R]
fetch_notifier[RW]
fetcher[RW]

Public Class Methods

create_with_callables(chunk_size, fetcher, fetch_notifier = nil) click to toggle source

Creates an instance with callables for fetch and fetch notify behaviors. Callables are usually lambdas but can be any object responding to the method name `call`. @param chunk_size the maximum number of objects to be buffered @param fetcher callable to be called to fetch to fill the buffer @param fetch_notifier callable to be called to when a fetch is done

# File lib/trick_bag/enumerables/buffered_enumerable.rb, line 45
def self.create_with_callables(chunk_size, fetcher, fetch_notifier = nil)
  instance = self.new(chunk_size)
  instance.fetcher = fetcher
  instance.fetch_notifier = fetch_notifier
  instance
end
Also aliased as: create_with_lambdas
create_with_lambdas(chunk_size, fetcher, fetch_notifier = nil)
new(chunk_size) click to toggle source

@param fetcher callable to be called to fetch to fill the buffer @param chunk_size the maximum number of objects to be buffered @param fetch_notifier callable to be called to when a fetch is done

in case the caller wants to receive notification, update counters, etc.
It's passed the array of objects just fetched, whose size may be
less than chunk size.
# File lib/trick_bag/enumerables/buffered_enumerable.rb, line 62
def initialize(chunk_size)
  @chunk_size = chunk_size
  @data = []
  @chunk_count = 0
  @fetch_count = 0
  @yield_count = 0
end

Public Instance Methods

each() { |object_to_yield| ... } click to toggle source

Enumerable.each method. Note that, like all Enumerable.each methods, if you pass it without a block, it will return an enumerator, and any cleanup that would normally be done after each's loop has completed will not happen.

# File lib/trick_bag/enumerables/buffered_enumerable.rb, line 89
def each
  return to_enum unless block_given?

  last_chunk = false
  loop do
    if data.empty?
      return if last_chunk
      fetch
      @chunk_count += 1
      return if data.empty?
      self.fetch_count = self.fetch_count + data.size
      last_chunk = true if data.size < chunk_size
      fetch_notify
    end

    self.yield_count = self.yield_count + 1
    object_to_yield = data.shift
    yield(object_to_yield)
  end
end
fetch() click to toggle source

Unless you use self.create_with_callables to create your instance, you'll need to override this method in your subclass.

# File lib/trick_bag/enumerables/buffered_enumerable.rb, line 73
def fetch
  fetcher.(data, chunk_size) if fetcher
end
fetch_notify() click to toggle source

Unless you use self.create_with_callables to create your instance, you'll need to override this method in your subclass.

# File lib/trick_bag/enumerables/buffered_enumerable.rb, line 80
def fetch_notify
  fetch_notifier.(data) if fetch_notifier
end