class Checklinks::Worklist

A worklist is a list of jobs to do and possibly a pool of threads working on it

Public Class Methods

close(*worklists) click to toggle source

Close a set of worklists

# File lib/checklinks/worklist.rb, line 65
def self.close(*worklists)
  worklists.map(&:close)
end
collector() click to toggle source

Create a worklist that collects values into an array and returns it when closed

# File lib/checklinks/worklist.rb, line 71
def self.collector
  worklist = Worklist.new
  worklist.process([]) do |value, collected|
    collected.push value
  end
  worklist
end
new(*jobs) click to toggle source

Create a worklist, possibly with a list of jobs to start with

# File lib/checklinks/worklist.rb, line 6
def initialize(*jobs)
  @state = nil
  @queue = Queue.new
  @active = Queue.new
  @processing = false
  push *jobs
end

Public Instance Methods

close() click to toggle source

Declare that you're not going to push any more jobs, and wait for the current jobs to be processed if you've started that

# File lib/checklinks/worklist.rb, line 55
def close
  @threads.size.times do
    @queue.push nil
  end
  @queue.close
  @threads.each &:join
  @state
end
process(state=nil) { |job, state| ... } click to toggle source

Process the jobs in the worklist in a background thread. You can pass a state object which is then passed into the block to process each job.

# File lib/checklinks/worklist.rb, line 24
def process(state=nil)
  @state = state
  process_concurrently(1) do |job|
    yield job, @state
  end
end
process_concurrently(n) { |job| ... } click to toggle source

Process the jobs in the worklist in multiple concurrent background threads

# File lib/checklinks/worklist.rb, line 32
def process_concurrently(n)
  raise if @threads
  @threads = n.times.map {
    Thread.new {
      until @queue.closed? && @queue.empty?
        job = @queue.pop
        next unless job
        @active.push :active
        yield job
        @active.pop
      end
    }
  }
end
push(*jobs) click to toggle source

Push more jobs onto the worklist

# File lib/checklinks/worklist.rb, line 15
def push(*jobs)
  jobs.each do |job|
    raise unless job
    @queue.push job
  end
end
size() click to toggle source

How many jobs are left to process

# File lib/checklinks/worklist.rb, line 48
def size
  # This is a a bit racy - we don't update these two queues atomically
  @queue.size + @active.size
end