class Checklinks::Worklist
A worklist is a list of jobs to do and possibly a pool of threads working on it
Public Class Methods
close(*worklists)
click to toggle source
Close a set of worklists
# File lib/checklinks/worklist.rb, line 65 def self.close(*worklists) worklists.map(&:close) end
collector()
click to toggle source
Create a worklist that collects values into an array and returns it when closed
# File lib/checklinks/worklist.rb, line 71 def self.collector worklist = Worklist.new worklist.process([]) do |value, collected| collected.push value end worklist end
new(*jobs)
click to toggle source
Create a worklist, possibly with a list of jobs to start with
# File lib/checklinks/worklist.rb, line 6 def initialize(*jobs) @state = nil @queue = Queue.new @active = Queue.new @processing = false push *jobs end
Public Instance Methods
close()
click to toggle source
Declare that you're not going to push any more jobs, and wait for the current jobs to be processed if you've started that
# File lib/checklinks/worklist.rb, line 55 def close @threads.size.times do @queue.push nil end @queue.close @threads.each &:join @state end
process(state=nil) { |job, state| ... }
click to toggle source
Process the jobs in the worklist in a background thread. You can pass a state object which is then passed into the block to process each job.
# File lib/checklinks/worklist.rb, line 24 def process(state=nil) @state = state process_concurrently(1) do |job| yield job, @state end end
process_concurrently(n) { |job| ... }
click to toggle source
Process the jobs in the worklist in multiple concurrent background threads
# File lib/checklinks/worklist.rb, line 32 def process_concurrently(n) raise if @threads @threads = n.times.map { Thread.new { until @queue.closed? && @queue.empty? job = @queue.pop next unless job @active.push :active yield job @active.pop end } } end
push(*jobs)
click to toggle source
Push more jobs onto the worklist
# File lib/checklinks/worklist.rb, line 15 def push(*jobs) jobs.each do |job| raise unless job @queue.push job end end
size()
click to toggle source
How many jobs are left to process
# File lib/checklinks/worklist.rb, line 48 def size # This is a a bit racy - we don't update these two queues atomically @queue.size + @active.size end