class Arachni::Support::Database::Queue
Flat-file Queue
implementation
Behaves pretty much like a Ruby Queue
however it transparently serializes and saves its entries to the file-system under the OS’s temp directory after a specified {#max_buffer_size} (for in-memory entries) has been exceeded.
It’s pretty useful when you want to reduce memory footprint without having to refactor any code since it behaves just like a Ruby Queue
implementation.
@author Tasos “Zapotek” Laskos <tasos.laskos@arachni-scanner.com>
Constants
- DEFAULT_MAX_BUFFER_SIZE
Default {#max_buffer_size}.
Attributes
buffer[R]
@return [Array<Object>]
Objects stored in the memory buffer.
disk[R]
@return [Array<String>]
Paths to files stored to disk.
max_buffer_size[RW]
@return [Integer]
How many entries to keep in memory before starting to off-load to disk.
Public Class Methods
new( *args )
click to toggle source
@see Arachni::Database::Base#initialize
Calls superclass method
Arachni::Support::Database::Base::new
# File lib/arachni/support/database/queue.rb, line 41 def initialize( *args ) super( *args ) @disk = [] @buffer = [] @waiting = [] @mutex = Mutex.new end
Public Instance Methods
<<( obj )
click to toggle source
@param [Object] obj
Object to add to the queue.
# File lib/arachni/support/database/queue.rb, line 59 def <<( obj ) synchronize do if @buffer.size < max_buffer_size @buffer << obj else @disk << dump( obj ) end begin t = @waiting.shift t.wakeup if t rescue ThreadError retry end end end
buffer_size()
click to toggle source
# File lib/arachni/support/database/queue.rb, line 108 def buffer_size @buffer.size end
clear()
click to toggle source
Removes all objects from the queue.
# File lib/arachni/support/database/queue.rb, line 125 def clear synchronize do @buffer.clear while !@disk.empty? path = @disk.pop next if !path delete_file path end end end
disk_size()
click to toggle source
# File lib/arachni/support/database/queue.rb, line 112 def disk_size @disk.size end
empty?()
click to toggle source
@return [Bool]
`true` if the queue if empty, `false` otherwise.
# File lib/arachni/support/database/queue.rb, line 118 def empty? synchronize do internal_empty? end end
free_buffer_size()
click to toggle source
# File lib/arachni/support/database/queue.rb, line 104 def free_buffer_size max_buffer_size - buffer_size end
num_waiting()
click to toggle source
# File lib/arachni/support/database/queue.rb, line 137 def num_waiting @waiting.size end
pop( non_block = false )
click to toggle source
@return [Object]
Removes an object from the queue and returns it.
# File lib/arachni/support/database/queue.rb, line 80 def pop( non_block = false ) synchronize do loop do if internal_empty? raise ThreadError, 'queue empty' if non_block @waiting.push Thread.current @mutex.sleep else return @buffer.shift if !@buffer.empty? return load_and_delete_file( @disk.shift ) end end end end
size()
click to toggle source
@return [Integer]
Size of the queue, the number of objects it currently holds.
# File lib/arachni/support/database/queue.rb, line 99 def size buffer_size + disk_size end
Also aliased as: length
Private Instance Methods
internal_empty?()
click to toggle source
# File lib/arachni/support/database/queue.rb, line 143 def internal_empty? @buffer.empty? && @disk.empty? end
synchronize( &block )
click to toggle source
# File lib/arachni/support/database/queue.rb, line 147 def synchronize( &block ) @mutex.synchronize( &block ) end