module RubberbandFlamethrower

Public Class Methods

auto(how_many_batches=3, per_batch=500, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet", id_overwrite="n") click to toggle source

Benchmarks a series of calls to the fire method (which inserts a batch of random data into Elastic Search) @param [Integer] how_many_batches - how many batches to run @param [Integer] per_batch - how many randomly generated data objects to insert @param [Integer] starting_id - starting id for randomly generated data objects, will increment from this number @param [String] server_url - url of the Elastic Search server @param [String] index - name of the Elastic Search index to insert data into @param [String] type - name of the Elastic Search type to insert data into

# File lib/rubberband_flamethrower.rb, line 27
def self.auto(how_many_batches=3, per_batch=500, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet", id_overwrite="n")
  require File.dirname(__FILE__)+"/rubberband_flamethrower/flamethrower.rb"
  flamethrower = Flamethrower.new
  puts "#{per_batch} documents inserted into Elastic Search per set"
  Benchmark.bm(8) do |x|
    how_many_batches.to_i.times do |i|
      x.report("set #{i+1} of #{how_many_batches}:")   { flamethrower.fire(per_batch, starting_id, server_url, index, type) }
      # increment the starting id  by the batch size on each loop unless "id_overwrite" is set to "y"
      # when it is set to "n" (which it is by default)
      # each batch will be writing new data with fresh IDs to the Elastic Search server
      # simulating a system where data is constantly being inserted and not updated
      # when it is set to "y" each batch will simulate overwriting existing data in the Elastic Search server
      # simulating a system where data is constantly being updated (after the initial batch)
      (starting_id = starting_id.to_i + per_batch.to_i) unless id_overwrite == "y"
    end
  end
end
fire(how_many=500, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet") click to toggle source

Benchmarks a call to the fire method (which inserts a batch of random data into Elastic Search) @param [Integer] how_many - how many randomly generated data objects to insert @param [Integer] starting_id - starting id for randomly generated data objects, will increment from this number @param [String] server_url - url of the Elastic Search server @param [String] index - name of the Elastic Search index to insert data into @param [String] type - name of the Elastic Search type to insert data into

# File lib/rubberband_flamethrower.rb, line 9
def self.fire(how_many=500, starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet")
  require File.dirname(__FILE__)+"/rubberband_flamethrower/flamethrower.rb"
  flamethrower = Flamethrower.new
  time = Benchmark.measure do
    flamethrower.fire(how_many, starting_id, server_url, index, type, 1)
  end
  puts "\nFinished Inserting #{how_many} documents into Elastic Search."
  puts "  user       system     total    real"
  puts time
end
generate_dataset(batch_size=500, filename="dataset", num_words=nil) click to toggle source
# File lib/rubberband_flamethrower.rb, line 45
def self.generate_dataset(batch_size=500, filename="dataset", num_words=nil)
  require File.dirname(__FILE__)+"/rubberband_flamethrower/data_generator.rb"
  data = DataGenerator.new
  data.generate_dataset(batch_size, filename, num_words)
end
help() click to toggle source

Displays help menu of the available help menu commands this is adapted from the file README.md things updated here should be updated there

# File lib/rubberband_flamethrower.rb, line 68
def self.help
  puts "\nRubberband Flamethrower is a gem for inserting faked data into an Elastic Search server"
  puts "and providing basic benchmarks. It creates and inserts fake data objects with three"
  puts "fields (message, username, and post_date) and reports timing information."
  
  puts "\nFlamethrower Commands Available:\n\n"
  puts "flamethrower fire #benchmark a batch insert of data to an elastic search server"
  puts "flamethrower auto #benchmark a series of batch inserts to an elastic search server"
  puts "flamethrower help #display this help message"
  
  puts "\nThe fire and auto commands can be configured by passing arguments."
  puts "The parameters accepted by fire and auto all have a default value if left blank."
  
  puts "\n\"fire\" parameters in order with their default values:"
  puts "(how_many=500, starting_id=1, server_url=http://localhost:9200, index=twitter, type=tweet)"

  puts "\n\"auto\" parameters in order with their default values:"
  puts "(how_many_batches=3, per_batch=500, starting_id=1, server_url=http://localhost:9200, index=twitter, type=tweet, id_overwrite=n)"
  
  puts "\nUsage Examples With Parameters:"
  puts "flamethrower fire 10000 #To Insert 10,000 instead of 500"
  puts "flamethrower fire 5000 5001 #To Insert 5,000 starting with the ID 5001"
  puts "flamethrower fire 2000 1 \"http://es.com:9200\" #Elastic Search Node located at http://es.com:9200"
  puts "flamethrower fire 500 1 \"http://localhost:9200\" \"facebook\" \"message\""
  puts "\t#Insert into an index named \"facebook\" instead of \"twitter\""
  puts "\t#with a type of \"message\" instead of \"tweet\""
  
  puts "\nThe id_overwrite parameter determines the ID strategy used for subsequent batches in the auto command."
  puts "When set to \"n\" (\"n\" is the default) each batch will be writing new data with unused IDs to the Elastic"
  puts "Search server, simulating a system where new data is constantly being inserted and not updated. When it is"
  puts "set to \"y\" each batch (after the first) will be overwriting existing data on the Elastic Search server,"
  puts "simulating a system where data is constantly being updated (after the initial batch)."
end
load_dataset(filename="dataset", starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet") click to toggle source
# File lib/rubberband_flamethrower.rb, line 51
def self.load_dataset(filename="dataset", starting_id=1, server_url="http://localhost:9200", index="twitter", type="tweet")
  require File.dirname(__FILE__)+"/rubberband_flamethrower/flamethrower.rb"
  flamethrower = Flamethrower.new
  time = Benchmark.measure do
    flamethrower.load_dataset(filename, starting_id, server_url, index, type, 1)
  end
  puts "\nFinished Inserting Dataset (#{filename}) into Elastic Search."
  puts "  user       system     total    real"
  puts time

end