Fire & Forget

Fire & Forget is a simple, framework agnostic, background task launcher for Ruby web apps.

What it does:

What it doesn’t:

Quick Start

gem install fire_and_forget

FAF works by running a simple UNIX socket server so before we want to use it we have to start this server.

$ fire_forget
FAF process 16235 listening on /tmp/fire_and_forget.sock ...

To change the path of the socket file use the ‘-s’ parameter:

$ fire_forget -s /path/to/socket
FAF process 16240 listening on /path/to/socket ...

If you do this you will need to set the web-app to use the right values:

FireAndForget.socket = "/path/to/socket"

Now inside our Ruby app:

require 'rubygems'
require 'fire_and_forget'

# First set register our task by giving it a name and a path to a script file
FireAndForget.add_task(:long_running_task, "/path/to/script")
# => #<FireAndForget::TaskDescription @name=:long_running_task ...>

# when we want to call the script we simple call #fire passing the name of the
# task and the options we want to pass
FireAndForget.fire(:long_running_task, {:param3 => "value3"})
# => "OK"

# Or, use the name as a method:
FireAndForget.long_running_task({:param3 => "value3"})
# => "OK"

This will result in the following command being exec’d in an independent process:

/path/to/script --param3="value3"

It up to the script to parse and deal with the command line options.

The options for the add_task call are as follows:

task_name: A symbol giving the (unique) name for this task
path_to_binary: The path on disk of the executable to run when this task is launched
niceness: an integer in the range 0-20 giving the 'niceness' of the task process.
default_params: Command line parameters to pass to each invocation of this task
env: settings to add to this task's environment

The higher the nice value, the lower the priority of the task. Setting a high ‘nice’ allows you to launch intensive background tasks without affecting the performance of your front line HTTP services.

For example:

FireAndForget.add_task(:long_running_task, "/path/to/script", 12,
  {:param1 => "value1", :param2 => "value2", :param3 => "value3"},
  {"ENVIRONMENT_SETTING" => "Something"}
)

this will add some default parameters to all invocations of the task, so now the call

FireAndForget.fire(:long_running_task, {:param3 => "newvalue3"})

will issue the following command in the background

/path/to/script --param1="value1" --param2="value2" --param3="newvalue3"

as well as setting it’s nice value to 12 and updating ENV with the extra parameter “ENVIRONMENT_SETTING”.

Parameter values are encoded in a format compatible with Thor:

Arrays are encoded as –array=1 2 3 Hashes as –hash=key1:value1 key2:value2

Again, it is up to the script to decode these values.

Interprocess communication is relatively easy. Take the following as the source of the script “/path/to/script”

#!/usr/bin/env ruby

require 'rubygems'
require 'fire_and_forget'

# this will mix in the F&F status methods
# the F&F configuration (socket and task-name) is calculated automatically from ENV parameters
include FireAndForget::Daemon

30.times do |i|
  # update our status. What you put here is up to you, but it should be a String
  set_task_status("#{i+1} of 30")
  sleep(1)
end

Now in the client all we have to do to get the status for our task is:

FireAndForget.get_status(:long_running_task)
# => "18 of 30"

If we decided we’ve had enough then we can kill it:

# Send "SIG_TERM" to our task
FireAndForget.term(:long_running_task)

# Or send any signal (see the Process.kill documentation)
FireAndForget.kill(:long_running_task, "HUP")

Supporting multiple users

Tasks will attempt to run as the same user as the originating process. To enable this, and hence enable a single F&F server to launch tasks for any number of separate web-apps, simply run the F&F server as root. See the following section on Security for the implications of this and how to stop unlimited access to the service.

Security

F&F is intended to be run in a relatively trusted environment. You the UNIX socket file has its permissions set so that only the process owner and a defined group can write to the file. To change the UNIX group that has access pass the group name as a parameter to the server:

$ fire_forget -s /path/to/socket -g webapp_group

Only members of the ‘webapp_group’ will be able to launch processes through the server. Additionally, only scripts belonging to the same user as the F&F task description are executed.

In the meantime, as I say, this is defined to be run in a trusted environment, for instance on a single non-shared server (potentially running many sites).

TODO

Contributing to fire_and_forget

Copyright © 2010 Garry Hill. See LICENSE.txt for further details.