README.rdoc
= resque-state
{<img src="https://img.shields.io/badge/license-MIT-blue.svg" />}[https://github.com/nathan-v/resque-state/blob/master/LICENSE]
{<img src="https://img.shields.io/gem/v/resque-state.svg" /}[https://rubygems.org/gems/resque-state]
{<img src="https://codeclimate.com/github/nathan-v/resque-state/badges/gpa.svg" />}[https://codeclimate.com/github/nathan-v/resque-state]
{<img src="https://codeclimate.com/github/nathan-v/resque-state/badges/issue_count.svg" />}[https://codeclimate.com/github/nathan-v/resque-state]
{<img src="https://codeclimate.com/github/nathan-v/resque-state/badges/coverage.svg" />}[https://codeclimate.com/github/nathan-v/resque-state/coverage]
{<img src="https://travis-ci.org/nathan-v/resque-state.svg?branch=master" alt="Build Status" />}[https://travis-ci.org/nathan-v/resque-state]
{<img src="https://gemnasium.com/badges/github.com/nathan-v/resque-state.svg" />}[https://gemnasium.com/github.com/nathan-v/resque-state]
{<img src="https://img.shields.io/github/issues/nathan-v/resque-state.svg" />}[https://github.com/nathan-v/resque-state/issues]
resque-state is an extension to the resque queue system that provides simple trackable jobs.
== About
resque-state provides a set of simple classes that extend resque's default
functionality (with 0% monkey patching) to give apps a way to track specific
job instances and their state. It achieves this by giving job instances UUID's
and allowing the job instances to report their state from within their iterations.
== Installation
Ruby 2.2.2+ and JRuby 9.1+ are supported.
resque-state <b>requires Redis >= 1.1</b> (though I recommend getting the latest stable version).
You can download Redis here: http://redis.io/ or install it
using homebrew (brew install redis).
Install the resque-state gem (which will pull in the dependencies).
gem install resque-state
With newer Rails add this to your Gemfile:
# Gemfile
gem 'resque-state'
Then in an initializer:
# config/initializers/resque.rb
Resque.redis = "your/redis/socket" # default localhost:6379
Resque::Plugins::State::Hash.expire_in = (24 * 60 * 60) # 24hrs in seconds
== Usage
The most direct way to use resque-state is to create your jobs using the
Resque::Plugins::State module. An example job would look something like:
class SleepJob
include Resque::Plugins::State
def perform
total = (options['length'] || 1000).to_i
total.times do |i|
num = i+1
at(num, total, "At #{num} of #{total}")
sleep(1)
end
end
end
One major difference is that instead of implementing <tt>perform</tt> as a
class method, we do our job implementation within instances of the job class.
In order to queue a SleepJob up, we also won't use <tt>Resque.enqueue</tt>, instead
we'll use the <tt>create</tt> class method which will wrap <tt>enqueue</tt> and
creating a unique id (UUID) for us to track the job with.
job_id = SleepJob.create(length: 100)
This will create a UUID enqueue the job and pass the :length option on the SleepJob
instance as options['length'] (as you can see above).
Now that we have a UUID its really easy to get the state:
state = Resque::Plugins::State::Hash.get(job_id)
This returns a Resque::Plugins::State::Hash object, which is a Hash (with benefits).
state.pct_complete #=> 0
state.status #=> 'queued'
state.queued? #=> true
state.working? #=> false
state.time #=> Time object
state.message #=> "Created at ..."
Once the worker reserves the job, the instance of SleepJob updates the state at
each iteration using <tt>at()</tt>
state = Resque::Plugins::State::Hash.get(job_id)
state.working? #=> true
state.num #=> 5
state.total #=> 100
state.pct_complete #=> 5
If an error occurs within the job instance, the state is set to 'failed' and then
the error is re-raised so that Resque can capture it.
Its also possible to get a list of current/recent job statuses:
Resque::Plugins::State::Hash.statuses #=> [#<Resque::Plugins::State::Hash>, ...]
=== Passing back data from the job
You may want to save data from inside the job to access it from outside the job.
A common use-case is web-triggered jobs that create files, later available for
download by the user.
A Status is actually just a hash, so inside a job you can do:
set_status(filename: "myfilename")
Also, all the status setting methods take any number of hash arguments. So you could do:
completed('filename' => '/myfilename')
=== Kill! Kill! Kill!
Because we're tracking UUIDs per instance, and we're checking in/updating the status
on each iteration (using <tt>at</tt> or <tt>tick</tt>) we can kill specific jobs
by UUID.
Resque::Plugins::State::Hash.kill(job_id)
The next time the job at job_id calls <tt>at</tt> or <tt>tick</tt>, it will raise a <tt>Killed</tt>
error and set the status to killed.
=== Hold up: Pausing
Since we perhaps might want to just have a job sit and wait a bit rather than have it die
completely there's pause. This tells the job to sleep for 10 seconds before checking in
again.
Resque::Plugins::State::Hash.pause(job_id)
The next time the job at job_id calls <tt>at</tt> or <tt>tick</tt>, it will start a while
loop with a 10 second sleep until Resque::Plugins::State::Hash.unpause is called.
As of 1.1 pause! can be called from the job itself with an optional string argument
that allows you to set a specific status along with the paused state if you'd like.
=== Back it up
Perhaps you want a job to be able to undo what it did. Well; that's what revert is
all about. Jobs that support on_revert can be told to revert and will run insructions
in that method to undo whatever they might need to. While you could do similar things
with on_failure on_revert was added to provide a separate status for reverting and
reverted cases so you can see later which jobs failed (probably due to an error) or
which ones might have been intentionally reverted by a user or automation.
Resque::Plugins::State::Hash.revert(job_id)
This is all you need to tell that job to go back and undo it's work. For jobs that
do not support on_revert those jobs will pause themeslves and leave a note to the user
that the job doesn't allow that functionality. This lets the user decide if the job
should be killed or let continue.
=== Percent Complete and setting the message
Use <tt>at</tt> or <tt>tick</tt> to show progress in your job's <tt>perform</tt> function
(which is displayed on the resque-web state tab). This will also be where <tt>Killed</tt>
is raised if the job is killed.
at(steps_completed, total_steps, "${steps_completed} of #{total_steps} steps completed!")
=== Expiration
Since Redis is RAM based, we probably don't want to keep these statuses around forever
(at least until @antirez releases the VM feature). By setting expire_in, all statuses
and their related keys will expire in expire_in seconds from the last time theyre updated:
Resque::Plugins::State::Hash.expire_in = (60 * 60) # 1 hour
=== Testing
Recent versions of Resque introduced <tt>Resque.inline</tt> which changes the behavior to
instead of enqueueing and performing jobs to just executing them inline. In Resque
itself this removes the dependency on a Redis, however, <tt>Resque::State</tt> uses Redis
to store information about jobs, so though <tt>inline</tt> "works", you will still need
to use or mock a redis connection. You should be able to use a library like
https://github.com/causes/mock_redis alongside <tt>inline</tt> if you really want to
avoid Redis connections in your test.
=== resque-web
Though the main purpose of these trackable jobs is to allow you to surface the status
of user created jobs through your apps' own UI, I've added a simple example UI
as a plugin to resque-web. This adds a State tab to resque-web
To use, you need to setup a resque-web config file:
# ~/resque_conf.rb
require 'resque/state_server'
Then start resque-web with your config:
resque-web ~/resque_conf.rb
If you're using Rails you can just require 'resque/state_server' in your resque
initializer or application.rb.
== More
* Source: http://github.com/nathan-v/resque-state
* API Docs: http://rdoc.info/projects/nathan-v/resque-state
* Examples: http://github.com/nathan-v/resque-state/tree/master/examples
* Resque: https://github.com/resque/resque
== Thanks
Resque is awesome, @defunkt needs a shout-out.
== Note on Patches/Pull Requests
PRs are welcome. Please always include relevant tests and ensure your changes follow
Ruby style conventions as much as possible.
== Copyright
Copyright (c) 2016 Nathan V.
Copyright (c) 2010 Aaron Quint.
MIT licensed. See LICENSE file for details.