README.markdown
# astrails-safe
Simple database and filesystem backups with S3 and Rackspace Cloud Files support (with optional encryption)
* Home: [http://astrails.com/opensource/astrails-safe](http://astrails.com/opensource/astrails-safe)
* Code: [http://github.com/astrails/safe](http://github.com/astrails/safe)
* Blog: [http://astrails.com/blog/astrails-safe](http://astrails.com/blog/astrails-safe)
[](https://travis-ci.org/astrails/safe)
[](https://codeclimate.com/github/astrails/safe)
## Motivation
We needed a backup solution that will satisfy the following requirements:
* opensource
* simple to install and configure
* support for simple ‘tar’ backups of directories (with includes/excludes)
* support for simple mysqldump of mysql databases
* support for symmetric or public key encryption
* support for local filesystem, Amazon S3, and Rackspace Cloud Files for storage
* support for backup rotation. we don’t want backups filling all the diskspace or cost a fortune on S3 or Cloud Files
And since we didn't find any, we wrote our own :)
## Contributions
The following functionality was contributed by astrails-safe users:
* PostgreSQL dump using `pg_dump` (by Mark Mansour <mark@stateofflux.com>)
* Subversion dump using svndump (by Richard Luther <richard.luther@gmail.com>)
* SFTP remote storage (by Adam <adam@mediadrive.ca>)
* benchmarking output (By Neer)
* README fixes (by Bobby Wilson)
* improved config file parsing (by Fedor Kocherga <fkocherga@gmail.com>)
* mysql password file quoting (by Jonathan Sutherland <jonathan.sutherland@gmail.com>)
* Rackspace Cloud Files support (by H. Wade Minter <minter@lunenburg.org>)
* Plan FTP support (by seroy <seroy@bk.ru>)
* mongodump support (by Matt Berther <matt@mattberther.com>)
Thanks to all :)
## Installation
sudo gem install astrails-safe --source http://gemcutter.org
## Reporting problems
Please report problems at the [Issues tracker](http://github.com/astrails/safe/issues)
## Usage
Usage:
astrails-safe [OPTIONS] CONFIG_FILE
Options:
-h, --help This help screen
-v, --verbose be verbose, duh!
-n, --dry-run just pretend, don't do anything.
-L, --local skip remote storage, only do local backups
Note: CONFIG\_FILE will be created from template if missing
## Encryption
If you want to encrypt your backups you have 2 options:
* use simple password encryption
* use GPG public key encryption
> IMPORTANT: some gpg installations automatically set 'use-agent' option in the default
> configuration file that is created when you run gpg for the first time. This will cause
> gpg to fail on the 2nd run if you don't have the agent running. The result is that
> 'astrails-safe' will work ONCE when you manually test it and then fail on any subsequent run.
> The solution is to remove the 'use-agent' from the config file (usually /root/.gnupg/gpg.conf)
> To mitigate this problem for the gpg 1.x series '--no-use-agent' option is added by defaults
> to the autogenerated config file, but for gpg2 is doesn't work. as the manpage says it:
> "This is dummy option. gpg2 always requires the agent." :(
For simple password, just add password entry in gpg section.
For public key encryption you will need to create a public/secret keypair.
We recommend to create your GPG keys only on your local machine and then
transfer your public key to the server that will do the backups.
This way the server will only know how to encrypt the backups but only you
will be able to decrypt them using the secret key you have locally. Of course
you MUST backup your backup encryption key :)
We recommend also pringing the hard paper copy of your GPG key 'just in case'.
The procedure to create and transfer the key is as follows:
1. run 'gpg --gen-key' on your local machine and follow onscreen instructions to create the key
(you can accept all the defaults).
2. extract your public key into a file (assuming you used test@example.com as your key email):
`gpg -a --export test@example.com > test@example.com.pub`
3. transfer public key to the server
`scp test@example.com.pub root@example.com:`
4. import public key on the remote system:
$ gpg --import test@example.com.pub
gpg: key 45CA9403: public key "Test Backup <test@example.com>" imported
gpg: Total number processed: 1
gpg: imported: 1
5. since we don't keep the secret part of the key on the remote server, gpg has
no way to know its yours and can be trusted.
To fix that we can sign it with other trusted key, or just directly modify its
trust level in gpg (use level 5):
$ gpg --edit-key test@example.com
...
Command> trust
...
1 = I don't know or won't say
2 = I do NOT trust
3 = I trust marginally
4 = I trust fully
5 = I trust ultimately
m = back to the main menu
Your decision? 5
...
Command> quit
6. export your secret key for backup
(we recommend to print it on paper and burn to a CD/DVD and store in a safe place):
$ gpg -a --export-secret-key test@example.com > test@example.com.key
## Example configuration
safe do
verbose true
local :path => "/backup/:kind/:id"
s3 do
key "...................."
secret "........................................"
bucket "backup.astrails.com"
path "servers/alpha/:kind/:id"
end
cloudfiles do
user "..........."
api_key "................................."
container "safe_backup"
path ":kind/" # this is default
service_net false
end
sftp do
host "sftp.astrails.com"
user "astrails"
# port 8023
password "ssh password for sftp"
end
gpg do
command "/usr/local/bin/gpg"
options "--no-use-agent"
# symmetric encryption key
# password "qwe"
# public GPG key (must be known to GPG, i.e. be on the keyring)
key "backup@astrails.com"
end
keep do
local 20
s3 100
cloudfiles 100
sftp 100
end
mysqldump do
options "-ceKq --single-transaction --create-options"
user "root"
password "............"
socket "/var/run/mysqld/mysqld.sock"
database :blog
database :servershape
database :astrails_com
database :secret_project_com do
skip_tables "foo"
skip_tables ["bar", "baz"]
end
end
svndump do
repo :my_repo do
repo_path "/home/svn/my_repo"
end
end
pgdump do
options "-i -x -O" # -i => ignore version, -x => do not dump privileges (grant/revoke), -O => skip restoration of object ownership in plain text format
user "username"
password "............" # shouldn't be used, instead setup ident. Current functionality exports a password env to the shell which pg_dump uses - untested!
database :blog
database :stateofflux_com
end
tar do
options "-h" # dereference symlinks
archive "git-repositories", :files => "/home/git/repositories"
archive "dot-configs", :files => "/home/*/.[^.]*"
archive "etc", :files => "/etc", :exclude => "/etc/puppet/other"
archive "blog-astrails-com" do
files "/var/www/blog.astrails.com/"
exclude "/var/www/blog.astrails.com/log"
exclude "/var/www/blog.astrails.com/tmp"
end
archive "astrails-com" do
files "/var/www/astrails.com/"
exclude ["/var/www/astrails.com/log", "/var/www/astrails.com/tmp"]
end
end
end
## Contributing
1. Fork it
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request
## Copyright
Copyright (c) 2010-2013 Astrails Ltd. See LICENSE.txt for details.