Dirvish

References

http://www.dirvish.org/

http://edseek.com/~jasonb/articles/dirvish_backup/

http://www.dirvish.org/debian.howto.html

Move a Vault to Another Server

To move a dirvish vault from one server to another:

rsync -a -e ssh -H --delete --numeric-ids my_vault my_server:/data/vaults/

The -H flag preserves hard links. Otherwise you get tons of independent copies of the same data, which will overflow your new server.

Configuring the Server

After installing the dirvish package on the server, create /etc/dirvish/master.conf:

bank:
    /backup
Runall:
    server1  22:00
    server2   22:00
image-default: %Y%m%d
log: gzip
index: gzip
expire-default: +2 weeks
expire-rule:
#       MIN HR    DOM MON       DOW  STRFTIME_FMT
        *   *     *   *         1    +2 months
        *   *     1   *         1    +4 months
#       *   *     1-7 1,4,7,10  1
#       *   10-20 *   *         *    +4 days
#       *   *     *   *         2-7  +15 days

exclude:
  /dev/
  .gvfs
  /initrd/
  lost+found/
  /media/
  /mnt/
  /proc/
  /sys/
  tmp/

post-server: /sbin/copy-scans

The post-server directive lists a command that is executed after dirvish-runall completes. There are also pre-server, pre-client, and post-client.

This supposes that there is a /backup directory with server1 and server2 subdirectories. Config files must be placed in each of these subdirectories:

/backup/server1/dirvish/default.conf
/backup/server2/dirvish/default.conf

Here is an example of default.conf:

client: root@server1.com
tree: /

This logs into the client as root (via ssh) and crawls over the whole file system, transcending symlinks. To back up the local host, set the client to be the output of the "hostname" command.

To keep the backup constrained to a single filesystem (e.g. one disk, no NFS mounts, etc.), use the xdev: true option.

Now create an ssh key-pair as root with an empty passphrase:

ssh-keygen

Copy the public key to each client:

scp ~/.ssh/id_rsa.pub barney@server1:

Configuring the Client

Add the public key to root's authorized file:

[root@server1 ~]# cat ~barney/id_rsa.pub >> .ssh/authorized_keys

The permissions should be 700 for root's home directory and for .ssh, and 600 for authorized_keys. Add a command to the beginning of the authorized_keys file:

command="/root/.ssh/valid_rsync" ssh-rsa AAAAB3N...

and create the file valid_rsync:

#!/bin/sh

case "$SSH_ORIGINAL_COMMAND" in
*\&*)
echo "Rejected"
;;
*\(*)
echo "Rejected"
;;
*\{*)
echo "Rejected"
;;
*\;*)
echo "Rejected"
;;
*\<*)
echo "Rejected"
;;
*\`*)
echo "Rejected"
;;
rsync\ --server*)
$SSH_ORIGINAL_COMMAND
;;
pwd)
pwd
;;
*)
echo "Rejected"
;;
esac

Edit /etc/ssh/sshd_config to have this line:

PermitRootLogin forced-commands-only

From man sshd_config: If this option is set to "forced-commands-only", root login with public key authentication will be allowed, but only if the command option has been specified (which may be useful for taking remote backups even if root login is normally not allowed). All other authentication methods are disabled for root.

NOTE: The setting "UsePAM no" will cause passwordless key-based authentication to fail.

Clean up by deleting the public key file and restarting sshd. Test by running this from the server as root:

ssh root@server1 pwd

You shouldn't be asked for a password.

Usage

Initialize the client backup:

dirvish --vault server1 --init

A nightly cronjob will run all of the "Runall" jobs at the time specified in /etc/cron.d/dirvish.

To manually run a subsequent backup (not usually necessary):

dirvish --vault server1

Behind the scenes

First, rsync runs on the client to create a giant in-memory filesystem tree. Then it does the same on the server. Then it compares and transfers the differences. If rsync has any errors (the filesystem has changed since it created the first tree), the whole process will be performed all over again. When it's done, dirvish creates a big index of the copied filesystem (with find /backup/server1/20070428/tree -ls).