Backing up your files using rsync and ssh

Backing up your important files is something everyone should be doing regularly. There are obviously lots of options - like backing up to a CD or DVD, a network share, a USB drive or a server somewhere.

Rsync is a tool that comes standard on Unix/Linux/Mac machines, and provides command line capability to backup files from one location to another.

For this example, we'll be backing up our files to a remote server, and we'll use ssh to securely connect to that server, and rsync will send the files across that ssh connection.

You can build a shell scrip to backup all the various directories you'd like, or use some fancy exclude statements to select the content you want. At some point I'd like to use the exclude statements, but I have a few other projects going right now and I don't have too much time to play with it - so I'm going to just use a shell script.

First thing to do is install the latest version of rsync.

Get the latest version from the rsync site:

cd ~/Desktop
curl -O http://rsync.samba.org/ftp/rsync/rsync-3.0.6.tar.gz
tar -xzvf rsync-3.0.6.tar.gz
rm rsync-3.0.6.tar.gz
curl -O http://rsync.samba.org/ftp/rsync/rsync-patches-3.0.6.tar.gz
tar -xzvf rsync-patches-3.0.6.tar.gz
rm rsync-patches-3.0.6.tar.gz
cd rsync-3.0.6

Apply patches relevant to preserving Mac OS X metadata:
patch -p1 <patches/fileflags.diff
patch -p1 <patches/crtimes.diff
Configure, make, install:
./prepare-source
./configure
make
sudo make install
This should install the latest version of rsync. This first step is not necessary, as long as you have a copy of rsync already installed - but it's always good to use the latest version.

Now, you need to correctly craft your rsync command. Below is a breakdown of the different pieces to the command.

rsync [options] [directory_to_backup] [username@server]:[destination_directory]

The usual option flags are -av, which are 'archive mode' and 'verbose'. There is a third option flag -n, which means 'dry run', which will do everything except actually copy the files. It's usually best to use this -n option when you're developing your command, so you make sure you're copying the right files.

For this example, on my Mac in my home directory, I have a folder called 'pyscripts' which holds some python scripts I've written for different purposes. I'd like to backup that folder to a server I have access to. I've created a directory on the server in my home directory there, called 'backup'.

Username: raj
Server: my.server.com
Source Directory: ~/pyscripts/
Destination Directory: ~/backup/pyscripts/

With that in mind, here is the command I'm going to use:

/usr/local/bin/rsync -av ~/pyscripts/ raj@my.server.com:/home/raj/backup/pyscripts/

This will copy all the files from my local ~/pyscripts/ folder, to my backup folder.

Pretty simple really.

But... The one hitch.

We have to be present to supply our password for ssh. Later, I'll figure out how to use public and private keys to automatically log into the server, so I can truly automate the backup process.