My Ubuntu backup solution
I've had a string of unfortunate data losses recently. As far as I could tell, it was just bad luck (nothing malicious), but frustrating nonetheless. The really sad part is that I thought I had a pretty decent backup solution in place, yet still lost data.
After having experienced this type of data loss on a few occasions in a short period of time, I've had a chance to test the resiliency of my backup solution.
It started completely at random. First, my Ubuntu laptop decides to wipe out both its drives. Full day of recovery attempts yield nothing. I lost that day's work due to being in the "zone" too long and not committing/pushing up to origin.
Two days later, my Windows gaming PC's hard drive decides to die. Great.
Thankfully, I had Backblaze setup on that PC, so everything was safe. Also, I had Cubby running across my all three of my Windows PC's in the house (laptop, media center, gaming) sync'ing my Documents, Pictures, Videos and Music folders. Phew. No need to download 120GB of family photos & videos from Backblaze (as a sad aside, I butchered my backup on Backblaze... downloaded zip extracted 0 byte files... lost all the electronic music source files I wrote over 15 years). Of course I didn't notice until after I Shift+Deleted the zip file...)
Recovering my gaming PC was easy enough. A fresh install of Windows is always welcome, and I love the convenience of downloading my games from the cloud, i.e. Steam.
Setting up Ubuntu
But setting up my Ubuntu dev machine was quite a bit more work, simply because I had configured quite a few things manually. So I set myself up with a cron script that would backup my settings into Dropbox. In this same folder, I also started a
notes.txt file with oddball instructions when setting up. And on top of that, I installed Ubuntu One with the sole purpose of having it backup my entire Dropbox. Furthermore, I installed Crashplan to back up the whole damn machine while I was at it (I'd use Backblaze if it were available for Linux).
Disaster strikes... again
So at 4:30pm on a Friday, perhaps 1 month after all this went down, I proceed to run some poorly conceived (by me) automated script that
sudo rsync --deletes some nearly empty folder into root. Long story short, Ubuntu dev machine wiped out in a few seconds (hey, SSD's are fast...).
But this time around, I was able to get to a working state within about 1h30. The only big annoyance was the loss of my Windows VM, since that's huge and changes too often to go up on Crashplan.
As I was busy rebuilding my Ubuntu dev machine once again, I documented every single damn step of the way, so that my originally dismal
notes.txt file was full of ready-to-execute commands and useful instructions. I had also missed some configuration files in my manifest, so I made sure that I didn't miss anything this time around.
I also made a copy of my Windows VM onto other computers. It's a manual process, but one that I'll do only if I make big important changes to the VM. At least I'd have a pretty clean Windows VM to come back to.
Now I feel quite confident in my backup solution, as I've had a chance to test it out in depth. I suppose if Dropbox and Ubuntu One were to be deleted and had time to sync that deletion to the cloud, that would put a gaping hole in my backup solution. I'll keep thinking on that one. Other than that, I think I'm good!
If you're interested in this solution, here is my source (very minor omissions for settings that are not relevant to others). Be sure to substitute
mbouchard for your username/group:
Execute this file to setup the cron job. It'll execute hourly.
#!/bin/sh sudo ln -s $HOME/Dropbox/UbuntuSettingsBackup/backup.sh /etc/cron.hourly/UbuntuSettingsBackup
#!/bin/sh BACKUPNAME=$HOME/Dropbox/UbuntuSettingsBackup/backups/backup-`hostname`-`date +"%Y-%b"`-`uname -s`-`uname -r`.tar.gz MANIFEST=$HOME/Dropbox/UbuntuSettingsBackup/manifest.txt sudo tar czfpv $BACKUPNAME -T $MANIFEST sudo chown mbouchard:mbouchard $BACKUPNAME
You'll want to edit this to back up the files that matter to you. Note that whole directories are supported too, which is great. This file is ever-changing as I fine tune my setup over time.
/home/mbouchard/.ssh /home/mbouchard/.filezilla/sitemanager.xml /home/mbouchard/.gconf /home/mbouchard/.remmina /home/mbouchard/.config/sublime-text-2/Packages/User /home/mbouchard/.config/user-dirs.dirs /home/mbouchard/.config/monitors.xml /etc/php5/apache2/php.ini /etc/apache2/conf.d/fqdn /etc/apache2/envvars /etc/apache2/sites-available /etc/NetworkManager/NetworkManager.conf /etc/resolvconf/resolv.conf.d/head /etc/dnsmasq.d/my-local /etc/dnsmasq.conf /etc/nsswitch.conf /etc/sysctl.d/10-inotify.conf /etc/ssh/sshd_config /etc/hosts /etc/fstab
Similarly to the
manifest.txt file, this is a living, breathing text file that I continually update as necessary. This is a key habit to maintain in order to keep the restore process as smooth as possible.
UBUNTU INSTALL NOTES ==================== dnsmasq: -------- sudo apt-get install dnsmasq dev core: --------- sudo apt-get install build-essential libssl-dev curl git-core Show all startup apps in Startup Applications: ---------------------------------------------- sudo sed -i 's/NoDisplay=true/NoDisplay=false/g' /etc/xdg/autostart/*.desktop LAMP: ----- sudo apt-get install apache2 sudo apt-get install mysql-client mysql-server php5 php5-mysql php5-curl php5-xsl sudo /etc/init.d/apache2 restart Ruby: ----- sudo apt-get install ruby1.9.1 rubygems curl -L https://get.rvm.io | bash -s stable Java: ----- Install as per accepted answer (JRE and the browser plugin): http://superuser.com/questions/353983/how-do-i-install-the-sun-java-sdk-in-ubuntu-11-10-oneric-and-later-versions Meteor: ------- curl https://install.meteor.com | /bin/sh sudo apt-get install mongodb-clients Git: ---- sudo apt-get install git git config --global --unset core.autocrlf git config --global --add core.eol lf # should convert CRLF to LF permanently SVN: ---- sudo apt-get install subversion node.js: -------- sudo apt-get install python-software-properties sudo add-apt-repository ppa:chris-lea/node.js sudo apt-get update sudo apt-get install nodejs npm www: ---- sudo mkdir /var/www sudo chown mbouchard:mbouchard /var/www mkdir /var/www/mcmillan mkdir /var/www/mathieu ln -s /var/www $HOME/www Ubuntu Software Centre: ----------------------- - Pidgin - Ubuntu restricted extras Calculator shortcut: -------------------- sudo add-apt-repository ppa:scopes-packagers/ppa sudo apt-get update sudo apt-get install unity-lens-utilities unity-scope-calculator Desktop background: ------------------- See $HOME/Dropbox/Pictures/Desktop/ MyUnity: -------- sudo apt-get install myunity VMWare: ------- Download from VMWare site, install as per instructions: http://www.webupd8.org/2012/06/how-to-install-vmware-player-in-ubuntu.html Note: You may need to edit the .sh file and change the version number of VMWare to the latest Dropbox: -------- Install into $HOME/Dropbox (default) Ubuntu One: ----------- Install into $HOME/Ubuntu One (default) Be sure to add the $HOME/Dropbox folder as an added path for Ubuntu One to backup (for redundancy) ---> EXTRACT BACKUP NOW <--- Apache enabled/available: ------------------------- sudo ln -s /etc/apache2/mods-available/include.load /etc/apache2/mods-enabled/include.load sudo ln -s /etc/apache2/mods-available/rewrite.load /etc/apache2/mods-enabled/rewrite.load sudo ln -s /etc/apache2/mods-available/vhost_alias.load /etc/apache2/mods-enabled/vhost_alias.load # I've commented these out for this blog post, but leaving as examples: # sudo ln -s /etc/apache2/sites-available/webtools /etc/apache2/sites-enabled/001-webtools # sudo ln -s /etc/apache2/sites-available/local /etc/apache2/sites-enabled/002-local # sudo ln -s /etc/apache2/sites-available/gtracker /etc/apache2/sites-enabled/003-gtracker UbuntuSettingsBackup cron: -------------------------- $HOME/Dropbox/UbuntuSettingsBackup/setup-cron.sh