Justin Dugger ([info]jldugger) wrote,
@ 2009-05-02 02:24:00
Previous Entry  Add to memories!  Tell a Friend  Next Entry
Entry tags:backup, debian, linux, rsync, ubuntu

Backups galore

I thought I'd do a quick survey and figure out what backup tools are readily available in Ubuntu. I found 17 that might be worth mentioning. Many are front-ends, and front-ends-to-the-front-ends, so to keep them straight I cooked up a quick diagram with GraphViz:

backups galore

So there's a lot, and I may have missed a few nodes or edges. So far, my favorite is probably mrb:

Package: mrb
Description: Manage incremental data snapshots with make/rsync
 mrb is a single, self-documenting, executable makefile, which aims to make
 trivial the task of maintaining a set of compact, incremental, rsync mirrors
 of your important (and sometimes rapidly changing) data.
 .
 It relies only on the time-hardened industry tools GNU make and rsync.
 Snapshots may be taken at any opportune interval.  Multiple snapshot targets
 can be configured in a modular fashion, so fast changing data can be separated
 from static bulk data, with snapshots of each scheduled or triggered on demand,
 as may be appropriate for each.
At first you think, "Perfect, how hard can a make frontend to rsync be?" 15 kilobytes of Makefile later, you realize this may not be as brilliant as it sounds on paper. But it is well commented and relatively user friendly, which may actually impede the code's readability (as if anything involving whitespace syntax could be readable). Such gems as:
# If I have to explain this one, then I guess you are just reading this
# 'for the articles' -- but I hope you'll have enjoyed it anyway...
I haven't tried mrb yet, but I might put it on the TODO list. But first, I'll need to give jwz's method a full vetting.



(15 comments) - (Post a new comment)

backup.dot , for the curious
[info]jldugger
2009-05-02 07:40 am UTC (link)
digraph G {
   node [style=filled];

# root tools
	 dump;
	 rsync;
	 tar;
	 dar;

#1st derivatives
   rsync -> "rdiff-backup";
   rsync -> duplicity;
   rsync -> dirvish;
	 rsync -> rsnapshot;
	 rsync -> hdup;
	 rsync -> mrb;
	 rsync -> flexbackup;
	 tar -> flexbackup;
   tar -> sbackup;
	 tar -> backup2l;
	 tar -> amanda;
	 tar -> "backup-manager";
	 dump -> flexbackup
	 
# 2nd derivatives
   duplicity -> "Deja Dup";
	 duplicity -> "backup ninja";
	 "rdiff-backup" -> "backup ninja";
	 "rdiff-backup" -> keep;
	 "rdiff-backup" -> pybackpack;
	 
}

(Reply to this) (Thread)

Re: backup.dot , for the curious
[info]oei.yungchin.nl
2009-05-03 11:06 pm UTC (link)
If you look at the dependencies for rdiff-backup and duplicity, you actually won't find rsync. You will find librsync, which seems to have been written taking lots of inspiration from rsync, but it is a separate codebase. In spirit they do belong under rsync's heritage of course, so I'd say your tree is fine this way.

As for the other comments regarding backuppc: I suppose the question is, what makes a tool "worth mentioning"? Looking at debian popcon data, backuppc is up there with amanda, and definitely beats tools like hdup, dirvish, duplicity, dar and sbackup when it comes to user base.


(Reply to this) (Parent)(Thread)

Re: backup.dot , for the curious
[info]jldugger
2009-05-03 11:54 pm UTC (link)
I wrote the tree by hand, so there may not be a direct correlation between apt-cacher outputs. I decided to treat librsync as building on rsync itself. One could conceivably write a tool to answer this question with less room human error.

BackupPC is worth mentioning, yes. Mainly I wrote the diagram to inform my self about what things are based on what. Tools like backupPC and amanda are really so complex they're outside the usual, and inherit from everything. I have notes on them in Tomboy, but I'm working on clarifying my use case(s) and criteria to test all these with.

(Reply to this) (Parent)

What about backuppc?
(Anonymous)
2009-05-02 08:23 am UTC (link)
backuppc is my favorite backup solution. See http://packages.ubuntu.com/jaunty/backuppc

(Reply to this) (Thread)

Re: What about backuppc?
[info]jldugger
2009-05-02 08:41 am UTC (link)
Ah right. It's so far out in left field. They way I read it, it scans the to be backed up host rather than running as a cron job. It didn't seem right to place tar or rsync as its parent...

(Reply to this) (Parent)

Bacula
[info]ivoks [launchpad.net]
2009-05-02 08:34 am UTC (link)
Bacula is one of the most advanced backup solutions on the market. How did you manage to miss that one? :)

(Reply to this) (Thread)

Re: Bacula
[info]jldugger
2009-05-02 08:42 am UTC (link)
I didn't, but I think I made the diagram with an eye towards "what's based on what", so bacula didn't make the cut. Then again, dar did, so who knows.

(Reply to this) (Parent)

Also missing...
[info]maco.myopenid.com
2009-05-02 11:19 am UTC (link)
Mondo & Mindi: Bare-metal backup & recovery.

(Reply to this) (Thread)

tar och duplicity
[info]andreasolsson.se
2009-05-02 01:55 pm UTC (link)
Perhaps you'd also like to draw an arrow from tar to duplicity?

(Reply to this) (Parent)

safekeep?
[info]jspaleta
2009-05-02 04:11 pm UTC (link)
http://safekeep.sourceforge.net/

(Reply to this) (Thread)

Re: safekeep?
[info]jldugger
2009-05-02 07:53 pm UTC (link)
Not in Ubuntu. The newest package they publish is Hardy. The reason this is important is recovery. Ideally, we'd have a supported recovery tool on the LiveCD. Having it available via apt-get is a close second. Downloading a tarball and compiling it in a live environment is a distant last in my book.

Now, it does look interesting and having a package sounds like a good idea at the moment.

(Reply to this) (Parent)(Thread)

Re: safekeep?
[info]jspaleta
2009-05-03 12:51 am UTC (link)
Dear god! A software package that is not available for the latest Ubuntu release!
The horror! Its almost like you expect upstream project developers to do the work of distro specific release management for you. How big is the MOTU team? less than 150 people? Are they the full extent of the packaging workforce that Ubuntu has? If they aren't too overworked, how about you drop safekeep into their laps and see if they can't get it into Universe for you.

-jef

(Reply to this) (Parent)(Thread)

Re: safekeep?
[info]jldugger
2009-05-03 07:54 am UTC (link)
Chill out bro. It's just my selection criteria for the day. I'm not saying that SafeKeep isn't a good tool, or that they should keep packaging it for Ubuntu out of the kindness of their hearts. As the graphic shows, there's some 17 backup and recovery tools already available through apt-get, and there's over a thousand in the SourceForge backup category. I'm not trying to rule out packaging it if it's the tool I'm looking for, but it only makes sense to look at what's already been done first.

Probably, it's important to understand motivations. I'm looking at this from a perspective of what's already out there and what's needed. Every release we tell people "Don't forget to backup before you upgrade!" and nobody does it. Nobody. Every extra step you have to take at install is a diminished user base, and every step you have to take during restore is another person cursing your name.

What this means is that we need a simple install (i.e. apt-get and as few hardware resources as possible), sane and effective defaults, and simple recovery. safekeep's recovery using rdiff-backup is a plus, since that IS packaged in Ubuntu, and can be retrieved into a liveCD environment, or possibly even placed on the CD itself. I'm not quite yet comfortable with recommending a command line restore tool, however.

(Reply to this) (Parent)(Thread)

another bump for backuppc
(Anonymous)
2009-05-03 07:53 pm UTC (link)
I've been very satisfied with backuppc. It took like 5 minutes to set up, and another hour to config to back up all my servers just the way I liked.

(Reply to this) (Parent)(Thread)

Re: another bump for backuppc
[info]jldugger
2009-05-03 08:14 pm UTC (link)
Interesting, but you don't mention how long it took to restore. I have to assume you haven't tried it under any conditions, let alone adverse ones.

(Reply to this) (Parent)


(15 comments) - (Post a new comment)

Create an Account
Forgot your login or password?
Login w/ OpenID
English • Español • Deutsch • Русский…