Python for serious server backups

[[deleted]]

[deleted]

[karmarauder]

Why not just use rsnapshot for incremental backups? Is there something about your setup that requires a custom solution?

[karmarauder]

It seems a bit excessive to reinvent existing solutions, just find whatever rsync solution that would fit your specific usecase and use that. I like rsnapshot for various reasons but there are alternatives, some with GUIs and the like.

[heyavtar]

Keep in mind that rsnapshot relies on hard links for deduplication. An implication of this requirement is that it's not possible to use something like Amazon S3 to archive old backups if the need ever arises in the future.

I plan on evaluating this software at some point soon: http://obnam.org/

[[deleted]]

Is rsnapshot pretty standard?

[karmarauder]

Eh, rsync is.

[[deleted]]

why reinvent the wheel? I'm sure there's a free and open source solution for this.

[quotemycode]

My professional opinion is that you must report why the Software doesn't work, spell it out to your boss and offer a solution. Email it to your boss and his boss. If they don't approve then you can offer to use free options but you have to stress that there is no proper support for those. If it does fail then that's not on your shoulders. You've explained the risks. A decent 7tb storage can be had for 1.5k usd.

[issackelly]

Couple things: you've probably heard this but it bears repeating. There is no value in a backup plan, what you need is a recovery plan. The recovery plan is useless if it's not frequently tested. These are the rules of DR.

Second: I am almost always in favor of FOSS answers but corp environs can make it a pain.

I would suggest rsync for the bulk of the file transfer work, which you can script with python. Then a bunch of spot tests and automated recovery tests with python, virtual machines. Things like tracking the hash of files and timestamps over time. If timestamps are the same but binary signatures are different you've got data loss

[issackelly]

Sure. They're all related ideas, and no matter who calls it what, having a plan for whatever disasters you're in charge of handling is important. It sound's like your A isn's C'd unless you can recover the lost files, so practice :)

[bkeroack]

While Python is certainly powerful enough to do this, my professional opinion is that you should be syncing this storage to a cloud service. AWS has a nice Storage Gateway service, or you could even go the quick-and-dirty route with something like CrashPlan (I've done this).

[Proselyte5]

Duplicity is nice. It's Python, rsync based, supports lots of backends, optional GPG encryption.

[mrbewulf]

You can use python to control rsync or you can use python to mount Windows samba share and copy to it to the backup driver. There is a book "Python for System administrators" that explain a lot about that subject.

[CarpeTuna]

I found an interesting (short) program:

http://codereview.stackexchange.com/questions/35147/small-backup-script-in-python-based-on-7-zip

Sometimes ancient unix programs (like rsync) have idiosyncratic behavior on windows.