Backups

Posted on Sunday, January 9, 2022 by TheBlackzone

At work I was recently in the situation that I needed to recover data from backups of a file server that had crashed due to a hardware failure. The incident has apparently shaken up a few people which subsequently approached me with questions on how they should organize their own personal backups at home.

Although I couldn’t advise each of them to their individual needs, my general recommendations on this topic were:

• Make backups frequently. It doesn’t help much if your last backup is from one year ago and you loose all data that has been added since. Adjust your backup frequency to how often your data changes or new data is added.

• In general it is a good idea to automate backups so you do not forget to create them. You may want to choose a specialized software or service to do so. But be careful not to be locked-in to some proprietary solution that prevents you from accessing your backups from other computers.

• Sometimes it may suffice to just have a simple copy of your data but most often it is better to have multiple complete sets from different points in time. Depending on the volume of your data, you could create a new copy of your data each time you back it up. Or you may want to use some backup software that does this for you.

• Always have more then one backup and keep them on multiple devices. These days it is quite common to use external USB disk drives, which are great to handle but could easily fail or become defective. You should have at least two or three different devices to save your data to.

• Make sure you have at least one copy of your data off-site. That means, keep a copy of your data outside your home, for example in a bank deposit box or at a friend’s home. Make sure your data is encrypted in this case and do not forget to freshen these backups frequently as well.

• An alternative to the previous point may be to use an online storage or backup provider to keep off-site copies of your data. This can be a lot more convenient than passing around physical devices. But never rely entirely on such services as they may become unavailable for any number of reasons. Needless to say that encryption (under your control) is an absolute must before any data passes the wire.

• Last but not least, verify that your backups are valid and complete. And that you are able to restore your data from them.

In the end, the most important thing is to implement a backup routine at all and to keep backups up-to-date regularly. You do not need a highly sophisticated solution for it. Just make sure that it’s there and holds all the data that is valuable to you.

That being said, you may be interested on how I have set up my own personal backup…

Before I start, please note that I run the Debian Linux operating system on my computers, so the tools mentioned below are specific to this platform (and other Linux distributions, of course).

My backup setup

I have a lot of digital photos, digitized documents as well as a lot of documents written by myself. Not to mention source code from my projects, archived data from my various old computer systems, the list goes on. Naturally I am quite cautious not to lose this data and have a solid backup strategy.

As always, I’m using open source tools to perform this task, namely

• BorgBackup
• rsync

I use ‘rsync’ mainly for ad-hoc, one-time backups of specific files or folders, while ‘BorgBackup’ is my main backup solution and does all the heavy lifting. I have chosen BorgBackup specifically for its ability of deduplicating data and store backups encrypted. It also frees me from the decision for a specific backup strategy, like, for example, ‘incremental’, because each backup in Borg is basically a ‘full backup’. That means I can have any number of ‘snapshots’ and keep the full state of my data at each individual backup.

As an aside: Before ‘BorgBackup’, I have been using Back In Time to create my backups. It’s a viable alternative with an easy to use graphical user interface. The reasons for my switch to BorgBackup had been mainly to consolidate my toolset.

Backup destinations

As mentioned above, it’s always a good idea to have multiple copies of your data stored in different locations. I follow this recommendation and create backups to various media and store them in different locations.

My current backup scenario comprises two ‘online’ backups and two ‘offline’ backups and looks like this:

Now, let me walk you through this…

First Backup Stage - to secondary disk

My laptop has two SSD drives built in. One of them is holding my home directory with all my data, the second one is used exclusively for backups.

In this first stage I create frequent, fully automated backups with BorgBackup to the second builtin hard disk. If I every accidentally delete something, I can easily recover it from there. The same would be the case if my main hard drive fails. I could replace it, re-install the operating system and recover my data from the second disk and that’s it.

This first stage is for my convenience and only covers the mentioned damage scenarios. It is, of course, absolutely insufficient because my laptop could be stolen, lost or destroyed.

This is where the following stages come into play.

Second Backup Stage - to external VPS

In this stage I do exactly the same as in the first one, except the destination is a virtual private server (VPS) that I have rented at some hosting provider.

BorgBackup is using SSH to securely transfer data to this server and the backups are fully encrypted using Borg’s builtin encryption functionality.

This stage prevents me (hopefully) from losing data through major incidents where my hardware get lost or destroyed.

But, you never know, as the hosting service I am using could become unavailable for any reason.

So I have implemented a third stage.

Third stage - to external USB disks

In addition to the previous stages, which are fully automated, I also use external hard drives to regularly create ‘offline’ backups. I have set up a reminders in my calendar to perform those backups.

The disk are used interchangeably, each holding backups created with Borg. ‘Offline’ in this case means that the disks are only connected to the computer for the sole reason to create backups. Other from that, they are stored away in remote locations.

Because I do not have immediate access to those disks, backup frequency is less than in the previous stages. Apart from that the line of action is the same as in the previous stages.

Conclusion

When it comes to securing my data I live by the premise “better safe than sorry”. For me, the solution I have described here works quite well. Some may think that I am exaggerating it, while others may be even more paranoid about their backups.

Luckily I have not yet been in the situation (keeping fingers crossed) to be forced to restore all of my data from a backup. Nevertheless I have been more than once happy to be able to access a previous state of my data and recover an accidentally deleted file or even something I intentionally delete because I was sure I would never need it again.

So, guys, do your backups…

Tags: computer