Backup Linux system to web

Question

First I create a .tar of the root directory with all the file permissions saved and excluding all irrelevant directories (are these the right folders to be excluding?).

sudo tar -cvpzf /HDD/systemBackup/backup.tar.gz --one-file-system --exclude=/HDD --exclude=/NAS --exclude=/proc exclude=/tmp --exclude=/mnt --exclude=/dev --exclude=/sys /

Then the .tar is split into smaller files of 100mb for easier upload efficiency.

split -b 100MB /HDD/systemBackup/backup.tar.gz "/HDD/systemBackup/$date-"

Then I upload the parts to the Internet.

---

I am looking to optimise ease of restoration and speed efficiency (ignoring (to some extent) the speed of the upload to the Internet).

Will this method also work on both a Raspberry Pi and an Ubuntu system?

Is this a good method of backing up my Linux system to the web?

Answer 1

When it comes to system backups, I think it's more important to know what you include than to know what you exclude. So instead of specifying what you exclude, I think it's better to think about what you need to include. It might be a shorter list, and less noisy without all the --exclude flags.

Your current method is wasteful: the backup takes twice the size it needs, once for the .tar.gz, and once for the split version. You could pipe the output of the tar command to split to save some space, for example:

sudo tar -cvpz --one-file-system /usr /etc /home ... | split -b 100MB - "/HDD/systemBackup/$date-"

Most importantly, as @tarleb also said, make sure to test restoring from backup. An all too common mistake is to not test restoring, and one day when you really need it, your backups turn out to be useless.

Will this method also work on both a Raspberry Pi and an Ubuntu system?

The list of directories you want to backup might be a little bit different. If you have to inspect both systems and see. Other than that, the same technique should work, yes.

Is this a good method of backing up my Linux system to the web?

This is one way to do it. It's a full backup. Its advantage is that it's simple. Its drawback is that it wastes a lot of space, because much of the data in each backup for different $date values will be duplicated. And transferring over the network will be relatively slow.

An alternative would be using incremental backup, which would save disk space and network bandwidth, at the expense of considerably more complex backup and restore procedure.

One more way would be to rsync the interesting directories, and take create a timestamped archive at the other side. That would save disk space locally, save network bandwidth, and still be relatively simple.

Answer 2

It's a solid way to do backups. I'm assuming you already know about rdiff-backup and wrappers like backupninja, so you've likely already decided that those don't fit your use-case.

Since your code is likely going to be wrapped in a shell script, the --exclude parameters could be created in a loop. This would prevent errors such as the missing -- prefix in the example you gave. Don't exclude / or you won't get any backups.

EXCLUDE_PATHS="/HDD /NAS /proc /tmp /mnt /dev /sys /var/lock /var/tmp"

EXCLUDE_PARAMS=""
for p in ${EXCLUDE_PATHS}
do
    EXCLUDE_PARAMS="${EXCLUDE_PARAMS} --exclude ${p}"
done

sudo tar -cvpzf /HDD/systemBackup/backup.tar.gz --one-file-system $EXCLUDE_PARAMS

Consider adding a longer suffix to the split fragments by adding -a 3 to the split command. Otherwise we might get unexpected results (e.g. irregular suffix lengths). However, given that you are on a (probably small) raspberry pi, and that the default of a 2-char suffix will work fine for up to 576 files, this can likely be skipped here.

Restoring the full tree will be possible with

 find /path/to/backups -name "${date}-*" | sort | xargs cat | tar zvxf -

Please test your backup/restoration process at least once to make sure everything works as intended. Don't wait till disaster strikes to possibly realize that important parts of the system were accidentally left out.

Since you plan to do a backup to the cloud, protecting your data from nosy people might be a concern. Integrating gpg into the pipeline should be fairly straightforward.