Bash script for user backups with BorgBackup

Question

I use BorgBackup to handle backups for my personal computer. The following is a bash script that creates backups of my system to various different targets. I have little experience with bash and hope to get some pointers regarding best practices and possible security issues/unanticipated edge cases.

One specific question is about the need to unset environment variables (in particular BORG_PASSPHRASE) like I have seen people doing (for example here) From my understanding this should not be necessary because the environment is only local.

Ideally I would also like to automatically ensure the integrity of the Borg repositories. I know there is borg check which I could run from time to time, but I am not sure if this is even necessary when using create which supposedly already makes sure the repository is in a healthy state?

Some notes regarding the code below:

• noti is a very simple script for notifications with i3status but could be replaced with anything else
• Some paths and names are replaces with dummies
• I can not exit properly on errors of borg create because I backup /etc where some files have wrong permissions and BorgBackup will throw errors
• The TODO comments in the code are things I may want to look into at some time, but the code works for now

Bash script:

#!/bin/bash

# User data backup script using BorgBackup with options for different target repositories
# usage: backup.sh <targetname>
# Each target must have a configuration file backup-<targetname>.conf that provides:
# - pre- and posthook functions
# - $repository - path to a valid borg repository or were one should be created
# - $backup - paths or exclusion paths
# - $pruning - borg pruning scheme
# Additional borg environment variables may be provided and will not be overwritten.
# Output is logged to LOGFILE="$HOME/.local/log/backup/<date>"

# INSTALLATION
# Place script and all target configs in $HOME/.local/scripts
# $HOME/.config/systemd/user/borg-backup.service
# ```
# [Unit]
# Description=Borg User Backup
# [Service]
# Environment=SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
# ExecStart=%h/.local/scripts/backup.sh target1
# Nice=19
# IOSchedulingClass=2
# IOSchedulingPriority=7
# ```
#
# $HOME/.config/systemd/user/borg-backup.timer
# ```
# [Unit]
# Description=Borg User Backup Timer
# [Timer]
# OnCalendar=*-*-* 8:00:00
# Persistent=true
# RandomizedDelaySec=10min
# WakeSystem=false
# [Install]
# WantedBy=timers.target
# ```
# $ systemctl --user import-environment PATH
# reload the daemon
# $ systemctl --user daemon-reload
# start the timer with
# $ systemctl --user start borg-backup.timer
# and confirm that it is running
# $ systemctl --user list-timer
# you can also run the service manually with
# $ systemctl --user start borg-backup


function error () {
    RED='\033[0;91m'
    NC='\033[0m'
    printf "${RED}%s${NC}\n" "${1}"
    notify-send -u critical "Borg" "Backup failed: ${1}"
    noti rm "BACKUP"
    noti add "BACKUP FAILED"
    exit 1
}


## Targets

if [ $# -lt 1 ]; then
    echo "$0: Missing arguments"
    echo "usage: $0 targetname"
    exit 1
fi

case "$1" in
    "target1"|"target2"|"target3")
        target="$1"
        ;;
    *)
        error "Unknown target"
        ;;
esac

# TODO abort if specified target is already running
# exit if borg is already running, maybe previous run didn't finish
#if pidof -x borg >/dev/null; then
#    error "Backup already running."
#fi


## Logging and notification

# notify about running backup
noti add "BACKUP"

# write output to logfile
log="$HOME/.local/log/backup/backup-$(date +%Y-%m-%d-%H%M%S).log"
exec > >(tee -i "$log")
exec 2>&1

echo "$target"


## Global Prehook

# create list of installed software
pacman -Qeq > "$HOME/.local/log/package_list.txt"

# create list of non backed up resources
ls -R "$HOME/misc/" > "$HOME/.local/log/resources_list.txt"

# create list of music titles
ls -R "$HOME/music/" > "$HOME/music/music_list.txt"


## Global Config

# set repository passphrase
export BORG_PASSCOMMAND="cat $HOME/passwd.txt"

compression="lz4"


## Target specific Prehook and Config

CONFIGDIR="$HOME/.local/scripts"
source "$CONFIGDIR"/backup-"$target".conf

# TODO make non mandatory and only run if it is defined
prehook || error "prehook failed"


## Borg

# TODO use env variables in configs instead?
# export BORG_REPO=$1
# export BORG_REMOTE_PATH=borg1
# borg create ::'{hostname}-{utcnow:%Y-%m-%dT%H:%M:%S}' $HOME

SECONDS=0
echo "Begin of backup $(date)."

borg create                    \
    --verbose                  \
    --stats                    \
    --progress                 \
    --compression $compression \
    "$repository"::"{hostname}-{utcnow:%Y-%m-%d-%H%M%S}" \
    $backup
#  || error "borg failed"

# use prune subcommand to maintain archives of this machine
borg prune                 \
    --verbose              \
    --list                 \
    --progress             \
    "$repository"          \
    --prefix "{hostname}-" \
    $pruning               \
  || error "prune failed"

echo "End of backup $(date). Duration: $SECONDS Seconds"


## Cleanup

posthook
noti rm "BACKUP"
echo "Finished"

exit 0

Example configuration file:

backup="$HOME
--exclude $HOME/movie
--exclude $HOME/.cache
--exclude $HOME/.local/lib
--exclude $HOME/.thumbnails
--exclude $HOME/.Xauthority
"

pruning="--keep-daily=6 --keep-weekly=6 --keep-monthly=6"

repository="/run/media/username/DRIVE"

prehook() { :; }  # e.g. mount drives/network storage

posthook() { :; }  # unmount ...

Answer 1

Your specific question

One specific question is about the need to unset environment variables (in particular BORG_PASSPHRASE) like I have seen people doing (for example here) From my understanding this should not be necessary because the environment is only local.

When you execute a Bash script with path/to/script.sh or bash path/to/script.sh, the script runs in a sub-shell, and cannot modify its caller environment. No matter if the script has export FOO=bar or unset FOO, these will not be visible in the caller shell. So the example you linked to, having BORG_PASSPHRASE="" (and followed by exit) is completely pointless.

Use an array for $backup

This setup will break if ever $HOME contains whitespace:

backup="$HOME
--exclude $HOME/movie
--exclude $HOME/.cache
--exclude $HOME/.local/lib
--exclude $HOME/.thumbnails
--exclude $HOME/.Xauthority
"

Even if that's unlikely to happen, it's easy enough and a good practice to make it safe, by turning it into an array:

backup=(
    "$HOME"
    --exclude "$HOME/movie"
    --exclude "$HOME/.cache"
    --exclude "$HOME/.local/lib"
    --exclude "$HOME/.thumbnails"
    --exclude "$HOME/.Xauthority"
)

And then in the calling command:

borg create                    \
    --verbose                  \
    --stats                    \
    --progress                 \
    --compression $compression \
    "$repository"::"{hostname}-{utcnow:%Y-%m-%d-%H%M%S}" \
    "${backup[@]}"

I suggest to do the same thing for $pruning too. Although there is no risk there (with the current value) of something breaking, it's a good practice to double-quote variables used on the command line. So it should be:

pruning=(--keep-daily=6 --keep-weekly=6 --keep-monthly=6)

And then when using it: "${pruning[@]}"

Use stricter input validation

Looking at:

if [ $# -lt 1 ]; then
    echo "$0: Missing arguments"
    echo "usage: $0 targetname"
    exit 1
fi

According to the usage message, only one argument is expected, but the validation logic allows any number above that, and will simply ignore them.

I suggest to strengthen the condition:

if [ $# != 1 ]; then

Output error messages to stderr

The error function, and a few other places print error messages on stdout. It's recommended to use stderr in these cases, for example:

if [ $# -lt 1 ]; then
    echo "$0: Missing arguments" >&2
    echo "usage: $0 targetname" >&2
    exit 1
fi

Don't use SHOUT_CASE for variables

It's not recommended to use all caps variable names, as these may clash with environment variables defined by system programs.

Use modern function declarations

The modern syntax doesn't use the function keyword, like this:

error() {
    # ...
}

Simplify case patterns

In this code:

case "$1" in
    "target1"|"target2"|"target3")
        target="$1"
        ;;

You can use glob patterns to simplify that expression:

case "$1" in
    "target"[1-3])
        target="$1"
        ;;

Why exit 0 at the end?

The exit 0 as the last statement is strange: Bash will do the same thing automatically when reaching the end of the script. You can safely remove it.

SECONDS

Wow, I completely forgot about this feature of Bash, very cool, thanks for the reminder!

Answer 2

    RED='\033[0;91m'
    NC='\033[0m'
    printf "${RED}%s${NC}\n" "${1}"

Please don't hard-code terminal-specific escape sequences like that. Standard output isn't necessarily a terminal, nor is it always the kind of terminal you're expecting. I recommend you use appropriate tput setaf commands to generate the appropriate sequences for the actual output stream in use. And shouldn't that message be going to the error stream, rather than the output stream?

---

I recommend set -u and probably also set -e to help avoid the common mistakes of continuing with unexpected state due to a typo or an unexpectedly failing command.