I’ve written this bash backup script. It uses the --link-dest
option of rsync
; that way, the user have access to the backed data at any time stamped with relatively affordable data overhead. Any duplicated data should be hard linked; the overhead mostly come from the directory structure.
It’s mostly based on this very nice guide by Mike Rubel and various other contributors, as well as a few answers from Unix SE and other web reference.
The script is meant to be run at regular (typically, hourly) intervals with cron
and other scripts are in charge of safely keeping daily/weekly backups.
Of course, I want to minimize the size of the backups. I also want to delete older backups before newer ones. To do so, I build ${backups}
, an array of every backup1 sorted by modification date from newest to oldest. Thus ${backups[0]}
(if it exists) is the latest complete backup and ${backups[@:$n]}
for some integer $n
lists every backup but the $n
newest (from 0 to $n
- 1).
As always with bash scripts, I’m especially afraid of quoting issues, but any remark is welcome.
I in particular quite dislike how I use find
with both -mindepth
and maxdepth
, but couldn’t find any way around it.
Most of the “standard” commands, such as cut
, sort
or grep
, are provided by BusyBox 1.16.1 and may not have every option available on most recent Linux distribution. cut
, in particular, does not understand the -d
option, hence the ugly tr
trick.
#!/bin/bash
# Check if we are root (no one else should run this)
# ==================================================
if (( $(id -u) != 0 )); then
echo "/ ! \ Only root can run this script. Backup cancelled." >&2
exit 3
fi
# Functions
# =========
# Given a date (or a placeholder), returns the corresponding hourly backup name
function scheme {
local token="$@"
echo "hourly ${token}"
}
# Parameters
# ==========
# password_file=/etc/backup/passwd # Network yet untested
backup_directory=/path/to/backups # ABSOLUTE PATH required
source_directory=/path/to/data
backup_count=24 # number of backups to keep
# Name new daily backup
# =====================
new_backup="${backup_directory}"/$(scheme $(date +"%-d-%m-%Y a %Hh%M") )
# Check that we can run
# =====================
# Check that we don’t overwrite anything
if [[ -e "${new_backup}" ]]; then
echo "/ ! \ ${new_backup} already exists! We don’t want to overwrite it; backup cancelled." >&2
exit 4
fi
# Create the directory which contains all the backups if it doesn’t exist yet
if [[ ! -e "${backup_directory}" ]]; then
echo "Creating directory ${backup_directory}"
mkdir -p "${backup_directory}"
elif [[ ! -d "${backup_directory}" ]]; then
echo "/ ! \ Destination ${backup_directory} already exists but is not a directory! Backup cancelled." >&2
exit 4
fi
# Create a temporary working directory
# ====================================
temp_backup=$(mktemp -d -p "${backup_directory}")
# Manage previous backups
# =======================
# List every previous backup and put it into an array
backups=()
while read -r -d ''; do
backups+=("${REPLY}")
done < <( find "${backup_directory}" -mindepth 1 -maxdepth 1 -name "$(scheme \*)" -printf "%A@:%p\0" | \
sort -z -t: -n -r | \
tr '\n\0' '\0\n' | cut -d: -f2 - | tr '\n\0' '\0\n' \
)
# If it exists, select the latest backup as a reference for rsync --link-dest
if (( ${#backups[@]} > 0 )); then
latest_backup="${backups[0]}"
else
latest_backup=""
fi
# Compute the backups to remove
# We add one backup before cleaning up
# Thus, we keep $backup_count - 1 from the ${backups[@]}
old_backups=("${backups[@]:${backup_count} - 1}")
# Cleanup function
# ================
# We now have everything we need to define a cleanup function
# It will be called only if the backup succeeds
function cleanup {
echo
echo "Cleaning up"
echo "==========="
echo
if (( ${#old_backups[@]} > 0 )); then
echo "Deleting ${#old_backups[@]} backup(s)!"
echo
# echo rm -rf "${old_backups[@]}"
(set -x; rm -rf "${old_backups[@]}")
else
echo "There is nothing to delete."
fi
}
# User feedback
# =============
echo "Backing up ${source_directory}"
echo "Backing up ${source_directory}" | sed "s/./=/g"
echo
echo "New backup: ${new_backup}"
# Setting up rsync options
# ========================
RSYNC_FLAGS=("--archive" "--stats")
# Set rsync --password-file if the matching variable is defined and
# we are using rsync (::) **YET UNTESTED**
if [[ "${password_file}" != "" && "${source_directory}" =~ "::" ]]; then
RSYNC_FLAGS+=("--password-file=${password_file}")
fi
# Use rsync to backup. If a previous backup exists,
# uses --link-dest to hard link to it.
if [[ "${latest_backup}" != "" ]]; then
echo "Previous backup: ${latest_backup}"
RSYNC_FLAGS+=("--link-dest=${latest_backup}")
else
echo "This is the first backup ever, it might take a while."
fi
echo
# Backing-up
# ==========
# TODO Check if something was actually written before creating a new backup
# TODO Add an exclusion file
(set -x; rsync "${RSYNC_FLAGS[@]}" "${source_directory}" "${temp_backup}") && \
(set -x; mv "${temp_backup}" "${new_backup}") && cleanup
echo
- Actually the “name of the repository which contains the backup”, of course.
find
when there is no other way. The syntax is just f***ing bizarre to me and thus highly error-prone. Good job on thorough use of double-quotes. A|
alone will also do line continuation in bash. \$\endgroup\$find
is very welcome \$\endgroup\$${backup}
is meaningful at that wasn’t clear in the previous version. \$\endgroup\$