5
\$\begingroup\$

This is a script designed to make an inventory file which records the contents of a directory allowing verification that a backup has the same data. The same script can also

A full github repo is available at https://github.com/mikedlr/backup-tools including various test cases using python behave.

This is written as a shell script for reasonably rapid development and maxium ease of rapid portability. If it's useful for others I might rewrite in another language/other languages as well.

Comments appreciated especially for

  • style
  • safety / correctness
  • portability

I have deleted a few irrelevant comments from the code to save your reading time. I'd happily make this portable to non bash shells but I think the use of arrays is pretty unavoidable.

#!/bin/bash

# inventory - record an inventory with checksums of the files in a directory
# Copyright (C) 2016 Michael De La Rue

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU Affero General Public License for more details.

# You should have received a copy of the GNU Affero General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

TEMP=$(getopt -o ho:i:x:cv --long help,output:,input:,check,verbose -n 'inventory.sh' -- "$@" )

if [ $? != 0 ] ; then echo "Argument parsing fail; terminating..." >&2 ; exit 1 ; fi

eval set -- "$TEMP"

usage() { 
  cat <<EOF 
inventory - create or verify an inventory of a directory, typically for backup verification
 -h --help  - output usage information
 -c --check - read inventory file and verify directory
 -v --verbose - verbose output
 -o --output <file> - output to <file>
 -x --exclude <file> - exclude filenames matching expressions in <file>
 -i --input <file> - use <file> for input (as an inventory file), together with -c
EOF
}

OFILE=""
IFILE=""
XFILE=""
CHECK="false"
VERBOSE="false"
while true ; do
    case "$1" in
        -h|--help) usage; exit 0;;
        -c|--check) CHECK="true" ; shift ;;
        -v|--verbose) VERBOSE="true" ; shift ;;
        -o|--output)
        if [ "" != "$OFILE" ]
        then
            echo "Only one output file allowed.  Terminating" >&2
            exit 1
        fi
        OFILE=$2; shift 2 ;;
        -i|--input)
        if [ "" != "$IFILE" ]
        then
            echo "Only one input file allowed.  Terminating" >&2
            exit 1
        fi
        IFILE=$2; shift 2 ;;
        -x|--exclude)
        if [ "" != "$XFILE" ]
        then
            echo "Only one exclude file currently allowed.  Terminating" >&2
            exit 1
        fi
        XFILE=$2; shift 2 ;;
        --) shift ; break ;;
        *) echo "Internal error!" ; exit 1 ;;
    esac
done

if [ "" = "$1" ]
then
echo "must have at least one directory argument to run inventory on" >&2
exit 1
fi

if [ "false" = "$CHECK" ]
then
if [ "" != "$IFILE" ]
then
    echo "cannot give input file when generating inventory" >&2
    exit 1
fi
fi


check_inventory ()
{
local check_dir="$1"
if [ "false" = "$VERBOSE" ]
then
    SHAOPTS="--check --quiet"
else
    SHAOPTS="--check"
fi

if [ "" != "$OFILE" ]
then
    echo "cannot give output file when checking inventory" >&2
    exit 1
fi
( if [ "" = "$IFILE" ]
then
    head -n-2
else
    head -n-2 "$IFILE"
fi )| tail -n+2 | (cd "$check_dir"; sha384sum $SHAOPTS - )
SHASUMRES=$?
exit $SHASUMRES
}

create_inventory ()
{
local record_dir="$1"
set -e
local TEMPFILE
TEMPFILE=$(mktemp)
( echo inventoryfile-0 at "$(date --rfc-3339=seconds --utc)" directory: "$(readlink -f "$1")" ) > "$TEMPFILE"
( cd "$record_dir"
    {   find . -type f "${FINDFILTER[@]}" -exec sha384sum {} + | sort -k 2
    echo -----------------------------------------------
    } >> "$TEMPFILE"
    #split to two lines to avoid reading and writing at the same time
    FOOT="inventory checksum $(sha384sum "$TEMPFILE" | sed 's/ .*//')"
    echo "$FOOT" >>  "$TEMPFILE"
)
if [ "" = "$OFILE" ]
then
   cat "$TEMPFILE"
else
   mv "$TEMPFILE" "$OFILE"
fi
}

if [  "true" = "$CHECK" ]
then
check_inventory "$1"
fi

FINDFILTER=()

if [ "" != "$XFILE" ]
then
if [ ! -r "$XFILE" ] 
then
    echo "$XFILE no such file or directory" >&2
    exit 5
fi
while read line           
do           
    #TODO: comments
    if [ "" != "$line" ]
    then
       FINDFILTER=("${FINDFILTER[@]}" -not -path "$line")
    fi
done <"$XFILE"
fi

create_inventory "$1"
\$\endgroup\$
1
\$\begingroup\$

Use a fatal function

This kind of code appears in many places:

echo "Only one output file allowed.  Terminating" >&2
exit 1

I suggest to introduce a fatal function:

fatal() {
    echo "$*  Terminating" >&2
    exit 1
}

So that you can replace the example above with a simpler single line:

fatal "Only one output file allowed."

Checking for empty value

Instead of this:

if [ "" != "$IFILE" ]

A shorter way to accomplish the same:

if [ ! "$IFILE" ]

Appending to arrays

Instead of this:

FINDFILTER=("${FINDFILTER[@]}" -not -path "$line")

The correct way to append to an array is using the += operator:

FINDFILTER+=(-not -path "$line")

Pointless local variable

Instead of this:

SHASUMRES=$?
exit $SHASUMRES

You could skip the local variable:

exit $?

Too compact writing style

Instead of this:

if [ $? != 0 ] ; then echo "Argument parsing fail; terminating..." >&2 ; exit 1 ; fi

It will be much more readable to split to multiple lines:

if [ $? != 0 ]; then
    echo "Argument parsing fail; terminating..." >&2
    exit 1
fi

Inconsistent indentation

I see multiple indentation styles for nested code blocks (for example if-statements): indent with 2 spaces, 4 spaces, or none. I suggest to indent consistently (I like 4 spaces).

\$\endgroup\$
  • \$\begingroup\$ I've accepted this as an answer since there's plenty for me to work on. Thanks @janos. Obviously any more answers appreciated. \$\endgroup\$ – Michael Aug 1 '16 at 22:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.