Shell script to download and extract a tarball from GitHub

Question

I don't want anything to execute if any preceding step fails:

#!/bin/sh

file="v0.9"
renamed=$file".tar.gz"
dir="utils/external/firepython/"
location="https://github.com/darwin/firepython/tarball/$file"

wget --no-check-certificate $location --output-document=$renamed && \
mkdir -p $dir && \
gunzip $renamed && \
echo "extracting to $dir" && \
tar xf $file".tar" --directory $dir --strip-components 1 && \
echo "Cleaning up..." && \
rm -r $file".tar" && \
echo "Done"

Answer 1

You're looking for set -e. From POSIX:

-e When this option is on, if a simple command fails for any of the reasons listed in Consequences of Shell Errors or returns an exit status value >0, and is not part of the compound list following a while, until, or if keyword, and is not a part of an AND or OR list, and is not a pipeline preceded by the ! reserved word, then the shell shall immediately exit.

In other words, plain commands cause the shell to exit if they fail. (You can use something like command || true to allow command to return nonzero.) If you need to perform some cleanup, you can set a trap for the EXIT pseudo-signal.

And better double-quote all your variable expansions. That way your script won't fail horribly if you ever point it at a directory or an URL containing ? or * or a space.

dry -r
wget --no-check-certificate "$location" --output-document="$renamed"
mkdir -p "$dir"
gunzip "$renamed"
echo "extracting to $dir"
tar xf "$file.tar" --directory "$dir" --strip-components 1
echo "Cleaning up..."
rm -r "$file.tar"
echo "Done"

---

Another useful shell idiom to pass optional arguments to a shell script without hassle is to set variables only if they're unset. That way you can pass arguments through the environment, e.g. file=v0.9.1 myscript.

: "${file=v0.9}"
: "${renamed=$file.tar.gz}"
: "${dir=utils/external/firepython/}"
: "${location=https://github.com/darwin/firepython/tarball/$file}"

Answer 2

@Gilles answer about set -e is right on target. An alternative way if only one or two commands in a script are must-haves, you can use important-command || exit as a way to drop out of the script if any one command fails.

I often include an auxilary function in my scripts called 'flunk' that handles any cleanup that needs to be done if something fails. It might look something like this:

function flunk () {
    echo "SCRIPT FAILED: $1"
    rm $TMPFILES
    exit 1
}

command
important-command || flunk "Could not do X"
command

Answer 3

You are probably may use pipe instead of creating and deleting downloaded file:

mkdir -p "$dir"
echo "extracting to $dir"
wget --no-check-certificate "$location" --output-document=- |
    tar zxvf - --strip-components 1 --directory="$dir"
echo "Done"

Answer 4

Sometimes using && or || can induce a race condition, so it may be better to rewrite this with if statements instead, as I do not see at a glance why exit status with the one-liner/"pipeline" approach is not doing it for you... And you can always test if the extracted files are present with like test -f "dir/whatever.file" || exit 1 or something.

Answer 5

Just a reminder, these kind of scripts sometimes tend to fail if wget or gunzip is not installed. This might happen in some minimal installations.