4
\$\begingroup\$

Recently, I've been making and refining installer and rescue images to boot from USB media, in the form of CD-ROM or hard disk images. After a few occasions where I re-ran the command after removing the media (and ending up with a large file in /dev/disk/by-id/ instead of writing to a device), I wrote a small wrapper to check that the target is a block device.

Since then, it's grown features - notably to take input direct from network sources (e.g. a build server) to save making a local copy first. In these cases, I want to be able to sudo write-image but still make the network connection as non-root user (partly as good practice, but also so that I can use my ~/.ssh/config and private keys in the normal way).

The usage() function ought to explain how it's intended to be used; if I need to explain further, then it's a sign that my documentation needs improving.

The reason that this needs Bash rather than POSIX shell is to use command substitution process substitution. The use of a shell function in process substitution seems to be particularly non-portable, but it works for me with Bash 4.4.

#!/bin/bash

set -eu

die() { echo "$@" >&2; exit 1; }

usage() {
    cat <<END
usage: $0 [source] <usb-device>
The source can be
  * a filename
  * a URL
  * ssh:[user@]<host>:<file>  (must be passwordless)
  * '-' or absent (for piping from standard input)
END
}

as_real_user() {
    exec ${SUDO_USER+sudo -n -H -u "$SUDO_USER" }"$@"
}

case $# in
    1)
        case "$1" in
            --help|-h)
                usage
                exit 0
                ;;
        esac
        ;;
    2)
        case "$1" in
            ssh:*)
                file="${1#*:}"
                host="${file%%:*}"
                file="${file#*:}"
                exec < <(as_real_user ssh "$host" cat "$file")
                ;;
            *://*)
                exec < <(as_real_user wget -q -O - "$url")
                ;;
            -)
                # standard input is already connected
                ;;
            *)
                test -e "$1" || die "$1: not found"
                test -f "$1" || die "$1: not a plain file"
                test -r "$1" || die "$1: not readable"
                exec < "$1"
                ;;
        esac
        shift
        ;;
    *)
        usage
        exit 1;
esac

test -b "$1" || die "$1: not a block device"
test -w "$1" || die "$1: not writable"
cut -d' ' -f1 /proc/mounts | grep -Fxq $(realpath "$1") && die "$1: is mounted"
outfile="of=$1"

# Explanation
# * nocreat - just checking, given above tests of destination
# * fsync - don't return until image is fully written
# * dsync,bs,status - show update every 1 MB written
ddflags="conv=nocreat,fsync oflag=dsync bs=1M status=progress"

exec dd $ddflags $outfile
\$\endgroup\$
2
\$\begingroup\$

Quoting

Some recommended double-quotes were missed here:

cut -d' ' -f1 /proc/mounts | grep -Fxq $(realpath "$1") && die "$1: is mounted"

Should be like this:

cut -d' ' -f1 /proc/mounts | grep -Fxq "$(realpath "$1")" && die "$1: is mounted"

This will break for example if $outfile contains spaces:

exec dd $ddflags $outfile

You could make it safe by using an array:

ddargs=(conv=nocreat,fsync oflag=dsync bs=1M status=progress "of=$1")

exec dd "${ddargs[@]}"

Safety tip

set -eu is a nice safeguard, I also like to add -o pipefail:

set -euo pipefail

Cosmetic issues

There is an unnecessary trailing ; in exit 1;

\$\endgroup\$
  • 1
    \$\begingroup\$ That's a really helpful review, and I've used all your recommendations. Thank you! \$\endgroup\$ – Toby Speight Mar 20 '18 at 8:22
1
\$\begingroup\$

An oversight

I forget to redirect usage message to standard error stream here:

*)
    usage >&2
    exit 1;

A typo

I obviously didn't test the network URI branch after I removed the $url variable:

        *://*)
            exec < <(as_real_user wget -q -O - "$url")
            ;;

That should be $1, not $url.

Better checking of SSH pathnames

We never check that $file can be split into host and path parts. That's easy to do:

        ssh:*:*)
            file="${1#*:}"
            host="${file%%:*}"
            file="${file#*:}"
            exec < <(as_real_user ssh "$host" cat "$file")
            ;;
        ssh:*)
            die "Malformed remote name - should be ssh:[user@]<host>:<file>"
            ;;
\$\endgroup\$
1
\$\begingroup\$

I'm unaware of how the

use of a shell function in command substitution seems to be particularly non-portable

but as far as I know, command substitution can be written as either `...` or $(...), and the latter is less portable while both are specified by POSIX. Process substitution <(...), on the other hand, originated in the Korn shell and is indeed non-portable.

I would probably remove set -eu from the script or limit its effect to a small block of code after a certain amount of testing. The reason is, briefly, that this feature is neither robust nor portable (read this article for more details), and since you're already doing some error checking yourself, you don't need to have it.

Just a comment, but test is a very archaic way to perform many file and string tests. It was the only way to do that function before [ was introduced as a shell builtin long, long back in the distant past. You would probably see this usage more often in Autoconf code only because of the limitations of M4 syntax:

The test program is the way to perform many file and string tests. It is often invoked by the alternate name ‘[’, but using that name in Autoconf code is asking for trouble since it is an M4 quote character.

I'm not sure which of the following is more portable, if they even achieve the exact same thing: cut -d' ' -f1 /proc/mounts or df -Pa | cut -d' ' -f1 | tail -n +2. I'm just posting it here as food for thoughts. Oh, and realpath the command isn't portable.

\$\endgroup\$
  • \$\begingroup\$ It's invoking shell functions in the substitution that doesn't work in other shells - presumably because they don't export the functions to the environment where the child shell can inherit them. I've read the article on set -e, and still prefer to have it (partly as a fallback in case I miss a check). I do tend to prefer test over its shorter spelling, but that's probably just a style thing. It just seems a little less cryptic. And thanks for the heads-up on realpath - it's probably okay here (Debian) where realpath and readlink are both in the (Essential) coreutils package. \$\endgroup\$ – Toby Speight Apr 12 '18 at 15:02
  • \$\begingroup\$ Er, yes, I did mean process substution and not merely command substitution. \$\endgroup\$ – Toby Speight Apr 12 '18 at 15:39
  • \$\begingroup\$ @TobySpeight Ah, OK, I was just about to ask you to elaborate on that as I didn't see any use of a shell function in command substitution in your code, and I would be very curious to know the details if this were true. After all, one never fully learns everything about UNIX shells, maybe except for Stéphane Chazelas. \$\endgroup\$ – Gao Apr 12 '18 at 15:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.