Simple Bash Parallel Tool (env_parallel dies on big env)

Question

This is a simple tool to parallelize execution of multiple tasks in Bash, since env_parallel can't handle the environment size (or something of the sort - dies for mysterious reasons.) I'd love to hear your critique.

#
# Simple parallel executor. Takes a list of commands and executes each in its own child process. Assumes commands are distinctly named.
# Returns with 0 if all are successful. Otherwise, prints out their stdout/stderr outputs and returns with 1.

function simple_par () {

    local TMP_FILE="${WRITABLE_DIR}/simple_par_$(uuidgen)"
    touch $TMP_FILE

    for prog in "$@"
    do
        local PROG_TMP="${TMP_FILE}_${prog}_$(uuidgen)"
        (
            ( $prog 1>$PROG_TMP 2>&1 )
            echo "$?" >> $TMP_FILE
        ) &
    done

    while true
    do
        RESULT_COUNT=$(wc -l ${TMP_FILE} | awk '{ print $1 }')

        if [[ $RESULT_COUNT -eq $# ]]; then
            if [[ $(cat $TMP_FILE) == *"1"* ]]; then
                echo "Failed at least one command"
                for prog in "$@"
                do
                    echo "OUTPUT FOR: $prog"
                    cat ${TMP_FILE}_${prog}_*
                    rm ${TMP_FILE}_${prog}_*
                done

                rm $TMP_FILE
                return 1
            else
                return 0
            fi
        fi

        sleep 5
    done
}

Answer 1

A couple of things could be written better, let's go from top to bottom.

---

It's recommended to double-quote variables when used as paths, to avoid globbing and word-splitting. Even if you're certain that $TMP_FILE won't have any troublesome characters, it's a good practice to double-quote anyway:

touch "$TMP_FILE"

Similarly, you should double-quote $prog in $prog 1>$PROG_TMP 2>&1.

---

You can simplify for arg in "$@"; do loop as for arg; do.

---

In Bash, instead of prog 1> somewhere 2>&1 you can write prog &> somewhere.

---

Instead of this:

RESULT_COUNT=$(wc -l ${TMP_FILE} | awk '{ print $1 }')

You could write simpler without awk like this:

RESULT_COUNT=$(wc -l < "$TMP_FILE")

The trick is that when wc is used with stdin, it won't print a file name, only the number. Although RESULT_COUNT will have some whitespace padding, if you use it in [ ... ] without double-quoting, it will behave as expected.

Notice again that I double-quoted $TMP_FILE.

---

Don't use -eq in [[ ... ]], use == or = instead.

---

This is pretty ugly:

if [[ $(cat $TMP_FILE) == *"1"* ]]; then

A better way to achieve the same thing is using grep:

if grep -q 1 "$TMP_FILE"; then