This is a simple tool to parallelize execution of multiple tasks in Bash, since env_parallel can't handle the environment size (or something of the sort - dies for mysterious reasons.) I'd love to hear your critique.

# Simple parallel executor. Takes a list of commands and executes each in its own child process. Assumes commands are distinctly named.
# Returns with 0 if all are successful. Otherwise, prints out their stdout/stderr outputs and returns with 1.

function simple_par () {

    local TMP_FILE="${WRITABLE_DIR}/simple_par_$(uuidgen)"
    touch $TMP_FILE

    for prog in "$@"
        local PROG_TMP="${TMP_FILE}_${prog}_$(uuidgen)"
            ( $prog 1>$PROG_TMP 2>&1 )
            echo "$?" >> $TMP_FILE
        ) &

    while true
        RESULT_COUNT=$(wc -l ${TMP_FILE} | awk '{ print $1 }')

        if [[ $RESULT_COUNT -eq $# ]]; then
            if [[ $(cat $TMP_FILE) == *"1"* ]]; then
                echo "Failed at least one command"
                for prog in "$@"
                    echo "OUTPUT FOR: $prog"
                    cat ${TMP_FILE}_${prog}_*
                    rm ${TMP_FILE}_${prog}_*

                rm $TMP_FILE
                return 1
                return 0

        sleep 5

1 Answer 1


A couple of things could be written better, let's go from top to bottom.

It's recommended to double-quote variables when used as paths, to avoid globbing and word-splitting. Even if you're certain that $TMP_FILE won't have any troublesome characters, it's a good practice to double-quote anyway:

touch "$TMP_FILE"

Similarly, you should double-quote $prog in $prog 1>$PROG_TMP 2>&1.

You can simplify for arg in "$@"; do loop as for arg; do.

In Bash, instead of prog 1> somewhere 2>&1 you can write prog &> somewhere.

Instead of this:

RESULT_COUNT=$(wc -l ${TMP_FILE} | awk '{ print $1 }')

You could write simpler without awk like this:


The trick is that when wc is used with stdin, it won't print a file name, only the number. Although RESULT_COUNT will have some whitespace padding, if you use it in [ ... ] without double-quoting, it will behave as expected.

Notice again that I double-quoted $TMP_FILE.

Don't use -eq in [[ ... ]], use == or = instead.

This is pretty ugly:

if [[ $(cat $TMP_FILE) == *"1"* ]]; then

A better way to achieve the same thing is using grep:

if grep -q 1 "$TMP_FILE"; then
  • \$\begingroup\$ Thanks a lot, @janos! I'm a bit of a bash dilettante, it's good to have good advice. Since then, I've rewritten the program to store the child processes' pids into an array/string, and wait on each of them. It seems to be a better use of the tools at hand. \$\endgroup\$
    – vivri
    Commented Dec 23, 2016 at 3:58
  • \$\begingroup\$ Cool, wanna post that up for review? ;-) \$\endgroup\$
    – janos
    Commented Dec 23, 2016 at 4:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.