Try every option for a command and make anagrams out of the successful ones

Question

What I have: I am trying to find what words you can make out of Linux command options like ls -cat or ps -elf. To do that I am iterating through an array of 50 commands, trying to run each with flags -[a-z] and then making anagrams out of the successful options. It is working and here is what I have:

#!/bin/bash

# Pull in a list of common Linux commands
commandList=( $(
  curl 'http://www.thegeekstuff.com/2010/11/50-linux-commands/' 2> /dev/null \
    | grep -o '<h3>\([0-9].*[a-z][a-z][a-z].*\)</h3>' \
    | awk '{ print $(NF-2) }' \
    | sed -nr '/rm|wget|less|shutdown|<h3>/!p'
) );

# Pipe successful "$command -$option" pairs to 'an' to generate anagrams
for command in ${commandList[@]} ; do
    (for option in {a..z} ; do
        timeout -k 5 5 "$command" -"$option" > /dev/null 2>&1;
            if (($? == 0)); then
                printf "$option" | tr -d '\n'
            else
                    :
            fi
    done) | xargs an -w -d saneWordlist -m 3 2> /dev/null \
          | sed 's/^/\ '$command' -/' >> commandOptions.log
done    

Self Criticism: I know it is unreliable to rely on exit codes that are not at all consistent and just hoping the devs used 0 to indicate success. I also know I abused sed and awk a lot. I also didn't try to write this in a very POSIX-y way.

What I want help on: I am not too worried about the web scraping part, I know it is cheap and messy. But, how can I improve the core section where I misuse sed and awk so much? I know there cleaner ways to accomplish what I want, but can't think of anything.

Answer 1

Some of the commands in the web scraping part can be simplified.

Instead of awk '{ print $(NF-2) }', why not simply awk '{ print $2 }'?

Instead of sed -nr '/rm|wget|less|shutdown|<h3>/!p', it would be simpler and more portable to use grep -vE 'rm|wget|less|shutdown|<h3>'.

Using a Bash array is a bit overkill, because the output of the pipeline is guaranteed to be one word per line, no other white space. So using a simple variable and word splitting should do the job, and simpler:

commandList=$(
  curl 'http://www.thegeekstuff.com/2010/11/50-linux-commands/' 2> /dev/null \
    | grep -o '<h3>\([0-9].*[a-z][a-z][a-z].*\)</h3>' \
    | awk '{ print $2 }' \
    | grep -vE 'rm|wget|less|shutdown|<h3>'
)

for command in $commandList; do

---

Many improvements are possible here:

timeout -k 5 5 "$command" -"$option" > /dev/null 2>&1;
    if (($? == 0)); then
        printf "$option" | tr -d '\n'
    else
            :
    fi

In Bash, you can use &>/dev/null to redirect both stdout and stderr.

Since $option only contains a single letter and printf doesn't print a trailing newline like echo, the tr command there is pointless. The double-quotes around $option are also unnecessary. The empty else branch should be removed.

Instead of using a condition on $?, you could use the command itself directly:

if timeout -k 5 5 "$command" -$option &> /dev/null; then
    printf $option
fi

---

You could rewrite the final sed simpler using double-quotes:

sed "s/^/ $command -/" >> commandOptions.log

I also removed a pointless \ from in front of a space within the quoted expression.