Running a script recursively in subdirectories

Question

I got tired of running the same command in multiple directories, so I thought "there has to be a way to make this easier". The commands I was running was mostly git status, git stash list, hg summary, mvn clean test, and so on and so on...

So what I did was to create a bash script that you can pass a parameter to. It will then run the same script or command in all subdirectories, until it finds a place where the script returns exit status 0 (meaning that it was run successfully), then it stops going deeper in those subdirectories.

As I am not that used to bash-scripting, I am wondering if there's anything I can improve here. I would also like you to comment on the usability of this script. You are also welcome to make feature-requests, bug reports, or pull requests (my favorite!) on my bash-recursive repository on GitHub.

The script is:

#!/bin/bash

RUN_SCRIPT=$1

recurseCheck() {
  local f
  for f in $1/* ; do
    local PREV_DIR=`pwd`
    if [ -d $f ]; then
      cd $f
      RESULT=`eval $RUN_SCRIPT`
      local RESULT_CODE=$?
      if [ $RESULT_CODE -eq 0 ]; then
        echo "$f"
        echo "$RESULT"
        echo ""
      else
        recurseCheck $f
      fi
      cd $PREV_DIR
    fi
  done;
}

START_DIR=`pwd`
recurseCheck $START_DIR

Usage example:

recurse.sh "git status 2>&1"

Will output something like:

/home/zomis/gitstuff/a/Duga
On branch master
nothing to commit, working directory clean

/home/zomis/gitstuff/a/SudokuSharp
On branch master
nothing to commit, working directory clean

/home/zomis/gitstuff/c/Brainduck
On branch master
nothing to commit, working directory clean

/home/zomis/gitstuff/c/CodeReview-Shield
On branch master
nothing to commit, working directory clean

/home/zomis/gitstuff/c/SE-Scripts
On branch master
nothing to commit, working directory clean

The reason 2>&1 is there is because otherwise this, which goes to stderr would be printed:

fatal: Not a git repository (or any of the parent directories): .git

There is also an "example" directory on my GitHub repo which contains a script that I used a lot while testing, run ./recurse.sh /path/to/bash-recurse/example/filecheck.sh and it will output something like:

/home/zomis/bash-recurse/example/b/d
Running filecheck in /home/zomis/recursescript/example/b/d
Hello World
This is a recursive script

/home/zomis/bash-recurse/example/c
Running filecheck in /home/zomis/recursescript/example/c
OK in directory 'c'

Any comments welcome.

Answer 1

Check yourself before you wreck yourself...

shellcheck.net is an awesome website that checks your Bash syntax. Among other things, it will tell you to:

• Double-quote your path variables
• Use $(...) instead of the legacy `...` for sub-shells

Avoid cd inside scripts

cd inside scripts is troublesome. In case something goes wrong, you might find yourself in unexpected places, and wreak havoc.

But if you must, like in your example, there's a better way than saving the previous directory in PREV_DIR and then cd "$PREV_DIR" to go back, using a (...) sub-shell environment:

(
cd "$f" || break
RESULT=$(eval "$RUN_SCRIPT")
local RESULT_CODE=$?
if [ $RESULT_CODE -eq 0 ]; then
    echo "$f"
    echo "$RESULT"
    echo
else
    recurseCheck "$f"
fi
)

Notice that there is only one cd in, no cd out. This is because the cwd changes only affect the environment inside the (...).

Also notice the cd "$f" || break: if the cd command fails for some reason, you probably don't want to execute the rest of the block. You might go as far as exit at that point (shellcheck recommends it too) and investigate the troublesome directory.

Minor things

You had a pointless ; in done;, and instead of echo "" you can write simply echo.

Input validation

What if you run this script without arguments? eval "" exits with success, so it will simply print all directories. If that's ok, then everything's fine. If not, then you might want to add some input validation, for example:

if [ ! "$RUN_SCRIPT" ]; then
    echo usage: $0 'some script'
    exit 1
fi

Suggested implementation

Putting it together (coming to you in a Pull Request very soon):

#!/bin/bash

RUN_SCRIPT=$1

recurseCheck() {
    local f
    for f in $1/* ; do
        if [ -d "$f" ]; then
            (
            cd "$f" || break
            RESULT=$(eval "$RUN_SCRIPT")
            local RESULT_CODE=$?
            if [ $RESULT_CODE -eq 0 ]; then
                echo "$f"
                echo "$RESULT"
                echo
            else
                recurseCheck "$f"
            fi
            )
        fi
    done
}

START_DIR=$(pwd)
recurseCheck "$START_DIR"

Answer 2

You're reinventing-the-wheel. You can use

find . -execdir git status \; -print -prune 2>&1

The difference is that it doesn't buffer and defer the printing of stdout, so the directory name appears after the command output.

Answer 3

It looks like all your repos are only two levels deep. If that is the case, you might try:

for i in */* ; do [ -d "$i" ] && git --git-dir="$i"/.git --work-tree="$i" status 2>/dev/null && echo "$i" && echo ;  done

Credits to this SO answer and this blog post for pieces of this answer.

EDIT: Even simpler, by adapting 200_success's answer:

find . -type d -name '.git' -print -execdir git status \; -exec echo \;

This works based on the fact that all valid git repositories contain a directory titled .git. The -execdir flag to find makes the given command run within the directory that contains the found file. So, what this command does is:

find .              # find recursively, starting in the current directory...
-type d             # ...files of type "directory"...
-name '.git'        # ...whose name is '.git'...
-print              # ...then, for each file found, print the name of the file (the dir) you found, and...
-execdir git status \;      # ...from the directory containing that .git dir, run 'git status', and...
-exec echo \;       # ...finally 'echo' to insert a blank line in the output for readability.

This is great for any git commands you need to run. Unfortunately, because it depends on the presence of the .git directory to establish which subdirs to run in, it's non trivial to extend it to run arbitrary commands.

In other words, it doesn't actually "run the command recursively in all subdirs until it finds one that works", as requested in the question post. Instead, it uses the presence of a specific file (in this case a .git dir) to establish which subdirs to run in, and then runs the command only in those specific dirs.

I have never used hg, but from a quick google search it appears that hg repos are similar to git repos and contain a .hg directory in each valid repo. So this command could probably work for hg commands also. Just replace .git with .hg and git status with the hg command you want to run.

So if there is a single file or directory whose presence will guarantee you want to run the command (and that it will succeed) and whose absence will guarantee that you don't want to run the command (i.e. that the command will fail), then you can use that file's presence or absence as a proxy for checking the exit status of the command, and that is what I am doing with find. I hope that's clear enough.

Looking at your filecheck.sh example, I see that it does have such a file which can be used as a proxy as described above: ok.txt. So the command would be something like:

find . -name 'ok.txt' -print -execdir /full/path/to/filecheck.sh \; -exec echo \;

I hope this helps.