7
\$\begingroup\$

I got tired of running the same command in multiple directories, so I thought "there has to be a way to make this easier". The commands I was running was mostly git status, git stash list, hg summary, mvn clean test, and so on and so on...

So what I did was to create a bash script that you can pass a parameter to. It will then run the same script or command in all subdirectories, until it finds a place where the script returns exit status 0 (meaning that it was run successfully), then it stops going deeper in those subdirectories.

As I am not that used to bash-scripting, I am wondering if there's anything I can improve here. I would also like you to comment on the usability of this script. You are also welcome to make feature-requests, bug reports, or pull requests (my favorite!) on my bash-recursive repository on GitHub.

The script is:

#!/bin/bash

RUN_SCRIPT=$1

recurseCheck() {
  local f
  for f in $1/* ; do
    local PREV_DIR=`pwd`
    if [ -d $f ]; then
      cd $f
      RESULT=`eval $RUN_SCRIPT`
      local RESULT_CODE=$?
      if [ $RESULT_CODE -eq 0 ]; then
        echo "$f"
        echo "$RESULT"
        echo ""
      else
        recurseCheck $f
      fi
      cd $PREV_DIR
    fi
  done;
}

START_DIR=`pwd`
recurseCheck $START_DIR

Usage example:

recurse.sh "git status 2>&1"

Will output something like:

/home/zomis/gitstuff/a/Duga
On branch master
nothing to commit, working directory clean

/home/zomis/gitstuff/a/SudokuSharp
On branch master
nothing to commit, working directory clean

/home/zomis/gitstuff/c/Brainduck
On branch master
nothing to commit, working directory clean

/home/zomis/gitstuff/c/CodeReview-Shield
On branch master
nothing to commit, working directory clean

/home/zomis/gitstuff/c/SE-Scripts
On branch master
nothing to commit, working directory clean

The reason 2>&1 is there is because otherwise this, which goes to stderr would be printed:

fatal: Not a git repository (or any of the parent directories): .git

There is also an "example" directory on my GitHub repo which contains a script that I used a lot while testing, run ./recurse.sh /path/to/bash-recurse/example/filecheck.sh and it will output something like:

/home/zomis/bash-recurse/example/b/d
Running filecheck in /home/zomis/recursescript/example/b/d
Hello World
This is a recursive script

/home/zomis/bash-recurse/example/c
Running filecheck in /home/zomis/recursescript/example/c
OK in directory 'c'

Any comments welcome.

\$\endgroup\$
8
\$\begingroup\$

Check yourself before you wreck yourself...

shellcheck.net is an awesome website that checks your Bash syntax. Among other things, it will tell you to:

  • Double-quote your path variables
  • Use $(...) instead of the legacy `...` for sub-shells

Avoid cd inside scripts

cd inside scripts is troublesome. In case something goes wrong, you might find yourself in unexpected places, and wreak havoc.

But if you must, like in your example, there's a better way than saving the previous directory in PREV_DIR and then cd "$PREV_DIR" to go back, using a (...) sub-shell environment:

(
cd "$f" || break
RESULT=$(eval "$RUN_SCRIPT")
local RESULT_CODE=$?
if [ $RESULT_CODE -eq 0 ]; then
    echo "$f"
    echo "$RESULT"
    echo
else
    recurseCheck "$f"
fi
)

Notice that there is only one cd in, no cd out. This is because the cwd changes only affect the environment inside the (...).

Also notice the cd "$f" || break: if the cd command fails for some reason, you probably don't want to execute the rest of the block. You might go as far as exit at that point (shellcheck recommends it too) and investigate the troublesome directory.

Minor things

You had a pointless ; in done;, and instead of echo "" you can write simply echo.

Input validation

What if you run this script without arguments? eval "" exits with success, so it will simply print all directories. If that's ok, then everything's fine. If not, then you might want to add some input validation, for example:

if [ ! "$RUN_SCRIPT" ]; then
    echo usage: $0 'some script'
    exit 1
fi

Suggested implementation

Putting it together (coming to you in a Pull Request very soon):

#!/bin/bash

RUN_SCRIPT=$1

recurseCheck() {
    local f
    for f in $1/* ; do
        if [ -d "$f" ]; then
            (
            cd "$f" || break
            RESULT=$(eval "$RUN_SCRIPT")
            local RESULT_CODE=$?
            if [ $RESULT_CODE -eq 0 ]; then
                echo "$f"
                echo "$RESULT"
                echo
            else
                recurseCheck "$f"
            fi
            )
        fi
    done
}

START_DIR=$(pwd)
recurseCheck "$START_DIR"
\$\endgroup\$
0
\$\begingroup\$

You're . You can use

find . -execdir git status \; -print -prune 2>&1

The difference is that it doesn't buffer and defer the printing of stdout, so the directory name appears after the command output.

\$\endgroup\$
  • \$\begingroup\$ Thanks, but no thanks. I'd like to know which directory it is being run in and not run it in more subdirectories than necessary. That said, it's still very likely that I am reinventing a wheel, just gotta find the original wheel. \$\endgroup\$ – Simon Forsberg Nov 28 '15 at 16:59
  • \$\begingroup\$ If you wanted to suppress the printing of stderr, wouldn't you use 2>/dev/null? \$\endgroup\$ – 200_success Nov 28 '15 at 17:05
  • \$\begingroup\$ I did not think of forwarding to dev/null. As I only output stdout if the exit status is 0, it has the same effect in this case. \$\endgroup\$ – Simon Forsberg Nov 28 '15 at 17:08
  • \$\begingroup\$ Your revised answer does print the directory, but it still prints multiple outputs for each git repository which is something I very much want to avoid. \$\endgroup\$ – Simon Forsberg Nov 28 '15 at 17:10
  • \$\begingroup\$ Very nice answer, actually. I adapted it slightly in mine to avoid extra runs of git status and actually avoid the need for redirecting stderr. \$\endgroup\$ – Wildcard Nov 29 '15 at 8:23
0
\$\begingroup\$

It looks like all your repos are only two levels deep. If that is the case, you might try:

for i in */* ; do [ -d "$i" ] && git --git-dir="$i"/.git --work-tree="$i" status 2>/dev/null && echo "$i" && echo ;  done

Credits to this SO answer and this blog post for pieces of this answer.

EDIT: Even simpler, by adapting 200_success's answer:

find . -type d -name '.git' -print -execdir git status \; -exec echo \;

This works based on the fact that all valid git repositories contain a directory titled .git. The -execdir flag to find makes the given command run within the directory that contains the found file. So, what this command does is:

find .              # find recursively, starting in the current directory...
-type d             # ...files of type "directory"...
-name '.git'        # ...whose name is '.git'...
-print              # ...then, for each file found, print the name of the file (the dir) you found, and...
-execdir git status \;      # ...from the directory containing that .git dir, run 'git status', and...
-exec echo \;       # ...finally 'echo' to insert a blank line in the output for readability.

This is great for any git commands you need to run. Unfortunately, because it depends on the presence of the .git directory to establish which subdirs to run in, it's non trivial to extend it to run arbitrary commands.

In other words, it doesn't actually "run the command recursively in all subdirs until it finds one that works", as requested in the question post. Instead, it uses the presence of a specific file (in this case a .git dir) to establish which subdirs to run in, and then runs the command only in those specific dirs.

I have never used hg, but from a quick google search it appears that hg repos are similar to git repos and contain a .hg directory in each valid repo. So this command could probably work for hg commands also. Just replace .git with .hg and git status with the hg command you want to run.

So if there is a single file or directory whose presence will guarantee you want to run the command (and that it will succeed) and whose absence will guarantee that you don't want to run the command (i.e. that the command will fail), then you can use that file's presence or absence as a proxy for checking the exit status of the command, and that is what I am doing with find. I hope that's clear enough.

Looking at your filecheck.sh example, I see that it does have such a file which can be used as a proxy as described above: ok.txt. So the command would be something like:

find . -name 'ok.txt' -print -execdir /full/path/to/filecheck.sh \; -exec echo \;

I hope this helps.

\$\endgroup\$
  • \$\begingroup\$ No, it is not that simple that all my repos are only two levels deep. I want it completely recursive. Your second version is better, but how can it be changed to run another script? (such as hg summary, or the filecheck.sh provided in my example directory) \$\endgroup\$ – Simon Forsberg Nov 29 '15 at 12:57
  • \$\begingroup\$ Awful quick with the downvotes, aren't you? Well, anyway—if the command fails to run on a particular directory, should it be attempted on all directories within that directory, or only until one of them succeeds? (I would think all or it would miss most of your repos—but you tell me.) \$\endgroup\$ – Wildcard Nov 29 '15 at 19:46
  • \$\begingroup\$ When answers suggest something that doesn't fulfill my requirements, I downvote. If it fails to run on one directory, it should run it recursively inside that directory until it finds a directory that it runs in. If ./a/b and ./a/c are successes, it will run in both does directories but not in ./a/b/subdir and ./a/c/subdir \$\endgroup\$ – Simon Forsberg Dec 2 '15 at 20:57
  • \$\begingroup\$ I see. In that case janos's answer is exactly what you need. And I suppose my answer is more suited to another site than codereview—for someone looking for "how to accomplish x" rather than "review my code to accomplish x". So I get your point. \$\endgroup\$ – Wildcard Dec 2 '15 at 21:47
  • \$\begingroup\$ If your answer would have met my requirements, it would have been a good one. But could you elaborate a bit more on the second part of your answer, as that met my requirements a bit better? \$\endgroup\$ – Simon Forsberg Dec 2 '15 at 21:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.