1
\$\begingroup\$

In my application, the user may or may not want to ignore some directories. I can do that, but it seems like I am repeating myself. Does anyone have an idea to refactor that?

from os import walk, path
exclude = ['dir1/foo']
for root, dirs, files in walk('.', topdown=True):
    if exclude != None:
        dirs[:] = [d for d in dirs if d not in exclude]
        for name in files:
            for excluded in exclude:
                if excluded not in root:
                    print path.join(root, name)
    else:
        for name in files:
            print path.join(root, name)

exclude is None when there are no dirs to exclude. I thought of setting it to an empty list, but then, this loop for excluded in exclude: won't execute at all. My ambition was to avoid such a big if/else. Any ideas?


Example:

gsamaras@pc:~/mydir$ ls */*
dir1/bar:
test.txt

dir1/foo:
test.txt

dir2/bar:
test.txt

dir2/foo:
test.txt

I am getting:

./dir2/foo/test.txt
./dir2/bar/test.txt

./dir1/bar/test.txt

If I want, I can do an exclude = ['foo'], and then get:

./dir2/bar/test.txt
./dir1/bar/test.txt

meaning that I ignored all directories named "foo".

\$\endgroup\$
  • \$\begingroup\$ Why are you using generic names (foo and bar) for this? \$\endgroup\$ – Jamal Nov 30 '18 at 1:25
  • \$\begingroup\$ Maybe I should have used subdir @Jamal, right? \$\endgroup\$ – gsamaras Nov 30 '18 at 8:42
  • \$\begingroup\$ Just make sure to use the original names that were used with the project. \$\endgroup\$ – Jamal Dec 1 '18 at 1:59
1
\$\begingroup\$

You are already modifiyin the list of directories, so that should be enough. But your exclude includes the full path, so your check in the list comprehension does not actually filter the excluded directories, you only do that in the for loop below when already having descended into those excluded directories (wasting time).

So, this should work:

from os import walk, path

exclude = {'./dir1/foo'}

for root, dirs, files in walk('.'):
    if exclude is not None:
        dirs[:] = [d for d in dirs if path.join(root, d) not in exclude]
    for name in files:
        print path.join(root, name)

Note that exclude needs to contain paths starting with the starting point of os.walk, so in this case ..

I also made exclude a set (\$\mathcal{O}(1)\$ in lookup), used the fact that topdown=True by default and used is not instead of != for comparison to None.


If you want to instead exclude folder names (regardless of their position in the directory tree), you can do that as well like this:

from os import walk, path

exclude = {'foo'}

for root, dirs, files in walk('.'):
    if exclude is not None:
        dirs[:] = [d for d in dirs if d not in exclude]
    for name in files:
        print path.join(root, name)

What is not possible with either of these two approaches is to include foo only in sub-directories of dir1, like in your example. However, I think this is more consistent behaviour, so you should choose one of the two, IMO.


As a last point, you should probably switch to Python 3 sooner rather than later if at all possible, because support for Python 2 will end in a bit more than a year (at the time of writing this).

\$\endgroup\$
  • \$\begingroup\$ Very pythonic answer. However, check my example (edited), where I have two directories named foo. With my ugly code, the user can ignore both `foo's, or specify further. Also, note that in your code, the user has to specify the absolute path, while in mine's, just the name in the generic approach. Still, a +1, but you might want to play around, if you have some time. As for the last point, not possible now. :/ Or maybe, I should switch to your approach, because the user might have forgotten that (s)he has two foo folders........... ;) \$\endgroup\$ – gsamaras Nov 29 '18 at 15:06
  • \$\begingroup\$ @gsamaras: Yeah, you have to decide if you only want to exclude folders by their full path (relative to the top) or all folders by that name. In the latter case, you can just do [d for d in dirs if d not in exclude] and should still not need your two cases, because it will never descend into excluded folders (so there is no need to continue checking if the path contains an excluded folder). \$\endgroup\$ – Graipher Nov 29 '18 at 16:00
  • \$\begingroup\$ Right! What I really liked with my code, is that I could say dir2, instead of ./dir2 with your approach. However, your code might be safer for the user, and also much cleaner. It's a trade-off that I will have to think, when I have time.. Thanks for your time, and since I talked about time, I want you to enjoy my God of Time 10 yrs Stackoverflow post, as a thank you gesture, if you have some time, of course! :) This comment selfdestruct at some time tomorrow... \$\endgroup\$ – gsamaras Nov 29 '18 at 16:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.