0
\$\begingroup\$

I have a backup job that walks across a huge directory with potentially millions of files. Python's os.walk() works just fine.

Now, I have implemented a feature to ignore files based in a black list of patterns. I just want it to behave like a .gitignore file.

Currently I do this:

import os
from fnmatch import fnmatch

some_dir = '/'
ignore_list = ['*.tmp', 'tmp/', '*.py']
for dirname, _, filenames in os.walk(some_dir):
    for filename in filenames:
        should_ignore = False
        for pattern in ignore_list:
            if fnmatch(filename, pattern):
                should_ignore = True
        if should_ignore:
            print 'Ignore', filename
            continue

It works, but I failed to write it in terms of iterators.

Can this routine be written in terms of python iterators? Can it be improved in terms of iterators or generators?

\$\endgroup\$
4
\$\begingroup\$

You can simplify the loop by using a generator expression like this:

for filename in filenames:
    if any(fnmatch(filename, pattern) for pattern in ignore_list):
        print 'Ignore', filename
        continue
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.