# Filtering a long list of files through a set of ignore patterns using iterators

I have a backup job that walks across a huge directory with potentially millions of files. Python's os.walk() works just fine.

Now, I have implemented a feature to ignore files based in a black list of patterns. I just want it to behave like a .gitignore file.

Currently I do this:

import os
from fnmatch import fnmatch

some_dir = '/'
ignore_list = ['*.tmp', 'tmp/', '*.py']
for dirname, _, filenames in os.walk(some_dir):
for filename in filenames:
should_ignore = False
for pattern in ignore_list:
if fnmatch(filename, pattern):
should_ignore = True
if should_ignore:
print 'Ignore', filename
continue


It works, but I failed to write it in terms of iterators.

Can this routine be written in terms of python iterators? Can it be improved in terms of iterators or generators?

for filename in filenames: