1
\$\begingroup\$

I have a parent dir that has X children dirs each with Y txt files. I want to list all txt files. Is there a way to simplify this?

for entry in os.scandir(DIR): 
        for subentry in os.scandir(entry.path):
            if subentry.is_dir():
                for file in os.scandir(subentry.path):
                    if file.is_file() and file.name.endswith("txt"):
                        print(file.name)
\$\endgroup\$
6
\$\begingroup\$

There are (at least) two ways to easily achieve this. The first one is what @Ludisposed suggested in the comments, glob. It can also recurse down into subdirectories with ** (in Python 3):

import glob

def get_txt_files(base_dir):
    return glob.iglob(f"{base_dir}/**/*.txt", recursive=True)

Or, if you are using Windows, which uses different delimiters for paths:

def get_txt_files(base_dir):
    return glob.iglob(rf"{base_dir}\**\*.txt", recursive=True)

The other one is basically what @Mast suggested, also in the comments, make this a fully recursive function:

def get_txt_files(base_dir):
    for entry in os.scandir(base_dir):
        if entry.is_file() and entry.name.endswith(".txt"):
            yield entry.name
        elif entry.is_dir():
            yield from get_txt_files(entry.path)
        else:
            print(f"Neither a file, nor a dir: {entry.path}")

Both functions return iterators. You can print all names with a simple for loop:

for name in get_txt_files("foo"):
    print(name)

If your directory X also contains txt files, they will be yielded from these functions. You can avoid this with glob by making the depth level explicit:

def get_txt_files(base_dir):
    return glob.iglob(f"{base_dir}/*/*.txt")

A third, very similar, way is to use pathlib.Path (Python 3 only), which also has a glob method (that directly returns an iterator). This way you can be sure that this works both on UNIX and Windows:

from pathlib import Path

def get_txt_files(base_dir):
    return Path(base_dir).glob("*/*.txt")
| improve this answer | |
\$\endgroup\$
  • \$\begingroup\$ Thank you for your thorough answer. I'll go with iglob since there is only one level of depth and it's shorter and easier to read \$\endgroup\$ – Julio Feb 4 '18 at 11:50
  • 1
    \$\begingroup\$ @Julio Note that if you are using Windows, it should be rf"{base_dir}\**\*.txt", though. For this reason I would recommend the pathlib.Path way. \$\endgroup\$ – Graipher Feb 4 '18 at 11:54
  • 1
    \$\begingroup\$ I see, I'll give Path a go then. \$\endgroup\$ – Julio Feb 4 '18 at 11:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.