# Using os.scandir to get all txt files in subfolders

I have a parent dir that has X children dirs each with Y txt files. I want to list all txt files. Is there a way to simplify this?

for entry in os.scandir(DIR):
for subentry in os.scandir(entry.path):
if subentry.is_dir():
for file in os.scandir(subentry.path):
if file.is_file() and file.name.endswith("txt"):
print(file.name)


There are (at least) two ways to easily achieve this. The first one is what @Ludisposed suggested in the comments, glob. It can also recurse down into subdirectories with ** (in Python 3):

import glob

def get_txt_files(base_dir):
return glob.iglob(f"{base_dir}/**/*.txt", recursive=True)


Or, if you are using Windows, which uses different delimiters for paths:

def get_txt_files(base_dir):
return glob.iglob(rf"{base_dir}\**\*.txt", recursive=True)


The other one is basically what @Mast suggested, also in the comments, make this a fully recursive function:

def get_txt_files(base_dir):
for entry in os.scandir(base_dir):
if entry.is_file() and entry.name.endswith(".txt"):
yield entry.name
elif entry.is_dir():
yield from get_txt_files(entry.path)
else:
print(f"Neither a file, nor a dir: {entry.path}")


Both functions return iterators. You can print all names with a simple for loop:

for name in get_txt_files("foo"):
print(name)


If your directory X also contains txt files, they will be yielded from these functions. You can avoid this with glob by making the depth level explicit:

def get_txt_files(base_dir):
return glob.iglob(f"{base_dir}/*/*.txt")


A third, very similar, way is to use pathlib.Path (Python 3 only), which also has a glob method (that directly returns an iterator). This way you can be sure that this works both on UNIX and Windows:

from pathlib import Path

def get_txt_files(base_dir):
return Path(base_dir).glob("*/*.txt")

• Thank you for your thorough answer. I'll go with iglob since there is only one level of depth and it's shorter and easier to read – Julio Feb 4 '18 at 11:50
• @Julio Note that if you are using Windows, it should be rf"{base_dir}\**\*.txt", though. For this reason I would recommend the pathlib.Path way. – Graipher Feb 4 '18 at 11:54
• I see, I'll give Path a go then. – Julio Feb 4 '18 at 11:59