Using several file system commands repeatedly. How can the code size be reduced?

Question

In the following code, there are some repeated commands. How can I reduce it and make the code more concise and automate it more? In the following code, I am trying to read different folder files. The folders have a fixed kind of structure for all different set of data. I want to access those folders in easier way such that i remove the repeated commands used. Directory + file structure is given as follows

main_path
├── Proband28
    ├── GAIT
         ├── FG_Color
         ├── FG_Depth
         ├── FG_Results
    ├── GAITWCOG
         ├── CB3_Color
         ├── CB3_Depth
         ├── CB3_Results
├── Proband29
    ├── GAIT
         ├── FG_Color
         ├── FG_Depth
         ├── FG_Results
    ├── GAITWCOG
         ├── CB3_Color
         ├── CB3_Depth
         ├── CB3_Results
├── Proband30 ...

Code is as follows:

path2 = "F:/Daten1/"  # Give the path of the folder of which the gait parameter is to be calculated.
print(path2)
print(len([name for name in os.listdir(path2)]))  # Gives the number of folders in the directory

# ----------------------------------------------------------------------------------------------------------------------

i = 1
for i in range(len([name for name in os.listdir(path2)])-5):
    '''
    The following part of code assign links to the variables with Proband numbers, 
    create the folder if it is not available and later also assigns the list of 
    files available in the folder.
    For example, lets take color_img
    color_img is assigned with the path which has multiple Probands. 
    So 'F:/Daten1/' is the path where multiple Probands are available with 
    Proband numbers in sequence. Proband{0} is the place where a particular number will be assigned with the help of i. Here if you want to access 
    Proband1 then you should have .format(i+1). If you want to access
    Proband28 then you should have .format(i+28).
    GAITWCOG/CB3_color is the path where color files of the CB3 experiment are stored.
    The later part of the code creates a folder of CB3_color if it not available.
    The list of files available in CB3_color is assigned to Pathh1 which is later used for deleting files when 3D Pose estimation is achieved.   
    '''
    color_img ='F:/Daten1/Proband{0}\GAITWCOG\CB3_color'.format(i+28)
    try:
        os.makedirs(color_img)
    except FileExistsError:
        # directory already exists
        pass
    pathh1 = os.listdir(color_img)

    depth_img ='F:/Daten1/Proband{0}\GAITWCOG\CB3_Depth'.format(i+28)
    try:
        os.makedirs(depth_img)
    except FileExistsError:
        # directory already exists
        pass
    pathh2 = os.listdir(depth_img)

    json = 'F:/Daten1/Proband{0}\GAITWCOG\CB3_color\json'.format(i+28)
    try:
        os.makedirs(json)
    except FileExistsError:
        # directory already exists
        pass
    pathh3 = os.listdir(json)

    Openpose = 'F:/Daten1/Proband{0}\GAITWCOG/CB3_color/2dOutput/'.format(i+28)
    twoDOutput = 'F:/Daten1/Proband{0}\GAITWCOG/CB3_color/2dOutput/'.format(i+28)
    try:
        os.makedirs(twoDOutput)
    except FileExistsError:
        # directory already exists
        pass
    pathh4 = os.listdir(twoDOutput)

    selected_color ='F:/Daten1/Proband{0}\GAITWCOG\CB3_color\selected'.format(i+28)
    try:
        os.makedirs(selected_color)
    except FileExistsError:
        # directory already exists
        pass
    pathh5 = os.listdir(selected_color)

    selected_depth ='F:/Daten1/Proband{0}\GAITWCOG\CB3_Depth\selected'.format(i+28)
    try:
        os.makedirs(selected_depth)
    except FileExistsError:
        # directory already exists
        pass
    pathh6 = os.listdir(selected_depth)

    mat = 'F:/Daten1/Proband{0}\GAITWCOG\CB3_color\mat'.format(i+28)
    try:
        os.makedirs(mat)
    except FileExistsError:
        # directory already exists
        pass
    pathh7 = os.listdir(mat)

    results = 'F:/Daten1/Proband{0}\GAITWCOG\CB3_results'.format(i+28)
    try:
        os.makedirs(results)
    except FileExistsError:
        # directory already exists
        pass
    pathh8 = os.listdir(results)

    Openpose2D = 'F:/Daten1/Proband{0}\GAITWCOG/CB3_color/2dOutput'.format(i+28)
    if not os.path.exists('Openpose2D'):
        os.makedirs('Openpose2D')

    excel = 'F:\Daten1/Results3\Proband{0}_CB3.xls'.format(i+28)
    '''
    excel is the path where the result of 3D Pose estimation is stored.

    '''

    print(color_img)

    '''
    This part deletes all the generated files to save memory for running the code
    for next set of folders.
    '''
    for item in pathh1:
        if item.endswith(".bmp"):
            os.remove(os.path.join(color_img, item))

    for item in pathh2:
        if item.endswith(".bmp"):
            os.remove(os.path.join(depth_img, item))

    for item in pathh3:
        if item.endswith(".json"):
            os.remove(os.path.join(json, item))

    for item in pathh4:
        if item.endswith(".png"):
            os.remove(os.path.join(twoDOutput, item))

    for item in pathh5:
        if item.endswith(".png"):
            os.remove(os.path.join(selected_color, item))

    for item in pathh6:
        if item.endswith(".png"):
            os.remove(os.path.join(selected_depth, item))

    for item in pathh7:
        if item.endswith(".mat"):
            os.remove(os.path.join(mat, item))

# ---------------------------------------------------------------------------------------------------------------------

1. In the above code, I am changing the pathname every time to start the loop at all the places like color_img, depth_img, etc and all these have a common part '.../Proband{number}/GAIT/'. So how can I reduce the code and make it in such a way that I give the main folder path and all other paths are assigned automatically?
2. Also in the above program, I also change (i+28), etc every time I start the loop according to the first folder name and number like Proband28, etc. Can I read the first folder name and get the Proband number directly (i.e number 28) and assign that value to a variable b such that the loop automatically starts from i+b?
3. I am currently running two sets of above code in such a way that I always change the path names from GAITWCOG/CB3_color/ to GAIT/FG_color etc to access files of another folder. Is it possible to somehow club both the codes in one file maybe with some if condition?

Answer 1

First, os.listdir already returns a list, so len(os.listdir(...)) is sufficient.

Next, you are mixing \ and / in your file paths. Decide which one you want to use. In any case, you should probably use pathlib.Path so you don't have to worry about compatibility between Windows and Linux anyway and can use the division operator (/) to concatenate paths.

os.makedirs has the exist_ok flag which just makes it ignore existing folders.

Finally, you should avoid repetition. All of your paths are almost the same, and need almost the same thing done. So, just put them in a datastructure and iterate over it:

patterns = {'CB3_Depth': '.bmp',
            'CB3_Depth/selected': '.png',
            'CB3_color': '.bmp',
            'CB3_color/2dOutput': '.png',
            'CB3_color/json': '.json',
            'CB3_color/mat': '.mat',
            'CB3_color/selected': '.png',
            'CB3_results': None}

start = 28
end = -5
for i in range(start, start + len(os.listdir(path2)) + end):
    base_path = Path(f"F:/Daten1/Proband{i}/GAITWCOG")
    excel = Path(f"F:/Daten1/Results3/Proband{i}_CB3.xls")
    for path, pattern in patterns.items():
        os.makedirs(base_path / path, exist_ok=True)
        if pattern is not None:
            for file in os.listdir(base_path / path):
                if file.endswith(pattern):
                    os.remove(base_path / path / file)

Note that I removed the duplicate entry CB3_color/2dOutput and used the relatively recent f-string for the formatting. I also moved setting the offsets into variables, this way you can give them a name and they are not just magic values.

Currently the variable excel is unused.