4
\$\begingroup\$

In the following code, there are some repeated commands. How can I reduce it and make the code more concise and automate it more? In the following code, I am trying to read different folder files. The folders have a fixed kind of structure for all different set of data. I want to access those folders in easier way such that i remove the repeated commands used. Directory + file structure is given as follows

main_path
├── Proband28
    ├── GAIT
         ├── FG_Color
         ├── FG_Depth
         ├── FG_Results
    ├── GAITWCOG
         ├── CB3_Color
         ├── CB3_Depth
         ├── CB3_Results
├── Proband29
    ├── GAIT
         ├── FG_Color
         ├── FG_Depth
         ├── FG_Results
    ├── GAITWCOG
         ├── CB3_Color
         ├── CB3_Depth
         ├── CB3_Results
├── Proband30 ...

Code is as follows:

path2 = "F:/Daten1/"  # Give the path of the folder of which the gait parameter is to be calculated.
print(path2)
print(len([name for name in os.listdir(path2)]))  # Gives the number of folders in the directory

# ----------------------------------------------------------------------------------------------------------------------

i = 1
for i in range(len([name for name in os.listdir(path2)])-5):
    '''
    The following part of code assign links to the variables with Proband numbers, 
    create the folder if it is not available and later also assigns the list of 
    files available in the folder.
    For example, lets take color_img
    color_img is assigned with the path which has multiple Probands. 
    So 'F:/Daten1/' is the path where multiple Probands are available with 
    Proband numbers in sequence. Proband{0} is the place where a particular number will be assigned with the help of i. Here if you want to access 
    Proband1 then you should have .format(i+1). If you want to access
    Proband28 then you should have .format(i+28).
    GAITWCOG/CB3_color is the path where color files of the CB3 experiment are stored.
    The later part of the code creates a folder of CB3_color if it not available.
    The list of files available in CB3_color is assigned to Pathh1 which is later used for deleting files when 3D Pose estimation is achieved.   
    '''
    color_img ='F:/Daten1/Proband{0}\GAITWCOG\CB3_color'.format(i+28)
    try:
        os.makedirs(color_img)
    except FileExistsError:
        # directory already exists
        pass
    pathh1 = os.listdir(color_img)

    depth_img ='F:/Daten1/Proband{0}\GAITWCOG\CB3_Depth'.format(i+28)
    try:
        os.makedirs(depth_img)
    except FileExistsError:
        # directory already exists
        pass
    pathh2 = os.listdir(depth_img)

    json = 'F:/Daten1/Proband{0}\GAITWCOG\CB3_color\json'.format(i+28)
    try:
        os.makedirs(json)
    except FileExistsError:
        # directory already exists
        pass
    pathh3 = os.listdir(json)

    Openpose = 'F:/Daten1/Proband{0}\GAITWCOG/CB3_color/2dOutput/'.format(i+28)
    twoDOutput = 'F:/Daten1/Proband{0}\GAITWCOG/CB3_color/2dOutput/'.format(i+28)
    try:
        os.makedirs(twoDOutput)
    except FileExistsError:
        # directory already exists
        pass
    pathh4 = os.listdir(twoDOutput)

    selected_color ='F:/Daten1/Proband{0}\GAITWCOG\CB3_color\selected'.format(i+28)
    try:
        os.makedirs(selected_color)
    except FileExistsError:
        # directory already exists
        pass
    pathh5 = os.listdir(selected_color)

    selected_depth ='F:/Daten1/Proband{0}\GAITWCOG\CB3_Depth\selected'.format(i+28)
    try:
        os.makedirs(selected_depth)
    except FileExistsError:
        # directory already exists
        pass
    pathh6 = os.listdir(selected_depth)

    mat = 'F:/Daten1/Proband{0}\GAITWCOG\CB3_color\mat'.format(i+28)
    try:
        os.makedirs(mat)
    except FileExistsError:
        # directory already exists
        pass
    pathh7 = os.listdir(mat)

    results = 'F:/Daten1/Proband{0}\GAITWCOG\CB3_results'.format(i+28)
    try:
        os.makedirs(results)
    except FileExistsError:
        # directory already exists
        pass
    pathh8 = os.listdir(results)

    Openpose2D = 'F:/Daten1/Proband{0}\GAITWCOG/CB3_color/2dOutput'.format(i+28)
    if not os.path.exists('Openpose2D'):
        os.makedirs('Openpose2D')

    excel = 'F:\Daten1/Results3\Proband{0}_CB3.xls'.format(i+28)
    '''
    excel is the path where the result of 3D Pose estimation is stored.

    '''

    print(color_img)

    '''
    This part deletes all the generated files to save memory for running the code
    for next set of folders.
    '''
    for item in pathh1:
        if item.endswith(".bmp"):
            os.remove(os.path.join(color_img, item))

    for item in pathh2:
        if item.endswith(".bmp"):
            os.remove(os.path.join(depth_img, item))

    for item in pathh3:
        if item.endswith(".json"):
            os.remove(os.path.join(json, item))

    for item in pathh4:
        if item.endswith(".png"):
            os.remove(os.path.join(twoDOutput, item))

    for item in pathh5:
        if item.endswith(".png"):
            os.remove(os.path.join(selected_color, item))

    for item in pathh6:
        if item.endswith(".png"):
            os.remove(os.path.join(selected_depth, item))

    for item in pathh7:
        if item.endswith(".mat"):
            os.remove(os.path.join(mat, item))

# ---------------------------------------------------------------------------------------------------------------------
  1. In the above code, I am changing the pathname every time to start the loop at all the places like color_img, depth_img, etc and all these have a common part '.../Proband{number}/GAIT/'. So how can I reduce the code and make it in such a way that I give the main folder path and all other paths are assigned automatically?
  2. Also in the above program, I also change (i+28), etc every time I start the loop according to the first folder name and number like Proband28, etc. Can I read the first folder name and get the Proband number directly (i.e number 28) and assign that value to a variable b such that the loop automatically starts from i+b?
  3. I am currently running two sets of above code in such a way that I always change the path names from GAITWCOG/CB3_color/ to GAIT/FG_color etc to access files of another folder. Is it possible to somehow club both the codes in one file maybe with some if condition?
\$\endgroup\$
2
  • 5
    \$\begingroup\$ To answer this question yourself, you should describe the purpose of the code. What should be the effect of running the code? Explain it to a non-programmer, using no programming terms at all. This explanation should be the introduction of the question. \$\endgroup\$ Feb 29, 2020 at 11:54
  • \$\begingroup\$ Why did you never fix your previous question? \$\endgroup\$
    – Mast
    Mar 3, 2020 at 19:06

1 Answer 1

6
\$\begingroup\$

First, os.listdir already returns a list, so len(os.listdir(...)) is sufficient.

Next, you are mixing \ and / in your file paths. Decide which one you want to use. In any case, you should probably use pathlib.Path so you don't have to worry about compatibility between Windows and Linux anyway and can use the division operator (/) to concatenate paths.

os.makedirs has the exist_ok flag which just makes it ignore existing folders.

Finally, you should avoid repetition. All of your paths are almost the same, and need almost the same thing done. So, just put them in a datastructure and iterate over it:

patterns = {'CB3_Depth': '.bmp',
            'CB3_Depth/selected': '.png',
            'CB3_color': '.bmp',
            'CB3_color/2dOutput': '.png',
            'CB3_color/json': '.json',
            'CB3_color/mat': '.mat',
            'CB3_color/selected': '.png',
            'CB3_results': None}

start = 28
end = -5
for i in range(start, start + len(os.listdir(path2)) + end):
    base_path = Path(f"F:/Daten1/Proband{i}/GAITWCOG")
    excel = Path(f"F:/Daten1/Results3/Proband{i}_CB3.xls")
    for path, pattern in patterns.items():
        os.makedirs(base_path / path, exist_ok=True)
        if pattern is not None:
            for file in os.listdir(base_path / path):
                if file.endswith(pattern):
                    os.remove(base_path / path / file)

Note that I removed the duplicate entry CB3_color/2dOutput and used the relatively recent f-string for the formatting. I also moved setting the offsets into variables, this way you can give them a name and they are not just magic values.

Currently the variable excel is unused.

\$\endgroup\$
4
  • \$\begingroup\$ Thank you for the answer. This removes lot of duplicates. I have just one more question. In the above code of mine, I assign the paths to a variable like color_img , pathh1, depth_img etc. How can I assign those variables in the above code? \$\endgroup\$ Mar 5, 2020 at 22:39
  • \$\begingroup\$ @AnkitJaiswal In your code you also don't need it, except for the removing of files. It depends on what you need it for, but in the end we can only give you advice on the code you posted, not anything else you might have that you are not showing us. You could certainly put them into a list, though. \$\endgroup\$
    – Graipher
    Mar 6, 2020 at 0:14
  • \$\begingroup\$ Okk. I thought those parts will just confuse everyone so didn't put it. There are some Python functions that have been defined and I m using those variables to pass it as shown in the above code. I have edited the code. \$\endgroup\$ Mar 6, 2020 at 7:04
  • 4
    \$\begingroup\$ @AnkitJaiswal: I'm sorry, but the question and answer nature of this site makes it so that you are not allowed to modify the code after having received any answers. Otherwise it would cause too much confusion in which version of the question answers target. Have a look at What should I do when someone answers my question?. I rolled your edit back for now. You can always ask a new question (referencing this question is encouraged, though). Feel free to incorporate feedback from my answer to avoid hearing the same advice again. \$\endgroup\$
    – Graipher
    Mar 6, 2020 at 12:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.