3
\$\begingroup\$

I am new to programming and I am trying to develop a machine learning approach for image segmentation (https://en.wikipedia.org/wiki/Image_segmentation) and here I want to evaluate the predicted (output) segmentation. In order to do this, I am using multiple (common) metrics (dice score, jacquard index, surface distance Hausdorff distance, diameter error and volume error).

The code works, but it really looks bad! I heard that one of the best practices is to avoid repetition in the code. For my piece I have to create multiple arrays which will all have different usage, but generating a lot of repetitions, and I am sure there might be better way to get better performance for this piece of code.

Do you have an idea of how to optimize this for speed and readability?

The details of the code:

folder = glab() : gets the name of the different folders containing the arrays that I want to open (glab() = glob() with natural sorting)

prediction/ground_truth : loads the predicted segmentation and ground_truth to be compared

scores() : calculates the different metrics (dice score, jacquard index, sensibility and specificity) between the ground truth and prediction

surfd() : calculates the surface distance between ground truth and prediction

diam_vol : calculates the diameter/volume and error between ground truth and prediction

Since I am evaluating multiple cases I want to know the mean() metrics to assess my model that's why I call for np.mean() for all the values at the end.

def metrics(savedir):

    """3D metrics for segmentation evaluation:

    dice score : https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient
    jacquard index : https://en.wikipedia.org/wiki/Jaccard_index
    sensibility & specificity : https://en.wikipedia.org/wiki/Sensitivity_and_specificity
    diameter and volume error
    Surface distance : https://www.cs.ox.ac.uk/files/7732/CS-RR-15-08.pdf
    Hausdorff distance : https://en.wikipedia.org/wiki/Hausdorff_distance
    """

    folder = glab(savedir+"/OutputData/*/")

    # array initialization
    np_dice = np.zeros([len(folder)])
    np_jacq = np.zeros([len(folder)])
    np_sens = np.zeros([len(folder)])
    np_spec = np.zeros([len(folder)])
    np_surf = np.zeros([len(folder)])
    np_haus = np.zeros([len(folder)])

    np_diam_GT = np.zeros([len(folder)])
    np_diam_pred = np.zeros([len(folder)])
    np_diam_err = np.zeros([len(folder)])

    np_vol_GT = np.zeros([len(folder)])
    np_vol_pred = np.zeros([len(folder)])
    np_vol_err = np.zeros([len(folder)])


    for number, folder_name in enumerate(folder):

        prediction = np.load(folder_name+"prediction.npy")
        ground_truth = np.load(folder_name+"test_label.npy")

        scores = scoring(ground_truth, prediction)
        surface_distance = surfd(prediction, ground_truth, [0.625, 0.625, 0.625], 1)
        diam_vol = diam(prediction, ground_truth)

        np_dice[number] = scores[0]
        np_jacq[number] = scores[1]
        np_sens[number] = scores[2]
        np_spec[number] = scores[3]
        np_surf[number] = surface_distance.mean()
        np_haus[number] = surface_distance.max()

        np_diam_GT[number] = diam_vol[0]
        np_diam_pred[number] = diam_vol[1]
        np_diam_err[number] = diam_vol[2]

        np_vol_GT[number] = diam_vol[3]
        np_vol_pred[number] = diam_vol[4]
        np_vol_err[number] = diam_vol[5]

    mean_dice = np.mean(np_dice)
    mean_jacq = np.mean(np_jacq)
    mean_sens = np.mean(np_sens)
    mean_spec = np.mean(np_spec)
    mean_surf = np.mean(np_surf)
    mean_haus = np.mean(np_haus)
    mean_diam_GT = np.mean(np_diam_GT)
    mean_diam_pred = np.mean(np_diam_pred)
    mean_diam_err = np.mean(np_diam_err)
    mean_vol_GT = np.mean(np_vol_GT)
    mean_vol_pred = np.mean(np_vol_pred)
    mean_vol_err = np.mean(np_vol_err)

    return ([mean_dice, mean_jacq, mean_sens, mean_spec, mean_surf, mean_haus, mean_diam_GT, mean_diam_pred, mean_diam_err, mean_vol_GT, mean_vol_pred, mean_vol_err])
\$\endgroup\$
  • \$\begingroup\$ Welcome to Code Review. Please read How to Ask. Tell us what this code accomplishes. By glab(), do you mean glob()? Is this real working code? What do scoring_baby(), surfd() and diam() do? \$\endgroup\$ – 200_success May 23 at 3:11
  • 1
    \$\begingroup\$ Welcome to Code Review. I have rolled back your last edit. Please do not update the code in your question to incorporate feedback from answers, doing so goes against the Question + Answer style of Code Review. This is not a forum where you should keep the most updated version in your question. Please see what you may and may not do after receiving answers. \$\endgroup\$ – Heslacher May 23 at 4:20
  • 2
    \$\begingroup\$ Hi, and thank you very much for your time. glab() is glob() with natural sorting (so I can find myself more easily in my data folder); scoring_baby() calculates the dice score, jacquard index, sensibility and specificity between a ground truth label (binary mask) and a predicted label (binary mask as well), surfd() calculates the surface distance between the two lables, and diam() calculates the different diameter and volume of the 3D segmentation. All the code is working, I just wanted to improve and understand more how to write proper code. \$\endgroup\$ – Unic0 May 23 at 4:24
  • \$\begingroup\$ It would be nice if you provide a description or link to explain segmentation, for those who know Python but don't have the same domain knowledge of the problem space. \$\endgroup\$ – Toby Speight May 24 at 8:28
2
\$\begingroup\$

Realy, dud its childish!

only ones:

lenf = len(folder)

just one 2d array:

len_types = 12
arr = np.zeros([lenf ,  12])

learn how to work with numpy arrays, you will avoid the loop as well. And for example the mean can be done in one line for 2d array:

means = np.mean(arr, axis = 1)

https://docs.scipy.org/doc/numpy/reference/generated/numpy.mean.html

\$\endgroup\$
  • \$\begingroup\$ Thank you very much, sorry I am new to this and clearly need improvement on the manipulation of matrices with numpy. Using you advice I was able to correct my code [see EDIT first post]. But I feel more can be done, do you have other suggestions ? (sorry for the noobism here) \$\endgroup\$ – Unic0 May 23 at 2:59
  • \$\begingroup\$ wait a bit, i can create good numpy tutorial from this example and ll post here later \$\endgroup\$ – user8426627 May 23 at 14:52
1
\$\begingroup\$

return lists

your functions seem to return lists or tuples. For 2 or 3 values, this is ok, but diam returns 6 values, metrics 12. This is very unclear, and unstable when you want to add, reorder or remove a metric. Better would be then to either return a dict or a namedtuple

spacing

pep-8 recommends around operators (+,...) I use a code formatter (black) to format my code, so I don't have to worry about this sort of details any more

variable naming

longer variable names don't make your program slower. And if you use a capable IDE, it doesn't slow the coding either. abbreviated variable names on the other hand make code more unclear. There is no need to shorten jacquard to jack

magical values

[0.625, 0.625, 0.625], 1 as arguments to surface_distance are magical values, and very unclear what they mean. Easier would be to replace them with a variable with a clear name, or use keyword arguments.

containers

If you have a lot of metrics, it is generally cleaner to keep them in 1 data structure (dict for example) instead of 12 different variables, and 12 other variables for the means.

pandas

Instead of using numpy directly, why not use a pandas DataFrame as data structure? This will allow you to work with labeled data.

import pandas as pd


def metrics(savedir):

    """3D metrics for segmentation evaluation:

    dice score : https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient
    jacquard index : https://en.wikipedia.org/wiki/Jaccard_index
    sensibility & specificity : https://en.wikipedia.org/wiki/Sensitivity_and_specificity
    diameter and volume error
    Surface distance : https://www.cs.ox.ac.uk/files/7732/CS-RR-15-08.pdf
    Hausdorff distance : https://en.wikipedia.org/wiki/Hausdorff_distance
    """

    folder = glab(savedir + "/OutputData/*/")
    results = pd.DataFrame(
        index=folder,
        columns=[
            "dice",
            "jacquard",
            "sensibility",
            "specificity",
            "surface",
            "hausdorff",
            "diam_GT",
            "diam_pred",
            "diam_err",
            "vol_GT",
            "vol_pred",
            "vol_err",
        ],
        dtype=float,
    )
    for folder_name in folder:
        prediction = np.load(folder_name + "prediction.npy")
        ground_truth = np.load(folder_name + "test_label.npy")

        scores = scoring(ground_truth, prediction)
        surface_distance = surfd(
            prediction, ground_truth, [0.625, 0.625, 0.625], 1
        )
        diam_vol = diam(prediction, ground_truth)
        results.loc[folder_name] = pd.Series(
            {
                "dice": scores[0],
                "jacquard": scores[1],
                "sensibility": scores[2],
                "specificity": scores[3],
                "surface": surface_distance.mean(),
                "hausdorff": surface_distance.max(),
                "diam_GT": diam_vol[0],
                "diam_pred": diam_vol[1],
                "diam_err": diam_vol[2],
                "vol_GT": diam_vol[3],
                "vol_pred": diam_vol[4],
                "vol_err": diam_vol[5],
            }
        )

    return results.mean().to_dict()
\$\endgroup\$
  • \$\begingroup\$ Thank you very much for you answer. Your solution is clean and very nice but I would try to use numpy as much as I can because I am learning and I feel like I am missing some of the basic mechanics of numpy which could help me in the future. But I'll look more into panda because I know that the dataframe class in panda can be interesting to work with. \$\endgroup\$ – Unic0 May 25 at 3:12
  • \$\begingroup\$ Pandas uses numpy under the hood. For the thing you want to do, with the support of labelled data, it's way better suited then raw numpy \$\endgroup\$ – Maarten Fabré May 25 at 5:39
  • \$\begingroup\$ Noted, I look more into it. But I noticed that when I use your method I have different results that doesn't seem correct (nothing else was changed but the piece of code you provided). And I don't know why. I'll keep investigating. Thank you very much \$\endgroup\$ – Unic0 May 26 at 1:36
  • \$\begingroup\$ Possibly, you need to add axis=1 as argument to mean \$\endgroup\$ – Maarten Fabré May 26 at 6:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.