Parallel image segmentation [closed]

Question

Follow up question Image processing using Python OOP library

I want to use multiprocessing to analyze several images in parallel:

class SegmentationType(object):
    DISPLAY_NAME = "invalid"

    def __init__(self, filename, path):
        self.filename = filename
        self.path = path
        self.input_data = None
        self.output_data = None


    def read_image(self):
        self.input_data =  cv2.imread(self.path + self.filename)[1]

    def write_image(self):
        cv2.imwrite(self.path + self.filename.split('.')[0] + '_' + self.DISPLAY_NAME + '.png', self.output_data)

    def process(self):
        # override in derived classes to perform an actual segmentation
        pass

    def start_pipeline(self):
        self.read_image()
        self.process()
        self.write_image()

class HSV_Segmenter(SegmentationType):
    DISPLAY_NAME = 'HSV'

    def process(self):
        source = rgb_to_hsv(self.input_data)
        self.output_data = treshold_otsu(source)


class LabSegmenter(SegmentationType):
    DISPLAY_NAME = 'LAB'

    def process(self):
        source = rgb_to_lab(self.input_data)
        self.output_data = global_threshold(source)


segmenter_class = {
'hsv': HSV_Segmentation,
'lab': LAB_Segmenter
}.get(procedure)

if not segmenter_class:
    raise ArgumentError("Invalid segmentation method '{}'".format(procedure))

for img in images:
    os.chdir(img_dir)
    processor =  = segmenter_class(img, img_dir, procedure)
    processor.start_pipeline()

What I tried so far:

image_lst = os.listdir(my_image_path)

# We split the list into sublist with 5 elements because of 512 GB RAM limitation
if len(image_lst) > 4:
    nr_of_sublists = (int(len(image_lst)/2.5))
    image_sub_lst  =(np.array_split(image_lst, nr_of_sublists))
else:
    image_sub_lst = [image_lst]

# We do the analysis for each sublist
for sub_lst in image_sub_lst:
    print (sub_lst)
    pool = multiprocessing.Pool(8)
    
    # Call the processor 
    processor =  = segmenter_class(img, img_dir, procedure)
    processor.start_pipeline()
    # How to call map???
    pool.map(?, sub_lst)
    pool.terminate()
    

Answer 1

A warning on version

Many of the following suggestions assume that you are using Python 3.

Bare inheritance

class SegmentationType(object):

can be

class SegmentationType:

Abstract statics

DISPLAY_NAME = "invalid"

should not really assign a value. Instead,

DISPLAY_NAME: str

Unpacking imread

The documentation is deeply unhelpful: it says that imread returns "retval". Given your usage it's obvious that reality is more complicated, because you're indexing into it. Try to unpack instead:

_, self.input_data = cv2.imread(self.path + self.filename)

Abstract methods

process should raise NotImplementedError in the base.

Factory

You have a factory dictionary that should be turned into a method, something like

def get_segmenter(name: str) -> Type[SegmentationType]:
    return {
        t.DISPLAY_NAME: t
        for t in (HSVSegmenter, LABSegmenter)
    }[name]

Outer parentheses

Neither of these:

nr_of_sublists = (int(len(image_lst)/2.5))
image_sub_lst  =(np.array_split(image_lst, nr_of_sublists))

needs outer parentheses.

Syntax

Surely this is a typo? This will not run:

processor =  = segmenter_class(img, img_dir, procedure)

nor will this:

pool.map(?, sub_lst)