# Image pre-processing and segmentation

I'm new learning python and image processing with python. For this reason, I took a project called "Classification of breast cancer images with deep learning".

I applied the following techniques: 1)Threshold, 2)K-Means

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread('n001.tif', 0)
_, blackAndWhite = cv2.threshold(img, 100, 255, cv2.THRESH_BINARY_INV)

nlabels, labels, stats, centroids = cv2.connectedComponentsWithStats(blackAndWhite, None, None, None, 8, cv2.CV_32S)
sizes = stats[1:, -1] #get CC_STAT_AREA component
img2 = np.zeros((labels.shape), np.uint8)

for i in range(0, nlabels - 1):
if sizes[i] >= 100:   #filter small dotted regions
img2[labels == i + 1] = 255

res = cv2.bitwise_not(img2)

cv2.imwrite('n001New.tif', res)
########################################################################

img = cv2.cvtColor(res, cv2.COLOR_BGR2RGB)

Z = img.reshape((-1,3))
Z = np.float32(Z)

criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 250 , 1.0)

K=2
ret, label, center= cv2.kmeans(Z, K, None, criteria, 250,
cv2.KMEANS_RANDOM_CENTERS)
center = np.uint8(center)
res = center[label.flatten()]
output = res.reshape((img.shape))

plt.imshow(output)

plt.show()


Data set consists of tif extension image: benign, inSitu, invasive and normal

Output images:

I have a few questions:

1. Are my methods correct?
2. Are the outputs correct? Do I find the right areas?
3. Did I use the K-Means algorithm correctly?

I would appreciate it if you could answer the questions above.

## 1 Answer

Are my methods correct?

That depends on what you want to accomplish. Since you are "learning python and image processing with python", it seems you picked some related methods to explore, which is good. But since your project is called "Classification of breast cancer images with deep learning", and you're not using deep learning, maybe you didn't pick the right methods...

See below for some more concepts.

Are the outputs correct? Do I find the right areas?

Again, that depends on what your goal is. Do you know what these images represent? If no, then you need to start there. Understand what the image represents, what is relevant in it, and understand what the output of your algorithms will be used for. Then you will be able to answer your question yourself.

It looks like you found mostly the nuclei, the results are quite OK if that is what you're after.

Did I use the K-Means algorithm correctly?

You've copy-pasted this from the OpenCV tutorial, so it's correct. But it also is a bit redundant, since the values of center are not useful to you in this case. The output label is an image with values 0 and 1, representing background and foreground. You should be able to directly display that (maybe multiply by 255 first).

Also the line where you convert BGR to RGB is redundant, the k-means result will be the same regardless, and you don't need the colors after that.

When dealing with brightfield microscopy, as you are, you might want to consider separating the stains instead of directly using the RGB values. These images are all H&E stain (Hematoxylin and Eosin). These are purple and pink, respectively. Because we're dealing with transmitted light, and the dyes absorb light of specific wavelengths, we see darker pixels of specific colors. The absorption is characterized by the Beer-Lambert law. This law states that the amount of light transmitted (and thus seen by the camera) is given by $$\10^{-A}\$$, with $$\A\$$ proportional to the amount of dye. Thus, given a pixel value $$\v\$$ and the illumination intensity $$\v_\text{max}\$$ (the whitest area in the image), you can compute $$\-log(v/v_\text{max})\$$, which is proportional to the amount of dye. You can do this separately for each of the channels R, G and B. Each dye has a different absorption coefficient for each channel. If you know these (you can compute them from the image data), you can now do a linear unmixing (solve a linear set of equations) to derive, for each pixel, the relative amount of Hematoxylin and the relative amount of Eosin at that location on the slide. These two values is what you should be working with.

• thank you for your answer. My goal is I will increase the accuracy of like CNN algorithms by dealing with pictures. But you are right. I don't know what image represent. What I need to find from picture with image processing. Which area should I deal with? I'm researching, but I don't quite understand. For example, when I classify pictures according to which features?color?nuclei position?size?.... My English not good I hope my problem is clear. @CrisLuengo Feb 8, 2019 at 23:31
• @dogac: What you need to find depends very much on the application. "CNN algorithm dealing with pictures" is not the application, it's the tool. Nuclei size can certainly be important, but it depends on the application and on the context. I cannot advise you like this. --- But I can strongly suggest you learn about Beer-Lambert and stain unmixing, and use that. Most people using CNNs in digital pathology don't, because they don't know digital pathology, they just apply CNNs blindly. I'm sure CNNs would work significantly better if given the right input. Feb 8, 2019 at 23:45
• yes, what should be the right inputs? I will research. Thank you again and good luck. @cris Feb 8, 2019 at 23:54