6
\$\begingroup\$

I am working on scanned documents (ID card, Driver licenses, ...). The problem I faced while I apply some preprocessing on them is that the documents occupy just a small area of the image, all the rest area is whether white/blank space or noised space. For that reason I wanted to develop a Python code that automatically trims the unwanted area and keeps only the zone where the document is located (without I predefine the resolution). That's possible with using findContours() from OpenCV. However, not all the documents (especially the old ones) have clear contour and not all the blank space is white, so this will not work.

The idea that came to me is:

  1. Read the image and convert it to gray-scale.
  2. Apply the bitwise_not() function from OpenCV to separate the background from the foreground.
  3. Apply adaptive mean threshold to remove as much possible of noise (and eventually to whiten the background).

    At this level, I have the background almost white and the document is in black but containing some white gaps.

  4. I applied erosion to fill the gaps.

  5. Read each row of the image and if 20% of it contains black, then keep it, if it is white, delete it. And do the same with each column of the image.
  6. Crop the image according to the min and max of the index of the black lines and columns.

Here is my code with some comments:

import cv2
import numpy as np

def crop(filename):
    #Read the image
    img = cv2.imread(filename)
    #Convert to grayscale
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    #Separate the background from the foreground
    bit = cv2.bitwise_not(gray)
    #Apply adaptive mean thresholding
    amtImage = cv2.adaptiveThreshold(bit, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 35, 15)
    #Apply erosion to fill the gaps
    kernel = np.ones((15,15),np.uint8)
    erosion = cv2.erode(amtImage,kernel,iterations = 2)
    #Take the height and width of the image
    (height, width) = img.shape[0:2]
    #Ignore the limits/extremities of the document (sometimes are black, so they distract the algorithm)
    image = erosion[50:height - 50, 50: width - 50]
    (nheight, nwidth) = image.shape[0:2]
    #Create a list to save the indexes of lines containing more than 20% of black.
    index = []
    for x in range (0, nheight):
        line = []

        for y in range(0, nwidth):
            line2 = []
            if (image[x, y] < 150):
                line.append(image[x, y])
        if (len(line) / nwidth > 0.2):  
            index.append(x)
    #Create a list to save the indexes of columns containing more than 15% of black.
    index2 = []
    for a in range(0, nwidth):
        line2 = []
        for b in range(0, nheight):
            if image[b, a] < 150:
                line2.append(image[b, a])
        if (len(line2) / nheight > 0.15):
            index2.append(a)

    #Crop the original image according to the max and min of black lines and columns.
    img = img[min(index):max(index) + min(250, (height - max(index))* 10 // 11) , max(0, min(index2)): max(index2) + min(250, (width - max(index2)) * 10 // 11)]
    #Save the image
    cv2.imwrite('res_' + filename, img)

Here is an example. I used an image from the internet to avoid any confidentiality problem. It is to notice here that the image quality is much better (the white space does not contain noise) than the examples I work on.

Input: 1920x1080

Input

Output: 801x623

Output

I tested this code with different documents, and it works well. The problem is that it takes a lot of time to process a single document (because of the loops and reading each pixel of the image twice: once with lines and the second with columns). I am sure that it is possible to do some modifications to optimize the code and reduce the processing time. But I am very beginner with Python and code optimization.

Maybe using numpy to process the matrix calculations or optimizing the loops would improve the code quality.

\$\endgroup\$
  • 1
    \$\begingroup\$ @Graipher, I added an example. The Image I used is from the internet to avoid any problem. \$\endgroup\$ – singrium Jan 28 at 16:22
  • 1
    \$\begingroup\$ I don't have time at this moment for a full review, but you don't have to loop over the image twice. You start looping over it from each side (4 times), but stop at the first line which is of the id. To identify a line, you also don't need to append the value of the pixel to a new list, but just increment a counter \$\endgroup\$ – Maarten Fabré Jan 29 at 16:10
  • \$\begingroup\$ @MaartenFabré, that seems so smart! Thank you for these hints, I'll work on them. \$\endgroup\$ – singrium Jan 29 at 16:13
2
\$\begingroup\$

Split the code

This long method does a lot of things:

  • it reads the file
  • preprocesses it
  • searches for the bounding box of the area of interest
  • crops the result
  • writes the result to a file

Better would be to split this into more parts. This way, you don't need to comment as much, but let the function names speak for themselves.

Comments

If you do feel the need to comment, you can do that in the docstring. If you want to comment on the code, explain why you do it, not how.

#Ignore the limits/extremities of the document (sometimes are black, so they distract the algorithm)

is a useful comment.

# Apply adaptive mean thresholding

is not. It doesn't explain what problem this adaptive thresholding solves, and how you got to the parameters you use: 255, ADAPTIVE_THRESH_MEAN_C, 35 and 15

Indexing

negative indices start counting from the back of a sequence, so

(height, width) = img.shape[0:2]
image = erosion[50:height - 50, 50: width - 50]

can be replaced by erosion[50:-50, 50:-50]

There is also no need to put the parentheses around the height, width tuple.

Magic numbers

There are a lot of magic number in your code: 15 and 35 in the adaptive threshold, 15 in the kerneling, 50 in the cropping,...

Better would be to give them names, and define them in the function or use them as parameters to pass into the function.

Keyword arguments

amtImage = cv2.adaptiveThreshold(bit, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 35, 15)

would be a lot clearer as:

blocksize = 35
constant = 15
max_value = 255 # 8 bits
amtImage = cv2.adaptiveThreshold(
    src=bit, 
    maxValue=max_value ,
    adaptiveMethod=cv2.ADAPTIVE_THRESH_MEAN_C, 
    thresholdType=cv2.THRESH_BINARY, 
    blockSize=blocksize , 
    C=constant,
)

Vectorizing

opencv uses numpy arrays internally, so you can use all the vectorisation goodies, instead of iterating over each pixel twice in python land.

bw_threshold = 150
limits = 0.2, 0.15

mask = image < bw_threshold
edges = []
for axis in (0, 1):
    count = mask.sum(axis=axis)
    limit = limits[axis] * image.shape[axis]
    index = np.where(count > limit)
    _min, _max = index[0][0], index[0][-1]
    edges.append((_min, _max))

does the same, but vectorized and about 1000 times faster.


Final result

def preproces_image(
    image,
    *,
    kernel_size=15,
    crop_side=50,
    blocksize=35,
    constant=15,
    max_value=255,
):
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    bit = cv2.bitwise_not(gray)
    image_adapted = cv2.adaptiveThreshold(
        src=bit,
        maxValue=max_value,
        adaptiveMethod=cv2.ADAPTIVE_THRESH_MEAN_C,
        thresholdType=cv2.THRESH_BINARY,
        blockSize=blocksize,
        C=constant,
    )
    kernel = np.ones((kernel_size, kernel_size), np.uint8)
    erosion = cv2.erode(image_adapted, kernel, iterations=2)
    return erosion[crop_side:-crop_side, crop_side:-crop_side]

def find_edges(image_preprocessed, *, bw_threshold=150, limits=(0.2, 0.15)):
    mask = image_preprocessed < bw_threshold
    edges = []
    for axis in (1, 0):
        count = mask.sum(axis=axis)
        limit = limits[axis] * image_preprocessed.shape[axis]
        index_ = np.where(count >= limit)
        _min, _max = index_[0][0], index_[0][-1]
        edges.append((_min, _max))
    return edges


def adapt_edges(edges, *, height, width):
    (x_min, x_max), (y_min, y_max) = edges
    x_min2 = x_min
    x_max2 = x_max + min(250, (height - x_max) * 10 // 11)
    # could do with less magic numbers
    y_min2 = max(0, y_min)
    y_max2 = y_max + min(250, (width - y_max) * 10 // 11)
    return (x_min2, x_max2), (y_min2, y_max2)

if __name__ == "__main__":

    filename_in = "NHnV7.png"
    filename_out = "res_NHnV7.png"

    image = cv2.imread(str(filename_in))
    height, width = image.shape[0:2]
    image_preprocessed = preproces_image(image)
    edges = find_edges(image_preprocessed)
    (x_min, x_max), (y_min, y_max) = adapt_edges(
        edges, height=height, width=width
    )
    image_cropped = image[x_min:x_max, y_min:y_max]
    cv2.imwrite(str(filename_out), image_cropped)
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.