Image Processing - Comparing 2 images and Ranking Similarity

Question

My code compares 2 images of any shape/dimension and ranks them in order of similarity. It starts with reading from a CSV file with columns image1, image2 which contain absolute paths and then outputting to a CSV file which contain columns image1, image2, similarity, time_elapsed

I'm more concerned with the formatting suggestions or simplifications to make the code more easy to read

import cv2
import numpy as np
import csv
import time

#read from CSV file
def read_data():
    print("starting to read data")
    with open('input_test_data.csv') as input_data:
        print("opened data file")
        global line_count
        csv_reader = csv.reader(input_data, delimiter=',')
        set_of_images=[row for idx, row in enumerate(csv_reader) if idx == line_count]
        print(set_of_images)
        original_image = set_of_images[0][0]
        image_to_compare = set_of_images[0][1]
        line_count += 1
        return [original_image, image_to_compare]

# wrote column titles for result CSV file
def write_header():
    global line_count
    with open('result_data.csv', mode='w') as result:
        result_writer = csv.writer(result, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
        result_writer.writerow(['image1', 'image2', 'similarity_ratio', 'time_elapsed'])
    print('wrote result header')
    line_count += 1

# count total number of entries in CSV file
def count_images():
    with open('input_test_data.csv') as input_data:
        print("counting number of rows in CSV file")
        csv_reader = csv.reader(input_data, delimiter=',')
        return sum(1 for row in csv_reader)


# reading images
def read_images(original_img: str, image_to_compare: str):
    original_image = cv2.imread(original_img)
    compared_image = cv2.imread(image_to_compare)
    print("Read images")
    return [original_image, compared_image]


# preliminary check of RGB values are same 
def check_identical(original_image , compared_image , start_time: float, image_names):
    # if images are the same shape and size - could be identical
    if original_image.shape == compared_image.shape:
        print("Images have the same size and channel")
        difference = cv2.subtract(original_image, compared_image)   # subtract RGB values between images to see difference between image organization
        b,g, r = cv2.split(difference)

        # if RGB value difference is 0 - same image
        if cv2.countNonZero(b) == 0 and cv2.countNonZero(g) == 0 and cv2.countNonZero(r) == 0:
            publish_results(image_names[0], image_names[1], 0.0,  start_time)
            print("The images are completely Equal")
            return True

        else:
            print("The images are NOT completely Equal")
            return False

# if images are not identical:
# use OpenCV's SIFT Algorithm to find key points and features between images
def generate_keypoints(original_image: str, compared_image: str):
    sift = cv2.xfeatures2d.SIFT_create()
    key_point_image1, descriptor_image1 = sift.detectAndCompute(original_image, None)
    key_point_image2, descriptor_image2 = sift.detectAndCompute(compared_image, None)
    print("Generated keypoints for both images")
    return [key_point_image1, descriptor_image1, key_point_image2, descriptor_image2]

# FLANN - Fast Library for Approximate Neighbors
# uses Euclidean distance between common key points to judge similarity  
def compare_images(key_point_image1, descriptor_image1, key_point_image2, descriptor_image2  ):
    index_params = dict(algorithm=0, trees=5)
    search_params = dict()
    flann = cv2.FlannBasedMatcher(index_params, search_params)
    matches = flann.knnMatch(descriptor_image1, descriptor_image2, k=2)
    print("found common points in images")
    return matches

# SIFT algorithm has high recall, but low precision
# so a threshold ratio needs to be set to filter inacurate results
def filter_results(matches: enumerate):
    good_points = 0
    ratio = 0.6 # approximated through trial and error
    print("Matches:" + str(len(matches)))
    for m, n in matches:
            if m.distance < ratio*n.distance:
                    good_points += 1
    print("Good Matches: " + str(good_points))
    return [good_points, len(matches)]


# Generate Similarity Score
def generate_similarity_score(correct_matches: int, total_matches: int ):
    return 1.0 - correct_matches/total_matches   # as defined in requirements, values close to 0 is most similar, so more correct matches mean similar pictures

# Publish to CSV
def publish_results(original_img: str, image_to_compare: str, similarity_ratio , start_time):
    print("writing to CSV file...")
    with open('result_data.csv', mode='a', newline='') as result:
        result_writer = csv.writer(result, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
        result_writer.writerow([original_img, image_to_compare, str(similarity_ratio), str(round(time.time() - start_time, 2))] )
        print("Write successful")

# Main Function
# reading data from CSV
print("starting program")
line_count = 0

# find number of items in CSV file to compare
total_images_to_compare = count_images()

# prepare column titles for result CSV file
write_header()

for img in range(line_count, total_images_to_compare):

    # start recording execution time
    start_time = time.time()

    # read set of images
    image_names = read_data()

    # read original image and image to compare into memory
    images = read_images(image_names[0], image_names[1])


    # determine if identical images
    if (check_identical(images[0], images[1], start_time, image_names)):
        print("continue")
       # continue

    print("images NOT EQUAL - starting SIFT")

    # if images are not equal - start using SIFT to generate keypoints and features of both images to measure similarity
    keypoints_descriptors = generate_keypoints(images[0], images[1])

    # Attempt at finding common points in both images
    matches = compare_images(keypoints_descriptors[0], keypoints_descriptors[1], keypoints_descriptors[2], keypoints_descriptors[3] )

    print("filtering results now...")
    # filter out outliers and mismatches
    acurate_points = filter_results(matches)

    print("generating a score...")
    # find a ratio for closely matched images are
    score = generate_similarity_score(acurate_points[0], acurate_points[1])

    # write data to CSV file
    publish_results(image_names[0], image_names[1], score, start_time)

    # increment next set of images to be compared
    img += 1

This is the input_test_data.csv:

image1  image2
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\black_and_white.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\blue_filter.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\blurred.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\cartoonized.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\exposured.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\george-washington-bridge.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\old_photo.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\original_golden_bridge.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\overlay.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\resized.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\rotated.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\sharpened.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\sunburst.jpg
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\textured.jpg

result_data.csv:

image1  image2  similarity_ratio    time_elapsed

D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\black_and_white.jpg  0.308852067 3.55
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\blue_filter.jpg  0.13136289  3.45
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\blurred.jpg  0.944469324 1.46
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\cartoonized.jpg  0.601731602 3.17
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\exposured.jpg    0.736229288 2.91
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\george-washington-bridge.jpg 1   6.25
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\old_photo.jpg    0.637557844 4.07
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\original_golden_bridge.jpg   0   0.14
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\duplicate.jpg    0   3.55
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\overlay.jpg  0.824600687 2.11
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\resized.jpg  0.907598149 1.2
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\rotated.jpg  0.150619495 3.51
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\sharpened.jpg    0.522316764 5.07
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\sunburst.jpg 0.922675026 8.92
D:\Programming\test\images\original_golden_bridge.jpg   D:\Programming\test\images\textured.jpg 0.494103598 4.84

I apologize for the formatting for the results.csv

I also intend to replace all print statements to logging statements to make it easier for future developers and more maintainable while being professional

Answer 1

Disclaimer: The code below is (partly) untested, so there can be some rough edges that need polishing.

---

Style

First, some style-related notes. Python has an official Style Guide, often just called PEP8. The part most relevant to your code IMHO is documentation strings. In essence, you should follow the official recommendation to put your function documentation in """triple quotes""" inside the function body. An example adapted from your code:

def compare_images(key_point_image1, descriptor_image1,
                   key_point_image2, descriptor_image2):
    """
    uses Euclidean distance between common key points to judge similarity
    in FLANN (Fast Library for Approximate Neighbors) matching
    """
    ...

You often use an explicit list to return multiple values from a function. This is unusual and likely unnecessary. Most often there is no explicit type when returning multiple values, e.g.

def swap(a, b):
    return b, a

Under the hood, Python returns a tuple from the function as can be seen if looking at type(swap(1, 2)) which will happily print tuple in an interactive Python interpreter.

Avoid global variables

You should try avoid global variables whenever possible. And in your case the global variable is absolutely not necessary. We will cover that in a second.

read_data

read_data is written wasteful. You always read the full file just to get a single line. The usual approach would be to read the file line by line and then process it accordingly. Or read the file as a whole and process its contents line by line. This would also allow you to get rid of line_count to keep track of which line you need to look at next. The name of the file to read should probably also be a parameter and not hardcoded. The new version of read_data might look like:

def read_data(input_file):
    """read from CSV file"""
    with open(input_file) as input_data:
        csv_reader = csv.reader(input_data, delimiter=',')
        return [row for row in csv_reader]

This implementation uses a list comprehension to read all lines of the csv file into memory. With this, count_images() could simply be replaced by

images_list = read_data(...)
total_images_to_compare = len(images_list)

You could also use a generator expression which is basically a list comprehension, but not all the elements have to be kept in memory at once. Note that with a generator expression the above would not work because len(...) is not defined for generator expressions.

Either version could be used as follows in the main part of the script:

for image_names in images_list:
    # read original image and image to compare into memory
    images = read_images(image_names[0], image_names[1])

    ...

check_identical

Next up on the list is check_identical. IMHO the implementation here is also a lot more complicated than it has to be. numpy can greatly help to simplifiy the code here:

def check_identical(original_image, compared_image):
    """preliminary check of RGB values are same"""
    if original_image.shape == compared_image.shape:
        # if images are the same shape and size - could be identical
        return np.count_nonzero(original_image - compared_image) == 0

    return False

In its core, the function still does what you did, but for the whole image instead of separately for each color channel. The major difference here is that start_time and image_names have vanished from the parameter list, since there is reason why check_indentical should have to care about writing anything to an output file.

tuple unpacking

There are several parts in your code where you do foobar(images[0], images[1]). If images has the same number of elements as expected by foobar(...), you can simply to foobar(*images). This is called tuple unpacking, but also works for other sequences like lists.

Also, sometimes its more understandable to assign multiple return values to named variables to better see what gets passed around, e.g.

correct_matches, total_matches = filter_results(matches)

score = generate_similarity_score(correct_matches, total_matches)

Sidenote: it would be also possible to do:

score = generate_similarity_score(*filter_results(matches))

Hard-coded values

There are also quite a lot of hard-coded values in your code. Some examples:

'input_test_data.csv'
'result_data.csv'
index_params = dict(algorithm=0, trees=5)
matches = flann.knnMatch(..., k=2)
ratio = 0.6 # approximated through trial and error

IMHO, they all should be function parameters that can be changed without touching the actual code. You can also assign them default values that match their current ones, so nothing in your program has to be changed. At the current state if you ever want to change some of these values, you would have to jump back to where the function is implemented and change it there directly. Though that might sound feasible for a few values, the trouble really starts if you ever get the idea to try several values in a single run (maybe in an effort to optimize a parameter).

The main function

Although your code boldly states # Main Function at the end, there is not an actual main function to be found here. But there definitely should be one! And since we are at it, it's also good practice to make sure the main function of the script is only called if the file is actually used as a script, and not imported into some other code. Enter the top-level script environment!

# ... all the import and other functions here ...

def main():
    """Read a list of image pairs from disk and check their similarity"""
    ...

if __name__ == "__main__":
    main()

See this SO post for an extended explanation about __name__.