We are developing an automatic text Captcha solver in python 3. The solver has a module that is responsible for extracting letters out of captcha images that contain 4 letters each. We would like to get a review of this module and understand if there are any ways to write it more elegantly.

Here is an example of a captcha image:

captcha image


import os
import os.path
import cv2
import glob
import imutils

CAPTCHA_IMAGE_FOLDER = "generated_captcha_images"
OUTPUT_FOLDER = "extracted_letter_images"

# Get a list of all the captcha images we need to process
captcha_image_files = glob.glob(os.path.join(CAPTCHA_IMAGE_FOLDER, "*"))
counts = {}

# loop over the image paths
for (i, captcha_image_file) in enumerate(captcha_image_files):
    print("[INFO] processing image {}/{}".format(i + 1, len(captcha_image_files)))

    # Since the filename contains the captcha text (i.e. "2A2X.png" has the text "2A2X"),
    # grab the base filename as the text
    filename = os.path.basename(captcha_image_file)
    captchaCorrectText = os.path.splitext(filename)[0]

    # Load the image and convert it to grayscale
    image = cv2.imread(captcha_image_file)
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

    # Add some extra padding around the image
    gray = cv2.copyMakeBorder(gray, 8, 8, 8, 8, cv2.BORDER_REPLICATE)

    # Threshold the image (convert it to pure black and white)
    Thresh = cv2.Threshold(gray, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]

    # find the contours (continuous blobs of pixels) the image
    contours = cv2.findContours(Thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    # Hack for compatibility with different OpenCV versions
    if imutils.is_cv2():
        contours = contours[0]
        contours = contours[1]

    letter_image_regions = []

    # Now we can loop through each of the four contours and extract the letter
    # inside of each one
    for contour in contours:
        # Get the rectangle that contains the contour
        (x, y, w, h) = cv2.boundingRect(contour)

        # Compare the width and height of the contour to detect letters that
        # are conjoined into one chunk
        if w / h > 1.25:
            # This contour is too wide to be a single letter!
            # Split it in half into two letter regions!
            half_width = int(w / 2)
            letter_image_regions.append((x, y, half_width, h))
            letter_image_regions.append((x + half_width, y, half_width, h))
            # This is a normal letter by itself
            letter_image_regions.append((x, y, w, h))

    # If we found more or less than 4 letters in the captcha, our letter extraction
    # didn't work correcly. Skip the image instead of saving bad training data!
    if len(letter_image_regions) != 4:

    # Sort the detected letter images based on the x coordinate to make sure
    # we are processing them from left-to-right so we match the right image
    # with the right letter
    letter_image_regions = sorted(letter_image_regions, key=lambda x: x[0])

    # Save out each letter as a single image
    for letter_bounding_box, letter_text in zip(letter_image_regions, captchaCorrectText):
        # Grab the coordinates of the letter in the image
        x, y, w, h = letter_bounding_box

        # Extract the letter from the original image with a 2-pixel margin around the edge
        letter_image = gray[y - 2:y + h + 2, x - 2:x + w + 2]

        # Get the folder to save the image in
        save_path = os.path.join(OUTPUT_FOLDER, letter_text)

        # if the output directory does not exist, create it
        if not os.path.exists(save_path):

        # write the letter image to a file
        count = counts.get(letter_text, 1)
        p = os.path.join(save_path, "{}.png".format(str(count).zfill(6)))
        cv2.imwrite(p, letter_image)

        # increment the count for the current key
        counts[letter_text] = count + 1
  • \$\begingroup\$ Welcome to Code Review! Would it be possible to include one or more example CAPTCHAs so a reviewer can easily test run your code? \$\endgroup\$
    – AlexV
    Commented May 3, 2019 at 12:52

2 Answers 2


Without some sample inputs I can only really comment on how the code reads, less on how it runs.

  • First, this should really be a few functions, I'll indicate below where might be good places to separate the code.

  • Why import os and os.path? Just import os then call os.path.<something>

  • You don't need to wrap i, captcha_image_file in brackets when using enumerate(); enumerate() also takes an optional start argument; so, using f-strings as well, you can write:

for count, captcha_image_file in enumerate(captcha_image_files, 1):
    print(f"[INFO] processing image {count}/{len(captcha_image_files)}")

count typically indicates starting from 1, where pos or index would indicate starting from 0. Avoid using short, non-descriptive variables like i.

  • The "Hack" can be shortened nicely with a ternary operator:
contours = contours[0] if imutils.is_cv2() else contours[1]
  • I would separate the block immediately following this into a function, which would yield, rather than append to a list. This will be faster, cleaner, and get rid of dummy variables.

  • I would use itemgetter rather than a lambda function here, entirely a style choice, though.

  • Contrary to what I said earlier, I actually think using x, y, w, h is fine here as they are commonly understood to be the x and y directions, width and height.

  • Again, this last block (for letter_bounding_box, letter_text ...) should be its own function.

  • Really, I think this whole piece of code should be a class, the globals can then be class variables, and you can have three methods. You can then have a main() function which calls these methods and keeps track of which file you're on etc. This main() should sit inside: if __name__ == '__main__':

  • Overall comments are good, somewhat excessive in parts but better than too sparse.

The bulk of the improvements would come from separating out your code into readable methods. Functions / methods should generally have one clear purpose. This also has the advantage of making generators easier to use as you can easily yield rather than constantly making lists.


In captchas, there are times where letters are overlapping, in this case it would be considered to be only one letter because of the algorithms you use. You should consider eroding your image to try to detach letters when they are slightly overlapping. If they overlap a lot, you'll need to find another algorithm.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.