Compressing large jpeg images

Question

I'm working with thousands of large image files in a regularly updated library. The following script does the job (on average reduces my file size ~95%) but costs me around 25 seconds to compress one image. Obviously, I can just let the script run overnight, but it would be cool if I can shave some time off this process. I'm mostly looking for any unnecessary redundancies or overhead in the script that can be trimmed out to speed up the process. I'm still new to Python, so go easy on me.

from PIL import Image
from pathlib import Path
import os, sys
import glob

root_dir = "/.../"

basewidth = 3500

for filename in glob.iglob(root_dir + '*.jpg', recursive=True):
    p = Path(filename)
    img = p.relative_to(root_dir)
    new_name = (root_dir + 'compressed/' + str(img))
    print(new_name)
    im = Image.open(filename)
    wpercent = (basewidth/float(im.size[0]))
    hsize = int((float(im.size[1])*float(wpercent)))
    im = im.resize((basewidth,hsize), Image.ANTIALIAS)
    im.save(new_name, 'JPEG', quality=40)

Answer 1

I think what greybeard is getting at in the comments is that you could get away with squishing these images much more than you presently are. It sounds like you're basically using the reduced versions as thumbnails, but these "thumbnails" are almost twice the width (over three times the area) of a standard HD monitor.

Dropping basewidth to 1920 (or possibly much lower still) seems like a good idea. Contrary to greybeard, I think your JPEG "quality" setting is fine, but you could play around with different variations of smaller-vs-larger and crisp-vs-compressed.

A minute of googling suggests that PIL is in fact the normal choice for image handling. Maybe there's something better, but I'll suppose not. Given that the originals are large (50MB? more?), it may simply be that there's a lot of work to be done. That said, there are some things you can try.

• General hardware: Given that this is a heavy task, you may have some luck just running the same code on a fancier computer. This is an expensive option; even if you feel like throwing money at the problem still do plenty of research first.
• Storage hardware: One possible bottleneck is the hard drive you're reading from and writing to. If you can easily try the task against a different (faster or slower) harddrive while keeping everything else the same, that may be informative.
• Memory: Does you computer have a lot of RAM? Is that memory available for this python script to use? If you have a way of making half the currently available memory unavaliable, check if that makes the process twice as slow.
• Memory leaks: I see the line im = Image.open(filename), and looking at the docs suggests that you should probably have a call to load or more likely close someplace.
• Parallelism: Your computer probably has multiple processor cores. It's hard to say if or how well this script is using all of those cores. If you try running your script twice at the same time (against non-overlapping target sets), how much slower is each concurrent process?
• It's not clear if ANTIALIAS is a valid option to the resize method in the 3.x version. Try NEAREST.
• Oh hey there's a thumbnail function that affects the way the file is read from disk!. Playing around with the underlying draft function may also be helpful.

And because this is code review:

• Move your format and quality constants up to the same place as basewidth.

Answer 2

Here is my updated script. Notable changes include the use of NEAREST instead of ANTIALIAS, as well as the inclusion of an EXIF copy and paste. I think the major hang on the original script was the inefficiency of ANTIALIAS, as this script gives me around 95% compression in about 2 seconds per image.

from PIL import Image
from pathlib import Path
import os, sys
import glob

root_dir = "/.../"

basewidth = 5504 #sets base width of new images

for filename in glob.iglob(root_dir + '*.jpg', recursive=True): #creates for loop to refeence all .jpg files in root directory
    p = Path(filename) #converts filename into Path object
    img = p.relative_to(root_dir) #uses Path function to parse out Path into components, then uses all components following that equal to the root_dir path name (in this case, our jpeg file names)
    new_name = (root_dir + 'compressed/' + str(img)) #creates new path to save compressed files to in subfolder "compressed" (note: must create subfolder before running)
    print(new_name)

    #resize and reduce
    im = Image.open(filename) #sets filename as object
    wpercent = (basewidth/float(im.size[0])) #uses the base width to establish aspect ratio
    hsize = int((float(im.size[1])*float(wpercent))) #scales image height using aspect ratio
    im = im.resize((basewidth,hsize), Image.NEAREST) #sets new resolution using basewidth and hsize
    exif = im.info['exif'] #copy EXIF data
    im.save(new_name, 'JPEG', exif = exif, quality=40)