4
\$\begingroup\$

I have a script I need to "duplicate" and modify to allow it to delete certain images based on properties I give this app/script. Currently the script just moves/sorts the images into folders based on numbers (e.g. each folder has 5000 images before a new folder is created and more images propagated).

The requirement of this is:

  • I need to remove any images that are; width<400 && height<400
  • I need to increase the speed of which they are filtered (currently 5/second)
  • I need to have interchangeable code so that I can apply/remove this to/from the sorting script and be able to add conditions (e.g. is the file corrupt?)

Issue:

The deletion method works, but only just. Where the moving of files into folders can do 60,000 images in around 30 seconds I can't delete and move at the same or similar rate. Instead I can do about 5 images per second.

Sorting Script:

def confirmIt():
    #======== Confirm Selection and Move files to new sub-directory:
    # Confirm selection and move files to new sub-directory

    source = folderPath.get()                                               # set source path
    if source:                                                              # make sure not blank
        fileCount = 0                                                       ## Counts the number of files going into a folder
        folderCount = 1                                                     ## Helps count the folders
        dest =  os.path.join(source, "Folder ("+str(folderCount)+")")       ## Set the folder destination for manual filtering

        for fname in os.listdir(source):
            if fname.lower().endswith(extensions):                            ## Ensure we are looking at images
                image_path = os.path.join(source, fname)                       ## set the source location of the image
                fileCount = fileCount+1                                        ## Start counting how many files we have in this folder

                if (fileCount > 5000 ):                                        ## If the file count is more than 5000 we want to put them in a new folder
                    fileCount = 1
                    folderCount = folderCount+1
                    dest = os.path.join(source, "Folder ("+str(folderCount)+")")

                if not os.path.exists(dest):
                    try:
                        os.makedirs(dest)                                       ## Create the destination if it doesn't exist
                    except OSError, e:                                          ## if failed, report it back to the user
                        print(e)

                os.rename(image_path, os.path.join(dest, fname))                ## Move the file to destination path
    else:
        print("don't be a dipstick")

Embedded deletion script:

This will be replacing the original "for fname" statement on line 24:

for fname in os.listdir(source):
        if fname.lower().endswith(extensions):                              ## Ensure we are looking at images
            image_path = os.path.join(source, fname)                        ## set the source location of the image


            progressbar["value"]=value

            try: 
                with open(image_path, 'rb') as filehandle:
                    img = Image.open(filehandle)
                    #img.load()
            except IOError, err:
                print(err)
            else:
                width, height = img.size                                ## get the dimensions
                size = width * height / 1000
                #img.close()

                if (size < 400 ):
                    os.remove(image_path)
                else:
                    fileCount = fileCount+1                                        ## Start counting how many files we have in this folder
                    if (fileCount > 5000 ):                                        ## If the file count is more than 5000 we want to put them in a new folder
                        fileCount = 1
                        folderCount = folderCount+1
                        dest = os.path.join(source, "Folder ("+str(folderCount)+")")

                    if not os.path.exists(dest):
                        try:
                            os.makedirs(dest)   
                        except OSError, e:                                          ## if failed, report it back to the user
                            print(e)

                    os.rename(image_path, os.path.join(dest, fname))

Information:

I have been working with this in an more pythonic way, however that just seems to hang completely because of how each name is stored in an array before further progress is made and I don't get any feed back on my computer. So for now I'm using this way.

\$\endgroup\$
  • \$\begingroup\$ So you're just gonna start over from where you were before? \$\endgroup\$ – Cyphase Aug 13 '15 at 4:39
  • \$\begingroup\$ @Cyphase I've done it this way because the code works currently and I need to use said code while I'm editing it ( I currently have a job on with 70,000 images I need to filter, while leaving the corrupt files in). I've still got the other one, but it's not working with the 400x400 specs and I was told to move this specific post here from stackoverflow. I need to use the filter to get rid of thousands of thumbnails so it's currently the priority. And thus the use of the "working" code, if you can edit what you already had feel free I will update this code at that Point in time. \$\endgroup\$ – CFNZ_Techie Aug 13 '15 at 4:47
  • \$\begingroup\$ @Cyphase also this has incorporated your original fix of tidiness which also happens to be the one that currently works the best. Like I mentioned the deletion process does work with this, only it works really slow, whereas the other falters on the 400x400. just in case that wasn't clear \$\endgroup\$ – CFNZ_Techie Aug 13 '15 at 4:57
  • \$\begingroup\$ Ah, I see. I did put up a version that removed small images, but that edit was reverted. Here's that revision: stackoverflow.com/revisions/31930807/8 \$\endgroup\$ – Cyphase Aug 13 '15 at 5:01
  • \$\begingroup\$ @Cyphase yea, I have that revision, but there are 2 things that don't work. a) the implementation of interchangeable functions (e.g. being able to use it with out the corrupt file check) b) The fact there is no visible feed back or it just crashes \$\endgroup\$ – CFNZ_Techie Aug 13 '15 at 22:05
2
\$\begingroup\$

I have managed to create a workaround while doing this:

  1. I have changed the GUI code to run in the Python CMD to let the user know files are being moved/deleted.
  2. I have used the code found in this post. This basically fixes errors as well as some time problems between python and Tkinker. I will take a further look into this when the need arises.

Answer:

  • Edited code with fix from Stack Overflow linked above.
  • Changed user feed back to Python-based command line instead of Tkinker GUI.
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.