# File deletion takes a long time to delete files

I have a script I need to "duplicate" and modify to allow it to delete certain images based on properties I give this app/script. Currently the script just moves/sorts the images into folders based on numbers (e.g. each folder has 5000 images before a new folder is created and more images propagated).

The requirement of this is:

• I need to remove any images that are; width<400 && height<400
• I need to increase the speed of which they are filtered (currently 5/second)
• I need to have interchangeable code so that I can apply/remove this to/from the sorting script and be able to add conditions (e.g. is the file corrupt?)

Issue:

The deletion method works, but only just. Where the moving of files into folders can do 60,000 images in around 30 seconds I can't delete and move at the same or similar rate. Instead I can do about 5 images per second.

Sorting Script:

def confirmIt():
#======== Confirm Selection and Move files to new sub-directory:
# Confirm selection and move files to new sub-directory

source = folderPath.get()                                               # set source path
if source:                                                              # make sure not blank
fileCount = 0                                                       ## Counts the number of files going into a folder
folderCount = 1                                                     ## Helps count the folders
dest =  os.path.join(source, "Folder ("+str(folderCount)+")")       ## Set the folder destination for manual filtering

for fname in os.listdir(source):
if fname.lower().endswith(extensions):                            ## Ensure we are looking at images
image_path = os.path.join(source, fname)                       ## set the source location of the image
fileCount = fileCount+1                                        ## Start counting how many files we have in this folder

if (fileCount > 5000 ):                                        ## If the file count is more than 5000 we want to put them in a new folder
fileCount = 1
folderCount = folderCount+1
dest = os.path.join(source, "Folder ("+str(folderCount)+")")

if not os.path.exists(dest):
try:
os.makedirs(dest)                                       ## Create the destination if it doesn't exist
except OSError, e:                                          ## if failed, report it back to the user
print(e)

os.rename(image_path, os.path.join(dest, fname))                ## Move the file to destination path
else:
print("don't be a dipstick")


Embedded deletion script:

This will be replacing the original "for fname" statement on line 24:

for fname in os.listdir(source):
if fname.lower().endswith(extensions):                              ## Ensure we are looking at images
image_path = os.path.join(source, fname)                        ## set the source location of the image

progressbar["value"]=value

try:
with open(image_path, 'rb') as filehandle:
img = Image.open(filehandle)
except IOError, err:
print(err)
else:
width, height = img.size                                ## get the dimensions
size = width * height / 1000
#img.close()

if (size < 400 ):
os.remove(image_path)
else:
fileCount = fileCount+1                                        ## Start counting how many files we have in this folder
if (fileCount > 5000 ):                                        ## If the file count is more than 5000 we want to put them in a new folder
fileCount = 1
folderCount = folderCount+1
dest = os.path.join(source, "Folder ("+str(folderCount)+")")

if not os.path.exists(dest):
try:
os.makedirs(dest)
except OSError, e:                                          ## if failed, report it back to the user
print(e)

os.rename(image_path, os.path.join(dest, fname))


Information:

I have been working with this in an more pythonic way, however that just seems to hang completely because of how each name is stored in an array before further progress is made and I don't get any feed back on my computer. So for now I'm using this way.

• So you're just gonna start over from where you were before? Aug 13 '15 at 4:39
• @Cyphase I've done it this way because the code works currently and I need to use said code while I'm editing it ( I currently have a job on with 70,000 images I need to filter, while leaving the corrupt files in). I've still got the other one, but it's not working with the 400x400 specs and I was told to move this specific post here from stackoverflow. I need to use the filter to get rid of thousands of thumbnails so it's currently the priority. And thus the use of the "working" code, if you can edit what you already had feel free I will update this code at that Point in time. Aug 13 '15 at 4:47
• @Cyphase also this has incorporated your original fix of tidiness which also happens to be the one that currently works the best. Like I mentioned the deletion process does work with this, only it works really slow, whereas the other falters on the 400x400. just in case that wasn't clear Aug 13 '15 at 4:57
• Ah, I see. I did put up a version that removed small images, but that edit was reverted. Here's that revision: stackoverflow.com/revisions/31930807/8 Aug 13 '15 at 5:01
• @Cyphase yea, I have that revision, but there are 2 things that don't work. a) the implementation of interchangeable functions (e.g. being able to use it with out the corrupt file check) b) The fact there is no visible feed back or it just crashes Aug 13 '15 at 22:05