I need to iterate over a large (in the thousands) list of files in a network folder and "concatenate" any image sequences into just one entry with the range of images (first image, last image in sequence). I am calling this "sequence pruning" and I created the following code which does work, but it seems incredibly un-pythonic to me and runs slowly. I'm certain there is a MUCH better way to do this, so I am looking for review to help clean/speed this up.
To elaborate a bit further on the issue lets say I have this as a list of files for the input:
img.001.png img.002.png img.003.png img_other.001.png random_file.txt yet another seq.0000.png yet another seq.0001.png yet another seq.0002.png yet another seq.0021.png yet another seq.0030.png
In the end I want to return something like this:
img.001.png, [1-3] img_other.001.png random_file.txt yet another seq.0000.png, [0-30]
FYI I can safely assume that the image sequence number is always going to be a series of digits at the very end of the filename (before the extension of course). However I cannot assume that they will be perfectly sequential, as there are sometimes "gaps" between numbers.
Here's my current code, python 2.7:
import os def split_padding(path): file, ext = os.path.splitext(path) pad_int = 0 while file[pad_int * -1 - 1].isdigit(): pad_int += 1 if pad_int == 0: return file, '0', ext clean_file = file[0:pad_int * -1] padding = file[pad_int * -1:] return clean_file, padding, ext def strip_padding(path): file, ext = os.path.splitext(path) while file[-1].isdigit(): file = file[:-1] return file def prune_files(paths): ''' sequences get put into arrays like so: [x_folder, z_folder, [test_a.000.png, 0, 2], [test_b.000.tif, 0, 3], test_C.000.png] :return: [file1, file2, [first_file, seq_start, seq_end]] ''' paths.sort(key=lambda s: s.lower()) # list has to be sorted for this to work # this odd bit of code turns all sequences into arrays of images. pruned_list =  switch = True for c, path in enumerate(paths): if c == 0: pruned_list.append(path) continue if not os.path.splitext(path) in ['.png', '.tif', '.tiff', '.exr', '.jpg', '.jpeg']: pruned_list.append(path) continue test = paths[c-1] if strip_padding(path) == strip_padding(test): if switch: pruned_list[-1] = [pruned_list[-1]] switch = False pruned_list[-1].append(path) else: pruned_list.append(path) switch = True # so now lets convert that to the format we want to return for c, item in enumerate(pruned_list): if type(item) == list: pruned_list[c] = [item, int(split_padding(item)), int(split_padding(item[-1]))] return pruned_list if __name__ == "__main__": test_dir = "some directory" print prune_files([path for path in os.listdir(test_dir) if os.path.isfile(os.path.join(test_dir, path))])