I have a list of tuples which may vary in size quite significantly. Typical number of tuples in the list would be somewhere around 100.000. I am trying to perform a down-selection on these tuples to end up with a more workable amount of them.
Each tuple contains (among other insignificant for this example data) an integer and 3 floats. These are of interest to me (shown bold below, positions 0, 2, 3, 4).
-23.412, -277.11, 12.567
, 12.24, 23.25, 150.14, 'bar', 'foobar')
And the criterion to implement is the following in pseudocode:
j leaves the list.
I wrote a couple algorithms for the task, timed them and improved them and finally I arrived here:
def clean3(lst): lst = sorted(lst, key=lambda x: abs(x), reverse=True) # the abs comparison is taken care off. remove = set() # for quicker membership checks for i, forces_i in enumerate(lst[:-1]): if forces_i not in remove: # the current entry might have already been removed for forces_j in lst[i+1:]: # forward check only - progressive gains if forces_j not in remove: # the current entry might have already been removed if all(forces_i[x] < forces_j[x] for x in range(2, 4)): # check with `all` now possible remove.add(forces_j) return list(set(lst) - remove)
To my eyes, this looks as streamlined as it could be, but:
- Can it be further improved, and if yes, how?
- Would an entirely different approach be better here?