3
\$\begingroup\$

I'm currently using SciPy's mode function to find the most occurring item in different iterable objects. I like the mode function because it works on every object type I've thrown at it (strings, floats, ints).

While this function seems reliable, it is very slow on bigger lists:

from scipy.stats import mode

li = range(50000)
li[1] = 0

%timeit mode(li)
1 loops, best of 3: 7.55 s per loop

Is there a better way to get the mode for a list? If so, would the implementation be different depending on the item type?

\$\endgroup\$
5
\$\begingroup\$

Originally I thought you must have been doing something wrong, but no: the scipy.stats implementation of mode scales with the number of unique elements, and so will behave very badly for your test case.

As long as your objects are hashable (which includes your three listed object types of strings, floats, and ints), then probably the simplest approach is to use collections.Counter and the most_common method:

In [33]: import scipy.stats

In [34]: li = range(50000); li[1] = 0

In [35]: scipy.stats.mode(li)
Out[35]: (array([ 0.]), array([ 2.]))

In [36]: timeit scipy.stats.mode(li)
1 loops, best of 3: 10.7 s per loop

but

In [37]: from collections import Counter

In [38]: Counter(li).most_common(1)
Out[38]: [(0, 2)]

In [39]: timeit Counter(li).most_common(1)
10 loops, best of 3: 34.1 ms per loop
|improve this answer|||||
\$\endgroup\$
  • 1
    \$\begingroup\$ Wow! What an improvement! \$\endgroup\$ – ChrisArmstrong Dec 29 '12 at 22:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.