7
\$\begingroup\$

I have an input like this:

x = '415,252,428,233,428,214,394,207,383,205,390,195'

And I want to end up with this

y = [(415,252),(428,233),(428,214),(394,207),(383,205),(390,195)]

The closes I got is

tmp = [int(i) for i in x.split(',')]
y = zip(tmp[::2], tmp[1::2])

How do I get rid of the intermediate list? I need alternative to [::2] that could be used on a generator.

\$\endgroup\$
  • 1
    \$\begingroup\$ Maybe you can use chunked or sliced from the more-itertools package? \$\endgroup\$ – Energya Feb 14 at 7:39
11
\$\begingroup\$

To reduce memory usage you can exploit the mechanics of zip and iter / iterator / generator expression.

  1. Make tmp an iterator.

    You can achieve this by changing the brackets from [] to (); changing it from a list comprehension to a generator expression.

    You can alternately wrap the list comprehension in an iter call. However that would still be using \$O(n)\$ memory.

  2. In case you don't know what an iterator is, than it's a very useful design pattern.

    In short the iterator contains in it some data, and a single method to get the next bit of information. In Python the next function calls this method.

    Take the iterator a list provides:

    >>> my_data = iter([1, 2, 3, 4])
    >>> next(my_data)
    1
    >>> next(my_data)
    2
    
  3. If you think of zip in terms of iterators, then you'll notice that with two input it works something like:

    def zip(a, b):
        while True:
            yield next(a), next(b)
    

    This means if a and b are the same then zip will get the first value and second value in a tuple. Then it will get the third and fourth, until it gets all values.

And so to improve memory usage you can change your code to:

tmp = (int(i) for i in x.split(','))
y = zip(tmp, tmp)

Note: this still runs in \$O(n)\$ space, as x.split() also makes an intermediary list. However it has cut the code from four intermediary lists to just one.


This is a common pattern, as provided in the itertools docs.

def grouper(iterable, n, fillvalue=None):
    "Collect data into fixed-length chunks or blocks"
    # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx"
    args = [iter(iterable)] * n
    return zip_longest(*args, fillvalue=fillvalue)
y = grouper((int(i) for i in x.split(',')), 2)
| improve this answer | |
\$\endgroup\$
  • \$\begingroup\$ Wow, I'm surprised it works that way. It makes sense, but I find this very counterintuitive \$\endgroup\$ – Cruncher Feb 13 at 21:37
  • \$\begingroup\$ May because you still think of tmp as a sequence, that's the mistake I made. \$\endgroup\$ – Kazz Feb 14 at 12:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.