Zipping two lists with an offset in Python

Question

Walking my first steps in Python from C, I find myself missing pointers from time to time.

In this case I want to go through a list processing two element at a time, where those elements are step places away. This is my initial approach, but I would be happy to learn a couple alternatives, and their advantages.

v = [0,1,2,3,4,5,6,7,8,9]
step = 3
for x, y in zip( v[:-step], v[step:] ) :
  print("\nx={}, y={}".format(x, y))

I'm interested in what can be done with naked Python alone first. But alternatives from a module, are welcome as well. I want to know which approach is closer to pointer use in C (which was my intent with the code above) in terms of efficiency.

Answer 1

You're on the right track, but it can be simplified to the following. Note also that the step variable name could add confusion because Python ranges and slices have a step attribute -- but it's not what you are doing:

start = 3
for x, y in zip(v, v[start:]):
    ...

Also note that syntax like v[start:] creates a new list. If you are dealing with large data volumes and want to avoid that, you can use itertools.islice.

from itertools import islice

v = [0,1,2,3,4,5,6,7,8,9]

start = 3
for x, y in zip(v, islice(v, start, None)):
    print(x, y)

Answer 2

Using itertools.tee to zip an iterable with itself

Since you are iterating twice on the same iterable, one possibility is to use itertools.tee:

from itertools import tee

def zip_offset(iterable, offset):
    a, b = tee(iterable)
    for _ in range(offset):
        next(b, None)
    return zip(a, b)

print(list( zip_offset(range(10), 3) ))
# [(0, 3), (1, 4), (2, 5), (3, 6), (4, 7), (5, 8), (6, 9)]

Using tee is particularly important if the iterable is an iterator. Consider:

from itertools import tee, islice

v = map(lambda x: 2*x, [0,1,2,3,4,5,6,7,8,9])
a, b = tee(v)
print(list( zip(a, islice(b, 3, None)) ))
# [(0, 6), (2, 8), (4, 10), (6, 12), (8, 14), (10, 16), (12, 18)]

v = map(lambda x: 2*x, [0,1,2,3,4,5,6,7,8,9])
print(list( zip(v, islice(v, 3, None)) ))
# [(0, 8), (10, 12), (14, 16)]

more_itertools.zip_offset

Alternatives from a module are welcome as well

Meet module more_itertools and its function zip_offset:

from more_itertools import zip_offset

print(list( zip_offset('01234', 'abcdefghijkl', offsets=(0, 3)) ))
# [('0', 'd'), ('1', 'e'), ('2', 'f'), ('3', 'g'), ('4', 'h')]

You can look at its source code:

from itertools import islice
from itertools import chain, repeat # used when negative offsets
from itertools import zip_longest   # used when longest=True

def zip_offset(*iterables, offsets, longest=False, fillvalue=None):
    if len(iterables) != len(offsets):
        raise ValueError("Number of iterables and offsets didn't match")
    staggered = []
    for it, n in zip(iterables, offsets):
        if n < 0:
            staggered.append(chain(repeat(fillvalue, -n), it))
        elif n > 0:
            staggered.append(islice(it, n, None))
        else:
            staggered.append(it)
    if longest:
        return zip_longest(*staggered, fillvalue=fillvalue)
    return zip(*staggered)

In the special case when the offset is 1, and assuming you're using python>=3.10, there is also pairwise in itertools:

from itertools import pairwise

print(list( pairwise('abcdef') ))
# [('a', 'b'), ('b', 'c'), ('c', 'd'), ('d', 'e'), ('e', 'f')]

Answer 3

I want to know which approach is closer to pointer use in C in terms of efficiency

Pointers are very much not a thing in Python; and attempts to get toward them often end up in non-Pythonic code. @FMc's answer is a "correct and Pythonic" method, but is very much not like pointers. zip and islice are stateful iterators that carry context.

If I had to show code that would be the closest equivalent to pointers, it would be plain indexing:

v = range(10)

start = 3
for i in range(len(v) - start):
    x = v[i]
    y = v[i + start]
    print(x, y)

But again, unless you have a really good reason (and have eliminated all other options than micro-optimizing), don't do this.

Answer 4

Without offset, i.e., with step = 0, I'd expect the same results as for normal zip. But you produce nothing, because v[:-step] is empty then.

For negative offsets you do produce outputs, but I'd say rather not what one might want. For example for v = [0,1,2,3,4,5,6,7,8,9] and step = -3 you produce (0, 7), (1, 8), etc. Might be desirable to produce (3, 0), (4, 1), etc.

Your print("\nx={}, y={}".format(x, y)) can be shortened: print(f"{x=}, {y=}") (same output, except I removed the leading newline which just caused extra empty lines which seemed pointless).