11
\$\begingroup\$

Walking my first steps in Python from C, I find myself missing pointers from time to time.

In this case I want to go through a list processing two element at a time, where those elements are step places away. This is my initial approach, but I would be happy to learn a couple alternatives, and their advantages.

v = [0,1,2,3,4,5,6,7,8,9]
step = 3
for x, y in zip( v[:-step], v[step:] ) :
  print("\nx={}, y={}".format(x, y))

I'm interested in what can be done with naked Python alone first. But alternatives from a module, are welcome as well. I want to know which approach is closer to pointer use in C (which was my intent with the code above) in terms of efficiency.

\$\endgroup\$
5
  • 1
    \$\begingroup\$ Just as an aside, your string formatting could be simplified as print(f"\n{x=}, {y=}") with modern f-strings. \$\endgroup\$
    – Dronir
    Jan 7 at 7:25
  • \$\begingroup\$ Valuable tip. Makes it more readable. Thank you. \$\endgroup\$
    – xcaliph
    Jan 7 at 9:32
  • \$\begingroup\$ print(f"\nx={x}, y={y}") I assume \$\endgroup\$
    – xcaliph
    Jan 7 at 15:21
  • \$\begingroup\$ @Dronir No need to repeat what's already in an answer :-P. xcaliph: Why do you assume that? Our version works. \$\endgroup\$ Jan 7 at 15:35
  • \$\begingroup\$ Maybe I too should have edited instead of commenting… xcaliph: {x=} is a shorthand for x={x}, they do the same thing. \$\endgroup\$
    – Dronir
    Jan 8 at 16:14

4 Answers 4

17
\$\begingroup\$

You're on the right track, but it can be simplified to the following. Note also that the step variable name could add confusion because Python ranges and slices have a step attribute -- but it's not what you are doing:

start = 3
for x, y in zip(v, v[start:]):
    ...

Also note that syntax like v[start:] creates a new list. If you are dealing with large data volumes and want to avoid that, you can use itertools.islice.

from itertools import islice

v = [0,1,2,3,4,5,6,7,8,9]

start = 3
for x, y in zip(v, islice(v, start, None)):
    print(x, y)
\$\endgroup\$
4
  • \$\begingroup\$ Thanks a lot @FMc Now I realize I was trying to make a view but in fact i was making a copy. I believe numpy arrays do use views when slicing, hence my confusion. Also, I see that zip does no need for the lists to be equal lenth which I somehow asumed. The islice suggestion seems very fit for my needs here too. If I were to use only python, do you think buffer (or memoryview) would be a good alternative ? \$\endgroup\$
    – xcaliph
    Jan 6 at 10:16
  • 3
    \$\begingroup\$ @xcaliph itertools is only Python. That's like suggesting stdbool.h isn't C. \$\endgroup\$
    – wizzwizz4
    Jan 6 at 17:03
  • 1
    \$\begingroup\$ I've seen people not realizing how very inefficient such islice usage can be (like when you want to zip the first ten elements of a million-elements list with the last ten), so maybe point that out as well. \$\endgroup\$ Jan 6 at 22:49
  • 1
    \$\begingroup\$ @wizzwizz4 . Yeah, you're right. Maybe you can understand it as a learning process. I'm getting used to the native tools first so I get a taste of the language. That way I'll feel the need for those standard modules that are really part of the core, as you say. Thank to your responses to a simple question, I'm getting a better grasp than simply checking the documentation. \$\endgroup\$
    – xcaliph
    Jan 7 at 1:04
12
\$\begingroup\$

Using itertools.tee to zip an iterable with itself

Since you are iterating twice on the same iterable, one possibility is to use itertools.tee:

from itertools import tee

def zip_offset(iterable, offset):
    a, b = tee(iterable)
    for _ in range(offset):
        next(b, None)
    return zip(a, b)

print(list( zip_offset(range(10), 3) ))
# [(0, 3), (1, 4), (2, 5), (3, 6), (4, 7), (5, 8), (6, 9)]

Using tee is particularly important if the iterable is an iterator. Consider:

from itertools import tee, islice

v = map(lambda x: 2*x, [0,1,2,3,4,5,6,7,8,9])
a, b = tee(v)
print(list( zip(a, islice(b, 3, None)) ))
# [(0, 6), (2, 8), (4, 10), (6, 12), (8, 14), (10, 16), (12, 18)]

v = map(lambda x: 2*x, [0,1,2,3,4,5,6,7,8,9])
print(list( zip(v, islice(v, 3, None)) ))
# [(0, 8), (10, 12), (14, 16)]

more_itertools.zip_offset

Alternatives from a module are welcome as well

Meet module more_itertools and its function zip_offset:

from more_itertools import zip_offset

print(list( zip_offset('01234', 'abcdefghijkl', offsets=(0, 3)) ))
# [('0', 'd'), ('1', 'e'), ('2', 'f'), ('3', 'g'), ('4', 'h')]

You can look at its source code:

from itertools import islice
from itertools import chain, repeat # used when negative offsets
from itertools import zip_longest   # used when longest=True

def zip_offset(*iterables, offsets, longest=False, fillvalue=None):
    if len(iterables) != len(offsets):
        raise ValueError("Number of iterables and offsets didn't match")
    staggered = []
    for it, n in zip(iterables, offsets):
        if n < 0:
            staggered.append(chain(repeat(fillvalue, -n), it))
        elif n > 0:
            staggered.append(islice(it, n, None))
        else:
            staggered.append(it)
    if longest:
        return zip_longest(*staggered, fillvalue=fillvalue)
    return zip(*staggered)

In the special case when the offset is 1, and assuming you're using python>=3.10, there is also pairwise in itertools:

from itertools import pairwise

print(list( pairwise('abcdef') ))
# [('a', 'b'), ('b', 'c'), ('c', 'd'), ('d', 'e'), ('e', 'f')]
\$\endgroup\$
2
\$\begingroup\$

I want to know which approach is closer to pointer use in C in terms of efficiency

Pointers are very much not a thing in Python; and attempts to get toward them often end up in non-Pythonic code. @FMc's answer is a "correct and Pythonic" method, but is very much not like pointers. zip and islice are stateful iterators that carry context.

If I had to show code that would be the closest equivalent to pointers, it would be plain indexing:

v = range(10)

start = 3
for i in range(len(v) - start):
    x = v[i]
    y = v[i + start]
    print(x, y)

But again, unless you have a really good reason (and have eliminated all other options than micro-optimizing), don't do this.

\$\endgroup\$
4
  • 1
    \$\begingroup\$ If the rationale for an approach like this is efficiency/speed, two notes: (1) measure to confirm that it's actually faster (my assumption would be yes, but I've been surprised before on topics like this); and (2) remember that efficiency concerns are often relevant when data volumes are large, and under those conditions len(v) is sometimes (maybe often) unknown. Neither of those notes should be viewed as a criticism of the answer itself, which is valuable (+1). \$\endgroup\$
    – FMc
    Jan 6 at 19:53
  • \$\begingroup\$ Thanks a lot. Indexing was in fact my very first approach. But as the goal was learning the tools of a new language, this seemed too much like camouflaged C for me. I was curious to learn both elegant and eficient ways to tackle a simple problem. In my example, we are extracting data, but if we wanted to alter the list, for example, I think we'll find solutions similar to those that increase eficiency for this problem. \$\endgroup\$
    – xcaliph
    Jan 7 at 0:42
  • \$\begingroup\$ "don't do this" Why? This code is much simpler and easier to understand than the higher-voted zip() based answers. \$\endgroup\$ Jan 13 at 18:33
  • \$\begingroup\$ @NoahYetter I would offer that it's easier to understand for people who do not have a background in Python, particularly if their only experience is with C. However, it's straightforwardly non-Pythonic, and if the zip solution is difficult to understand for someone, that exposes a need for the developer in question to invest some learning into how Pythonic iteration works. \$\endgroup\$
    – Reinderien
    Jan 13 at 18:37
1
\$\begingroup\$

Without offset, i.e., with step = 0, I'd expect the same results as for normal zip. But you produce nothing, because v[:-step] is empty then.

For negative offsets you do produce outputs, but I'd say rather not what one might want. For example for v = [0,1,2,3,4,5,6,7,8,9] and step = -3 you produce (0, 7), (1, 8), etc. Might be desirable to produce (3, 0), (4, 1), etc.

Your print("\nx={}, y={}".format(x, y)) can be shortened: print(f"{x=}, {y=}") (same output, except I removed the leading newline which just caused extra empty lines which seemed pointless).

\$\endgroup\$
1
  • \$\begingroup\$ I find the f-string shortening suggestion from you and @Dronir especially fit for auto debugging. I will use it from now on. At first I was confused cause my python enviroment wasn't fully updated. Very useful suggestion. \$\endgroup\$
    – xcaliph
    Jan 8 at 23:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.