1
\$\begingroup\$

itertools.cycle is cool, but it has an internal "storage" that uses up memory. And I wanted to practice "tampering" with built-in methods of classes, which is a new field for me. I tried to implement a class which mostly behaves like a list, but when we try to iterate over it (with for loop, for example), it would jump back to the beginning when it reaches the end, therefore cycling indefinitely. Here's what I've come up with:

class InfiniteListEmptyError(Exception):
    pass


class InfiniteList(list):
    """
    Pretty much a regular list, but when iterated over, it does not stop in the end. 
    Instead, it iterates indefinitely.
    """

    def __init__(self, arg):
        super(InfiniteList, self).__init__(arg)

    def __getattr__(self, name):
        if name == "it":
            # get iterator
            result = object.__getattribute__(self, name)
        else:
            try:
                result = super(InfiniteList, self).__getattribute__(name)
            except AttributeError:
                try:
                    result = self.it.__getattribute__(name)
                except AttributeError:
                    # Initialize iterator cuz it's not initialized
                    self.__iter__()
                    result = self.it.__getattribute__(name)
        return result

    def __iter__(self):
        it = super(InfiniteList, self).__iter__()
        self.it = it
        return self

    def __next__(self):
        try:
            result = next(self.it)
        except StopIteration:
            self.__iter__()
            try:
                result = next(self.it)
            except StopIteration:
                raise InfiniteListEmptyError("Could not iterate. List is empty!")

        return result

# TESTS

a = InfiniteList(tuple()) #empty list

print(a)
print(a.__length_hint__()) #testing iterator attributes
print(a.__eq__([1,3,5])) # false

# should raise exception, since the list is empty
try:
    for i in a:
        print(i)
except InfiniteListEmptyError:
    print("List is empty!")

a.append(1)
a.extend([3,5])
print(a)
print(a.__eq__([1,3,5])) #true

# infinite loop
for i in a:
    print(i)

It works fine, it does not store an auxiliary list, so it does not consume twice as much memory, like itertools.cycle. But I'm not sure about the possible caveats when it comes to handling attributes. And yeah, I had to implement iterator within the InfiniteList class itself, so I had to redirect the calls to iterator's methods (like __length_hint__) to the saved iterator self.it.

\$\endgroup\$
  • 1
    \$\begingroup\$ You know that by doing a = list(range(50)); b = InfiniteList(a) you are effectively copying your initial list, right? So what is the exact advantage over itertools.cycle? \$\endgroup\$ – 409_Conflict Oct 13 '16 at 18:24
1
\$\begingroup\$

Simplifying the class

def __init__(self, arg):
    super(InfiniteList, self).__init__(arg)

Is unnecessary. Not only it doesn't add any value over the default constructor of lists (which accept any iterable to initialize a list from it) but it prevent from building an empty list by using a constructor without argument, such as empty_list = list().


def __getattr__(self, name):
    if name == "it":
        # get iterator
        result = object.__getattribute__(self, name)
    else:
        try:
            result = super(InfiniteList, self).__getattribute__(name)
        except AttributeError:
            try:
                result = self.it.__getattribute__(name)
            except AttributeError:
                # Initialize iterator cuz it's not initialized
                self.__iter__()
                result = self.it.__getattribute__(name)
    return result

Contrary to __getattribute__, __getattr__ is only called if the lookup already failed. So you are assured that, if self.it has already been initialized, __getattr__ won't be called to access it. You also don't really need to store the iterator directly. You know that, given the use cases of your class, __getattr__ will most likely be called when trying to access an iterator method. You can thus build on that to create an iterator on demand and try to access its methods:

def __getattr__(self, name):
    """Redirect missing attributes and methods to
    those of the underlying iterator.
    """
    iterator = super(InfiniteList, self).__iter__()
    return getattr(iterator, name)

def __iter__(self):
    it = super(InfiniteList, self).__iter__()
    self.it = it
    return self

def __next__(self):
    try:
        result = next(self.it)
    except StopIteration:
        self.__iter__()
        try:
            result = next(self.it)
        except StopIteration:
            raise InfiniteListEmptyError("Could not iterate. List is empty!")

    return result

Is overly complicated as implementing __iter__ that returns self and having a __next__ method can be simplified to turning __iter__ into a generator most of the time:

def __iter__(self):
    if not self:
        raise InfiniteListEmptyError("Could not iterate. List is empty!")

    while True:
        iterator = super(InfiniteList, self).__iter__()
        yield from iterator

So the class can be defined as:

class InfiniteList(list):
    def __getattr__(self, name):
        """Redirect missing attributes and methods to
        those of the underlying iterator.
        """
        iterator = super(InfiniteList, self).__iter__()
        return getattr(iterator, name)

    def __iter__(self):
        if not self:
            raise InfiniteListEmptyError("Could not iterate. List is empty!")

        while True:
            iterator = super(InfiniteList, self).__iter__()
            yield from iterator

Differences with itertools.cycle

Building an InfiniteList out of an iterable will inevitably create memory to store the list and, as such, is no different from the memory used by itertools.cycle. Even building an InfiniteList out of an existing list will duplicate memory.

The only "advantage" being that you could build the list from scratch using append or extend and not use as much memory than itertools.cycle. However, such approaches are generally better handled using a list-comprehension or a generator expression. And feeding the generator expression to either itertools.cycle or InfiniteList will give the same memory footprint.

Except itertools.cycle:

  1. is optimized in C;
  2. doesn't consume the iterable upfront and, as such, can handle infinite generators (meaning itertools.cycle(itertools.count()) is fine albeit unneccessary whereas InfiniteList(itertools.count()) will eat up all your memory).

In short, unless you have a very specific use-case, most of the time your InfiniteList will have the exact same memory footprint than itertool.cycle with worse performances.

\$\endgroup\$
4
\$\begingroup\$

This seems kind of complicated compared to something like this:

def cycle(seq):
    "Generator that yields the elements of a non-empty sequence cyclically."
    i = 0
    while True:
        yield seq[i]
        i = (i + 1) % len(seq)

Or, as Joe Wallis points out in comments:

def cycle(seq):
    "Generator that yields the elements of a non-empty sequence cyclically."
    while True:
        yield from seq
\$\endgroup\$
  • 1
    \$\begingroup\$ You can remove the need for the i = if you use for item in seq: yield item like itertools.cycle does. But nice answer \$\endgroup\$ – Peilonrayz Oct 13 '16 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.