# Infinitely iterable list class in Python

itertools.cycle is cool, but it has an internal "storage" that uses up memory. And I wanted to practice "tampering" with built-in methods of classes, which is a new field for me. I tried to implement a class which mostly behaves like a list, but when we try to iterate over it (with for loop, for example), it would jump back to the beginning when it reaches the end, therefore cycling indefinitely. Here's what I've come up with:

class InfiniteListEmptyError(Exception):
pass

class InfiniteList(list):
"""
Pretty much a regular list, but when iterated over, it does not stop in the end.
"""

def __init__(self, arg):
super(InfiniteList, self).__init__(arg)

def __getattr__(self, name):
if name == "it":
# get iterator
result = object.__getattribute__(self, name)
else:
try:
result = super(InfiniteList, self).__getattribute__(name)
except AttributeError:
try:
result = self.it.__getattribute__(name)
except AttributeError:
# Initialize iterator cuz it's not initialized
self.__iter__()
result = self.it.__getattribute__(name)
return result

def __iter__(self):
it = super(InfiniteList, self).__iter__()
self.it = it
return self

def __next__(self):
try:
result = next(self.it)
except StopIteration:
self.__iter__()
try:
result = next(self.it)
except StopIteration:
raise InfiniteListEmptyError("Could not iterate. List is empty!")

return result

# TESTS

a = InfiniteList(tuple()) #empty list

print(a)
print(a.__length_hint__()) #testing iterator attributes
print(a.__eq__([1,3,5])) # false

# should raise exception, since the list is empty
try:
for i in a:
print(i)
except InfiniteListEmptyError:
print("List is empty!")

a.append(1)
a.extend([3,5])
print(a)
print(a.__eq__([1,3,5])) #true

# infinite loop
for i in a:
print(i)


It works fine, it does not store an auxiliary list, so it does not consume twice as much memory, like itertools.cycle. But I'm not sure about the possible caveats when it comes to handling attributes. And yeah, I had to implement iterator within the InfiniteList class itself, so I had to redirect the calls to iterator's methods (like __length_hint__) to the saved iterator self.it.

• You know that by doing a = list(range(50)); b = InfiniteList(a) you are effectively copying your initial list, right? So what is the exact advantage over itertools.cycle? – 301_Moved_Permanently Oct 13 '16 at 18:24

# Simplifying the class

def __init__(self, arg):
super(InfiniteList, self).__init__(arg)


Is unnecessary. Not only it doesn't add any value over the default constructor of lists (which accept any iterable to initialize a list from it) but it prevent from building an empty list by using a constructor without argument, such as empty_list = list().

def __getattr__(self, name):
if name == "it":
# get iterator
result = object.__getattribute__(self, name)
else:
try:
result = super(InfiniteList, self).__getattribute__(name)
except AttributeError:
try:
result = self.it.__getattribute__(name)
except AttributeError:
# Initialize iterator cuz it's not initialized
self.__iter__()
result = self.it.__getattribute__(name)
return result


Contrary to __getattribute__, __getattr__ is only called if the lookup already failed. So you are assured that, if self.it has already been initialized, __getattr__ won't be called to access it. You also don't really need to store the iterator directly. You know that, given the use cases of your class, __getattr__ will most likely be called when trying to access an iterator method. You can thus build on that to create an iterator on demand and try to access its methods:

def __getattr__(self, name):
"""Redirect missing attributes and methods to
those of the underlying iterator.
"""
iterator = super(InfiniteList, self).__iter__()
return getattr(iterator, name)


def __iter__(self):
it = super(InfiniteList, self).__iter__()
self.it = it
return self

def __next__(self):
try:
result = next(self.it)
except StopIteration:
self.__iter__()
try:
result = next(self.it)
except StopIteration:
raise InfiniteListEmptyError("Could not iterate. List is empty!")

return result


Is overly complicated as implementing __iter__ that returns self and having a __next__ method can be simplified to turning __iter__ into a generator most of the time:

def __iter__(self):
if not self:
raise InfiniteListEmptyError("Could not iterate. List is empty!")

while True:
iterator = super(InfiniteList, self).__iter__()
yield from iterator


So the class can be defined as:

class InfiniteList(list):
def __getattr__(self, name):
"""Redirect missing attributes and methods to
those of the underlying iterator.
"""
iterator = super(InfiniteList, self).__iter__()
return getattr(iterator, name)

def __iter__(self):
if not self:
raise InfiniteListEmptyError("Could not iterate. List is empty!")

while True:
iterator = super(InfiniteList, self).__iter__()
yield from iterator


# Differences with itertools.cycle

Building an InfiniteList out of an iterable will inevitably create memory to store the list and, as such, is no different from the memory used by itertools.cycle. Even building an InfiniteList out of an existing list will duplicate memory.

The only "advantage" being that you could build the list from scratch using append or extend and not use as much memory than itertools.cycle. However, such approaches are generally better handled using a list-comprehension or a generator expression. And feeding the generator expression to either itertools.cycle or InfiniteList will give the same memory footprint.

Except itertools.cycle:

1. is optimized in C;
2. doesn't consume the iterable upfront and, as such, can handle infinite generators (meaning itertools.cycle(itertools.count()) is fine albeit unneccessary whereas InfiniteList(itertools.count()) will eat up all your memory).

In short, unless you have a very specific use-case, most of the time your InfiniteList will have the exact same memory footprint than itertool.cycle with worse performances.

This seems kind of complicated compared to something like this:

def cycle(seq):
"Generator that yields the elements of a non-empty sequence cyclically."
i = 0
while True:
yield seq[i]
i = (i + 1) % len(seq)


Or, as Joe Wallis points out in comments:

def cycle(seq):
"Generator that yields the elements of a non-empty sequence cyclically."
while True:
yield from seq

• You can remove the need for the i = if you use for item in seq: yield item like itertools.cycle does. But nice answer – Peilonrayz Oct 13 '16 at 17:03