In a compiler project for an LL(1/2) grammar, I have had a need for a generator iterator with a push back function. I have been surprised not to find any perfectly applicable solution (clean, simple, obvious, standard, etc.). On the other hand I was surprised of how easy it was to create exactly what I wanted after I decided to invest a little time.
Here is what I came up with:
class Back_pushable_iterator: """Class whose constructor takes an iterator as its only parameter, and returns an iterator that behaves in the same way, with added push back functionality. The idea is to be able to push back elements that need to be retrieved once more with the iterator semantics. This is particularly useful to implement LL(k) parsers that need k tokens of lookahead. Lookahead or push back is really a matter of perspective. The pushing back strategy allows a clean parser implementation based on recursive parser functions. The invoker of this class takes care of storing the elements that should be pushed back. A consequence of this is that any elements can be "pushed back", even elements that have never been retrieved from the iterator. The elements that are pushed back are then retrieved through the iterator interface in a LIFO-manner (as should logically be expected). This class works for any iterator but is especially meaningful for a generator iterator, which offers no obvious push back ability. In the LL(k) case mentioned above, the tokenizer can be implemented by a standard generator function (clean and simple), that is completed by this class for the needs of the actual parser. """ def __init__(self, iterator): self.iterator = iterator self.pushed_back =  def __iter__(self): return self def __next__(self): if self.pushed_back: return self.pushed_back.pop() else: return next(self.iterator) def push_back(self, element): self.pushed_back.append(element) def main(): it = Back_pushable_iterator(x for x in range(10)) x = next(it) # 0 print(x) it.push_back(x) x = next(it) # 0 print(x) x = next(it) # 1 print(x) x = next(it) # 2 y = next(it) # 3 print(x) print(y) it.push_back(y) it.push_back(x) x = next(it) # 2 y = next(it) # 3 print(x) print(y) for x in it: print(x) # 4-9 it.push_back(x) y = next(it) # 9 print(x) if __name__ == "__main__": main()
Any feedback is appreciated. A better name? ;-)
Note: I have already posted this code as a very late answer to https://stackoverflow.com/questions/2425270/how-to-look-ahead-one-element-in-a-python-generator, but got no feeback.