# Memoizing decorator with retries - now with backoff added! (Part 3)

A continuation of Memoizing decorator with retries, part 2, and related to https://codereview.stackexchange.com/a/133493/47529. I liked my decorator before, but especially in my original use case of a spotty network connection it makes sens to allow for some delay between attempts. The specifics of this delay should be pretty configurable - I don't want to restrict myself or anyone else to my decision of a good backoff function, so I included the ability to pass a generator that handles everything. Also adds a bunch of docstrings (finally) to make it more understandable. I like the interface right now, and I think with the docstrings it makes more sense, but as always I'd like some feedback.

As a note - I got feedback on the second part after I wrote this question, and in particular w.r.t the dual responsibilities that this has (for both caching and retrying). If you want to comment on that feel free, but I've already considered that and will do so in my next iteration.

import functools
import random
import time
import itertools as it
from collections import namedtuple

def no_backoff():
"""Dummy generator that never delays."""

for delay in it.repeat(0):
yield delay

def doubling_backoff(start=1):
"""Double backoff time, always.

Parameters
==========
start: int, optional
The first delay to use. Defaults to 1.

Yields
======
int
The amount of time to delay.
"""

yield 0

while True:
for delay in it.count(start):
restart = yield 2 * delay
if restart:
break

def exponential_backoff(interval):
"""Exponential backoff algorithm over some interval.

Backs off such that for n successive failed attempts the delay
is calculated as interval * random[0, 2**n-1].

Parameters
==========
interval: int
How large of an interval to use.

Yields
======
int
The amount of time to delay.

Notes
=====
https://en.wikipedia.org/wiki/Exponential_backoff
"""

yield 0
while True:
for num_failed in it.count():
delay = interval * random.randint(0, 2**num_failed - 1)
restart = yield delay
if restart:
break

MemoizedData = namedtuple('MemoizedData', 'is_exception value')

class Memoizer:
"""Memoizing class with multiple extra features.

Supports the ability to retry several times by suppressing
certain exceptions, ability to capture and rethrow previously
unsuppressed but detected exceptions, and support for algorithmic
backoff algorithms.

Parameters
==========
retry_times: int, optional
How many times to retry the function before giving up. Defaults
to 0.
suppressed_exceptions: tuple, optional
Which exceptions to suppress and retry on. Defaults to an empty
tuple (no exceptions are suppressed).
capture_exceptions: bool, optional
Whether or not a thrown exception should be remembered and
rethrown if the same arguments are used once again. Does not
apply to suppressed exceptions. Defaults to False.
backoff_gen: generator, optional
Generator that is used to calculate the time to wait between
attempts. Defaults to a generator no_backoff which infinitely
yields 0. A generator supplied here is expected to first yield
a meaningless value, and accept a boolean value, i.e.

>>> backoff_gen.send(True)

If True is sent then one of two things has happened:

1. The function has been called for the first time - your
algorithm may need to be appropriately initialized.
2. The function has been called successfully - your generator
may need to be reset as appropriate.

If False is sent then the algorithm failed, and the backoff
should be adjusted as necessary.

Returns
=======
any
The value returned by the wrapped function

Raises
======
any
The exception raised by the wrapped function (may be cached).
This exception may have been internally suppressed up to
retry_times - 1 for a given function call.

Notes
=====
The wrapped function has an additional keyword argument added
to it named __replace which can be used to ignore any value
or exception that was previously cached.
"""

def __init__(self, retry_times=0, suppressed_exceptions=tuple(),
capture_exceptions=False, backoff_gen=no_backoff()):
self.retry_times = retry_times
self.suppressed_exceptions = suppressed_exceptions
self.capture_exceptions = capture_exceptions
self.backoff_generator = backoff_gen
self._generator_started = False

def _init_backoff_generator(self):
"""Initializes the backoff generator.

If the generator has not been started, gets the first value
from it and discards it. Then informs the generator that the
function has been started.

Notes
=====
Expects that the backoff generator will yield some value that
can be thrown away when initialized, and then handles a boolean
value as described previously.
"""

if not self._generator_started:
next(self.backoff_generator)
self._generator_started = True
self.backoff_generator.send(True)

def _handle_function(self, function, args, kwargs, raise_suppressed=False):
"""Tries to run the function and capture any values.

Parameters
==========
function: callable
The function to be called.
args: list
The function arguments.
kwargs: dict
The function keyword arguments.
raise_suppressed: bool, optional
Whether or not suppressed exceptions should raise. Defaults
to False.

Returns
=======
MemoizedData
Some memoized data of the result of the function

Raises
======
Exception
Any unsuppressed and uncaptured exception
"""

try:
return MemoizedData(False, function(*args, **kwargs))
except self.suppressed_exceptions:
if raise_suppressed:
raise
except Exception as e:
if self.capture_exceptions:
return MemoizedData(True, e)
raise

def __call__(self, function):
"""Actually wrap a function."""

d = {}

@functools.wraps(function)
def wrapper(*args, __replace=False, **kwargs):
self._init_backoff_generator()
key = (args, tuple(sorted(kwargs.items())))
if key not in d or __replace:
for _ in range(self.retry_times - 1):
result = self._handle_function(function, args, kwargs)
if result is not None:
d[key] = result
break
delay = self.backoff_generator.send(False)
time.sleep(delay)
else:
d[key] = self._handle_function(
function, args, kwargs, raise_suppressed=True
)

if d[key].is_exception:
raise d[key].value
else:
return d[key].value

return wrapper

• I've answered on pt2 because it looks like things were easier to fix then. You need 2 decorators, not 1: one to handle memoization and one to handle retries. Imagine what will happen when you think that memoizing into file is a good idea to preserve stuff between calls? With some more additions __init__ signature would become incomprehensible. – Daerdemandt Jul 11 '16 at 19:18
• Why not merge the back-off generators into a higher-order function and pass a function that gets the back-off time given the number of times already backed off? – Heman Gandhi Jul 11 '16 at 22:38
• @HemanGandhi I'm interested in hearing more about that if you want to write an answer, but the reason I wrote it as I did was to make it lightweight, easy to use, and simple to implement. – Dannnno Jul 12 '16 at 2:10

Your retries are oblivious to exception history. You could replace them with tuples of delays and things won't really change.

Instead, you can store history of exceptions caught and provide backoff generator with that.

Simulating existing behaviour is trivial - you just take len of that history and know what iteration we're in.

However, you'd be also able to handle situations like "I should retry immediately after EAGAIN and fail immediately after PERMISSION_DENIED".

Also,

def no_backoff():
"""Dummy generator that never delays."""
yield from it.repeat(0)


EDIT: (too long for comment)

why does it matter that they're oblivious to exception history?

Well, this would matter if you had 2 different errors requiring different approaches. At first I've thought about function that takes 2 params - number of current try and last exception - but replacing that with list of exceptions seemed more versatile.

I agree that keeping interface simple is a good thing.

One possible solution is to say that our time series are not supposed to depend on exceptions we encounter, so to react differently for different exceptions we'll just layer several decorators.

Another possible solution is to use send to pass current exception into backoff generator. If programmer wants to do something with that data - it can be stored and processed. This does not forbid simple generators - they would be simply oblivious to data coming from yield.

You use send to control generator's restart (or lack of it). Alternatively, instead of asking a generator to restart (and forcing programmer to implement that behaviour in every iterator), you could simply drop existing generator and use a new one.

Storing provided generator, but iterating over fresh copies of it would be nice, but copying generators is a somewhat tricky in Python.

Instead, you can accept not a generator, but a factory function that makes them. Use-wise, it would be as simple as replacing backoff_gen=doubling_backoff(5) with backoff_gen=lambda:doubling_backoff(5).

That would simplify generators - because one wouldn't need to reimplement restart each time.

In fact, even if you want to keep ysing send to trigger restarts, I'd recommend making generator decorator that handles the restart, leiitng inner generator handle generating delays.

You can use yield to both control restarts and pass exceptions but that does not look pretty to me.

• I'm not sure I understand your point - why does it matter that they're oblivious to exception history? I like the point about retrying working differently with different exceptions/whatever, but I think that is adding extra complexity before I actually need it. I think it would also greatly complicate the interface that other generators that potentially don't want to or shouldn't have to deal with. – Dannnno Jul 11 '16 at 20:52
• @Dannnno Updated my post with response. – Daerdemandt Jul 12 '16 at 13:42

So about the higher-order function for back-off generators (I had commented):

#python 3 only!
def back_off(amt_fn):
yield 0 #if we always must start with 0.
yield from map(amt_fn, it.count(0))

exp_back_off = back_off(lambda x: x ** 2)
no_delay = back_off(lambda x: 0)


This should make it much more readable and simpler to create back off functions.