7
\$\begingroup\$

I have not yet seen an implementation of the observer pattern in Python which satisfies the following criteria:

  1. A thing which is observed should not keep its observers alive if all other references to those observers disappear.
  2. Adding and removing observers should be pythonic. For example, if I have an object foo with a bound method .bar, I should be able to add an observer by calling a method on the method, like this: foo.bar.addObserver(observer).
  3. I should be able to make any method observable without subclassing. For example, I should be able to make a method observable just by decorating it.
  4. This must work for types which are unhashable, and must be able to be used on an arbitrary number of methods per class.
  5. The implementation should be comprehensible to other developers.

Here is my attempt (now on github), which allows bound methods to observe other bound methods (see the test example below of example usage):

import weakref
import functools

class ObservableMethod(object):
    """
    A proxy for a bound method which can be observed.

    I behave like a bound method, but other bound methods can subscribe to be
    called whenever I am called.
    """

    def __init__(self, obj, func):
        self.func = func
        functools.update_wrapper(self, func)
        self.objectWeakRef = weakref.ref(obj)
        self.callbacks = {}  #observing object ID -> weak ref, methodNames

    def addObserver(self, boundMethod):
        """
        Register a bound method to observe this ObservableMethod.

        The observing method will be called whenever this ObservableMethod is
        called, and with the same arguments and keyword arguments. If a
        boundMethod has already been registered to as a callback, trying to add
        it again does nothing. In other words, there is no way to sign up an
        observer to be called back multiple times.
        """
        obj = boundMethod.__self__
        ID = id(obj)
        if ID in self.callbacks:
            s = self.callbacks[ID][1]
        else:
            wr = weakref.ref(obj, Cleanup(ID, self.callbacks))
            s = set()
            self.callbacks[ID] = (wr, s)
        s.add(boundMethod.__name__)

    def discardObserver(self, boundMethod):
        """
        Un-register a bound method.
        """
        obj = boundMethod.__self__
        if id(obj) in self.callbacks:
            self.callbacks[id(obj)][1].discard(boundMethod.__name__)

    def __call__(self, *arg, **kw):
        """
        Invoke the method which I proxy, and all of it's callbacks.

        The callbacks are called with the same *args and **kw as the main
        method.
        """
        result = self.func(self.objectWeakRef(), *arg, **kw)
        for ID in self.callbacks:
            wr, methodNames = self.callbacks[ID]
            obj = wr()
            for methodName in methodNames:
                getattr(obj, methodName)(*arg, **kw)
        return result

    @property
    def __self__(self):
        """
        Get a strong reference to the object owning this ObservableMethod

        This is needed so that ObservableMethod instances can observe other
        ObservableMethod instances.
        """
        return self.objectWeakRef()


class ObservableMethodDescriptor(object):

    def __init__(self, func):
        """
        To each instance of the class using this descriptor, I associate an
        ObservableMethod.
        """
        self.instances = {}  # Instance id -> (weak ref, Observablemethod)
        self._func = func

    def __get__(self, inst, cls):
        if inst is None:
            return self
        ID = id(inst)
        if ID in self.instances:
            wr, om = self.instances[ID]
            if not wr():
                msg = "Object id %d should have been cleaned up"%(ID,)
                raise RuntimeError(msg)
        else:
            wr = weakref.ref(inst, Cleanup(ID, self.instances))
            om = ObservableMethod(inst, self._func)
            self.instances[ID] = (wr, om)
        return om

    def __set__(self, inst, val):
        raise RuntimeError("Assigning to ObservableMethod not supported")


def event(func):
    return ObservableMethodDescriptor(func)


class Cleanup(object):
    """
    I manage remove elements from a dict whenever I'm called.

    Use me as a weakref.ref callback to remove an object's id from a dict
    when that object is garbage collected.
    """
    def __init__(self, key, d):
        self.key = key
        self.d = d

    def __call__(self, wr):
        del self.d[self.key]

Here is a test routine, which also serves to illustrate use of the code:

def test():

    buf = []

    class Foo(object):

        def __init__(self, name):
            self.name = name

        @event
        def bar(self):
            buf.append("%sbar"%(self.name,))

        def baz(self):
            buf.append("%sbaz"%(self.name,))


    a = Foo('a')
    assert len(Foo.bar.instances) == 0

    # Calling an observed method adds the calling instance to the descriptor's
    # instances dict.
    a.bar()
    assert buf == ['abar']
    buf = []
    assert len(Foo.bar.instances) == 1
    assert Foo.bar.instances.keys() == [id(a)]
    assert len(a.bar.callbacks) == 0

    b = Foo('b')
    assert len(Foo.bar.instances) == 1
    b.bar()
    assert buf == ['bbar']
    buf = []
    assert len(Foo.bar.instances) == 2

    # Methods added as observers are called when the observed method runs
    a.bar.addObserver(b.baz)
    assert len(a.bar.callbacks) == 1
    assert id(b) in a.bar.callbacks
    a.bar()
    assert buf == ['abar','bbaz']
    buf = []

    # Observable methods can sign up as observers
    mn = a.bar.callbacks[id(b)][1]
    a.bar.addObserver(b.bar)
    assert len(a.bar.callbacks) == 1
    assert len(mn) == 2
    assert 'bar' in mn
    assert 'baz' in mn
    a.bar()
    buf.sort()
    assert buf == ['abar','bbar','bbaz']
    buf = []

    # When an object is destroyed it is unregistered from any methods it was
    # observing, and is removed from the descriptor's instances dict.
    del b
    assert len(a.bar.callbacks) == 0
    a.bar()
    assert buf == ['abar']
    buf = []
    assert len(Foo.bar.instances) == 1

    del a
    assert len(Foo.bar.instances) == 0
\$\endgroup\$
2
\$\begingroup\$

I like your code: its quite interesting. While reviewing, I was hoping to see if there was a better way if keeping track of the instances of each ObservableMethod or if we could combine the ObservableMethod and ObservableMethodDescriptor classes. However, I could not think of a better way to store the instances of each ObservableMethod.

With that being said, I have a few recommendations, mostly centered around naming:

  1. Rename your event function and Cleanup class.

    Currently event is quite generic. Plus, it doesn't follow the 'function-names-start-with-verbs' convention. I would recommend renaming it to make_observable. This conveys much better what it (and its decorator) actually does.

    As for renaming Cleanup: because of its verb-based name, it feels like a function. Classes are things, thus it makes sense to have a noun-based name; maybe CleanupHandler.

  2. Improve your variable names.

    Currently you have several 1-letter or 2-letter variable names. Make those names more descriptive:

    def __get__(self, inst, cls):
        . . . 
        if ID in self.instances:
            # World record, organic matter?
            wr, om = self.instances[ID]
    

    Yes, in context we can deduce the meaning of the variable names. However, it pays to be more explicit rather than trusting that everyone understands what is going on:

    def __get__(self, inst, cls):
        . . . 
        if ID in self.instances:
            # Much better.
            weak_ref, observable_method = self.instances[ID]
    

    In the example above is another point I want to make: ID is not constant (as convention says its name suggests). I would recommend renaming it to obj_id or something of the like. You could just use id but that may be a little confusing (and possibly dangerous) since id is a basic Python function.

    My final point about variable names deals with multiple word names. You used 'camelCase' in your code. Pythonic convention say that underscores_in_names is preferred.

  3. Spacing

    Insert blank lines to help group logical sections of code. Looking at your __get__ code, inserting blank lines helps the visual flow of the method:

    def __get__(self, inst, cls):
        if inst is None:
            return self
    
        ID = id(inst)
        if ID in self.instances:
            wr, om = self.instances[ID]
    
            if not wr():
                msg = "Object id %d should have been cleaned up"%(ID,)
                raise RuntimeError(msg)
        else:
            wr = weakref.ref(inst, Cleanup(ID, self.instances))
            om = ObservableMethod(inst, self._func)
            self.instances[ID] = (wr, om) 
    
        return om
    
  4. format() vs. %-Notation

    As this link says, %-notation isn't becoming obsolete. However, it says that using format() is the preferred method, especially if you are concerned about future compatibility.

    # Your way
    >>> print 'Hello %s!'%('Darin',)
    
    # New way
    >>> print 'Hello {}!'.format('Darin')
    

    This also saves you from having to create temporary tuples just to print information.

Outside of these comments, the code looks neat and Pythonic.

| improve this answer | |
\$\endgroup\$
  • \$\begingroup\$ Excellent feedback. Is the procedure to implement your comments by editing my original post, or should I make a new submission? \$\endgroup\$ – DanielSank May 20 '14 at 18:21
  • \$\begingroup\$ I would not overwrite your submission mainly because others may see errors that I may have missed in your original code. You could just append on an edit sections with code that you think could be improved more. \$\endgroup\$ – BeetDemGuise May 20 '14 at 18:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.