# Decorate instance method with (arbitrary) meta data in python

Requirements

What I need: attach meta information to methods, such that

1. it is 'easy' to retrieve all such methods for a given class instance
2. methods can still be called 'in a normal way', e.g. obj.method()
3. meta-data is accessible from the decorated method, e.g. obj.method.data
4. IDEs (PyCharm in particular) do not produce any warnings or errors (and, if possible, IDE support, e.g. auto-completion, should be able to handle the annotation)

Additionally, I would like the code to be readable/intuitive (not necessarily the super classes, though), generally robust and 'bug-free'. I accept the limitation that my decorators need to be the most 'outer' decorator for the automatic collection to take place.

From my point of view, overcoming function/method transformation while still exposing an arbitrary object type (not a function type -- thinking of this, maybe subclassing a FunctionType might be another idea?) is the hardest challenge.

What do you think of the following three solutions? Is there something I did miss?

Code

class MethodDecoratorWithIfInCall(object):
def __init__(self):
self._func = None

def __call__(self, *args, **kwargs):
if self._func is None:
assert len(args) == 1 and len(kwargs) == 0
self._func = args[0]
return self
else:
return self._func(*args, **kwargs)

def __get__(self, *args, **kwargs):
# update func reference to method object
self._func = self._func.__get__(*args, **kwargs)
return self

class MacroWithIfInCall(MethodDecoratorWithIfInCall):
def __init__(self, name):
super(MacroWithIfInCall, self).__init__()
self.name = name

class MethodDecoratorWithExplicitDecorate(object):
def __init__(self, *args, **kwargs):
# wildcard parameters to satisfy PyCharm
self._func = None

def __call__(self, *args, **kwargs):
return self._func(*args, **kwargs)

def __get__(self, *args, **kwargs):
# update func reference to method object
self._func = self._func.__get__(*args, **kwargs)
return self

def _decorate(self):
def _set_func(func):
self._func = func
return self
return _set_func

@classmethod
def decorate(cls, *args, **kwargs):
obj = cls(*args, **kwargs)
return obj._decorate()

class MacroWithExplicitDecorate(MethodDecoratorWithExplicitDecorate):
def __init__(self, name):
super(MacroWithExplicitDecorate, self).__init__()
self.name = name

class MacroWithoutSuperclass(object):
def __init__(self, func, name):
self.func = func
self.name = name

def __get__(self, *args, **kwargs):
# update func reference to method object
self.func = self.func.__get__(*args, **kwargs)
return self

def __call__(self, *args, **kwargs):
return self.func(*args, **kwargs)

@staticmethod
def decorate(name):
return lambda func: MacroWithoutSuperclass(func, name)

class Shell:
def __init__(self):
macros = [macro for macro in map(self.__getattribute__, dir(self))
if isinstance(macro, (MacroWithIfInCall, MacroWithExplicitDecorate, MacroWithoutSuperclass))]

for macro in macros:
print(macro.name, macro())

@MacroWithIfInCall(name="macro-with-if-in-call")
return "called"

@MacroWithExplicitDecorate.decorate(name="macro-with-explicit-decorate")
return "called"

@MacroWithoutSuperclass.decorate(name="macro-without-superclass")
return "called"

if __name__ == '__main__':
shell = Shell()


Output

macro-with-if-in-call called
macro-with-explicit-decorate called
macro-without-superclass called


From your three approaches, I think MacroWithoutSuperclass is probably the cleanest. I wanted to just comment a bit, but it turned out I had too much to say... Thus, here a few remarks followed by a maybe more intuitive solution as an inspiration (it's probably not perfect).

None of your methods supports arbitrary meta data, in case that is a requirement, but only name. In the following I assume that you in fact want to add arbitrary data.

You are assigning meta-data by name, so that e.g. macro.name is accessible. Is that a requirement? Or would macro.meta['name'] be enough? Below I will present both options, but it is not really clear from you question, only implicitly from your code.

All of your three methods seem in general very complicated. For example

if self._func is None:
assert len(args) == 1 and len(kwargs) == 0
self._func = args[0]
return self
else:
return self._func(*args, **kwargs)


is quite complicated and should have some more explanations.

The inheritance

class MacroWithIfInCall(MethodDecoratorWithIfInCall):
def __init__(self, name):
super(MacroWithIfInCall, self).__init__()
self.name = name


is not needed (the same applies to MacroWithExplicitDecorate), instead you can assign the name in MethodDecoratorWithIfInCall's init function – exactly as you did in the MacroWithoutSuperclass. Which is basically why I think a combination of the MacroWithIfInCall (the __call__ function) and the MacroWithoutSuperclass would be the best solution using your approach.

I don't understand why you need wildcard parameters (# wildcard parameters to satisfy PyCharm) for one class, but not for the others, you shouldn't need them in either case.

Anyways, I think you can fare much better with an even simpler decorator, as I will explain below.

1. it is 'easy' to retrieve all such methods for a given class instance

This is sufficiently easy using inspect.getmembers:

import inspect

types = (MacroWithIfInCall, MacroWithExplicitDecorate, MacroWithoutSuperclass)
macros = inspect.getmembers(Shell, lambda m: isinstance(m, types))


Note however, that macros is now a list containing tuples of the form (name, function). Dependending on your use case you might have to use any of the following lines:

names = [macro[0] for macro in macros]  # Get only the names
funcs = [macro[1] for macro in macros]  # Get only the functions
names, funcs = zip(*macros)  # Get names and functions in separate lists

1. methods can still be called 'in a normal way', e.g. obj.method()

This is fine, and will be the case for the classic decorator which follows this form:

import functools

def decorator(func):

@functools.wraps(func)  # Handles the docs and names properly
def wrapper(*args, **kwargs):
# Do something before the call

result = func(*args, **kwargs)

# Do something after the call

return result

return wrapper


In fact, for your problem slightly less complex, as you don't need to perform any tasks on execution but only when declaring the function, so in fact you can return the original function:

def meta(func):
func.meta = 'meta data here'  # Assign constant metadata
return func

1. meta-data is accessible from the decorated method, e.g. obj.method.data

As mentioned above, I am not sure whether you mean that all meta data is contained in data or that data is just one part, and name, age, abc would be others.

To have this, you could now just assign these values by extending the decorator meta(func) I presented above. Remember that in Python, functions are just objects (search for "User-defined functions"):

Function objects also support getting and setting arbitrary attributes, which can be used, for example, to attach metadata to functions. Regular attribute dot-notation is used to get and set such attributes.

This means, that instead of having your complex wrapper classes storing the meta data and the function you can just store the meta data at the function itself.

So the decorator meta needs to have a way to specify the meta data. The common way is building another function around it, which eventually returns the decorator.

def meta(**meta_data):
"""Attaches meta information to a method."""

def _attach_meta(func):
func.meta = meta_data
return func

return _attach_meta


To support the dot-notation (and not the meta-dictionary) the line func.meta = meta_data could to be changed to

for k, v in meta_data.items():
setattr(func, k, v)
func.has_meta = True  # flag to use for inspect.getmembers' predicate


I would use the dictionary version unless you have a very strict set of meta attributes, as then accessing data using function.meta.get(KEY, DEFAULT_VALUE) becomes fairly useful.

1. IDEs (PyCharm in particular) do not produce any warnings or errors (and, if possible, IDE support, e.g. auto-completion, should be able to handle the annotation)

I can't say anything about this, but I think it should work as you expect... I use YCM and am too lazy to install PyCharm now.

Additionally, I would like the code to be readable/intuitive (not necessarily the super classes, though)

This should be sufficiently satisfied with the presented solution, although it is also lacking some (in-code) documentation.

generally robust and 'bug-free'.

There can always be name clashes etc., and my solution does not have the strong typing your solution has. But for bug-free-ness, it is possible to add some tests, making it less buggy.

I accept the limitation that my decorators need to be the most 'outer' decorator for the automatic collection to take place.

This should still be the case, I fear. But I haven't tried to solve this.

Full working example

import inspect

def meta(**meta_data):
"""Attaches meta information to a method."""

def _attach_meta(func):
func.meta = meta_data
return func

return _attach_meta

def has_meta(func):
"""Predicate to check for meta data."""
return hasattr(func, 'meta')

class SomeAnnotated:
@meta(a=1, b=2)
def __call__(self):
"""The call method."""
return 'Hello'

@meta(x=3, y=4)
def annotated(self, abc):
"""The annotated method."""
return abc

def not_annotated(self):
"""The not annotated method."""
return 'Not annotated'

print('Meta information is available for:')
for name, func in inspect.getmembers(SomeAnnotated, has_meta):
print(f'{func.__qualname__}: {func.meta} -- {func.__doc__}')


Expected output:

Meta information is available for:
SomeAnnotated.__call__: {'a': 1, 'b': 2} -- The call method.
SomeAnnotated.annotated: {'x': 3, 'y': 4} -- The annotated method.


Edit: As per comment here is a solution more closely related to your idea around a non-data descriptor (i.e. using __get__):

import inspect
import functools

class MetaAnnotated(functools.partial):
pass

class MetaAnnotator:
def __init__(self, func, **meta_data):
self.func = func
self.meta = meta_data

def __get__(self, obj, type=None):
annotated_func = functools.wraps(self.func)(MetaAnnotated(self.func, obj))
annotated_func.meta = self.meta
return annotated_func

def meta(**meta_data):
"""Attaches meta information to a method."""

def _attach_meta(func):
return MetaAnnotator(func, **meta_data)

return _attach_meta

def has_meta(func):
"""Predicate to check for meta data."""
return isinstance(func, MetaAnnotated)


The above solution uses functools for two purposes: first, to retain the documentation etc (wraps). Second, and more importantly, to create a partial function which is called with our instance as self. In a step-by-step solution we would do this as follows (each block represents a change to understand the code, not a sequence of commands):

# annotated_func behaves the same as func, but will
# always use "obj" as its first argument.
annotated_func = functools.partial(func, obj)

# We can inherit from functools.partial to have a tagging class:
class MetaAnnotated(functools.partial): pass
annotated_func = MetaAnnotated(func, obj)

# Keep the doc string from func:
class MetaAnnotated(functools.partial): pass
annotated_func = functools.wraps(func)(MetaAnnotated(func, obj))


This way, no __call__ is needed, neither is a staticmethod: as soon as the function we decorated using MetaAnnotated is searched for on the original object, the properly wrapped function is returned (instead of the MetaAnnotator!).

This behaves almost the same as the solution without typing, but it does not have the func.__qualname__ needed in my test prints, so we need to change this. I also added a "proof" that it's properly callable:

print('Meta information is available for:')
for name, func in inspect.getmembers(SomeAnnotated, has_meta):
print(f'{name}: {func.meta} -- {func.__doc__}')

instance = SomeAnnotated()
print(instance())
print(instance.annotated(123))


I think this is a nice synthesis between your original ideas about using the descriptors and my ideas of having a very plain and straight forward decorator. All the "magic" happens in one place (the MetaAnnotator), the MetaAnnotated class transparently tags functions, and there is a very neat API: a user just needs to call @meta and can forget about everything else.

• Hey, welcome to Code Review! This is a very nice first answer! Jul 10, 2018 at 12:07
• Many thanks for your elaborate answer (+1). The only thing which is really missing is the type safety and hence, a clean way to get all the desired methods. But, as inheriting function types is not yet possible in python, one does always require the __get__ state-machine for function-to-method conversion when using a custom object. This is where the mess in my implementation comes from. Regarding PyCharm, cls(*args, **kwargs) produces a warning if __init__ of the base class does not accept wildcard parameters. I will accept your answer if no better oop-approach is presented! Jul 13, 2018 at 7:16
• I updated my answer to use a descriptor with __get__ in an additional example to retain type safety, while avoiding the use of a staticmethod and generally keeping the functions small and descriptive. Jul 13, 2018 at 13:37