Function overloading in Python

Question

Motivation

As an exercise, I wanted to try implementing function overloading in Python 3, i.e. to implement a way of defining multiple functions with the same name and then calling the appropriate function based on the given arguments in a function call. I know that this isn't necessary with Python and my understanding is that the Pythonic way to realize this kind of functionality is to define a single function and then inspect the arguments in order to determine the desired course of action, but I wanted to try this anyway and see how it went.

Features

There were a few basic features that I wanted to include:

• Collect functions by name, e.g. def f(x): pass and def f(y): pass should be considered as two versions of the same polymorphic function f (this is basically just the definition of function overloading, right?).

• Handle both stand-alone (unbound) functions and instance methods (bound functions).

• If there is a way to resolve a given function call, then do so (i.e. don't fail unless there is no function that can handle the given arguments).

Implementation

My idea was to use a controller class to store the different function definitions and use a decorator to register the functions with the controller. I wrote a module called overload.py with two classes: an OverloadedFunction class which stores a list of functions corresponding to the different overloaded versions of a given polymorphic function and an Overloader class which associates function names with OverloadedFunction objects.

The OverloadFunction objects have a lookup method which looks through the list of registered functions and returns the first one that matches the given set of arguments. The OverloadFunction objects are also callable and use their __call__ method to call the function obtained via the lookup method.

The Overload class defines an overload decorator which defines two wrapper functions: function_wrapper and method_wrapper. This handles both the cases of stand-alone (unbound) functions and instance methods (bound functions).

Here is the script:

#!/usr/bin/env python3
# coding: ascii
"""overload.py

Allow for multiple functions with different signatures
to have the same name (i.e. function overloading)."""

import inspect
from inspect import (
    getfullargspec,
    getmro,
    isfunction,
    ismethod,
)
from functools import (
    wraps,
)

class OverloadedFunctionError(Exception):
    """Exception class for errors related to the OverloadedFunction class."""
    pass

class OverloadedFunction(object):
    """An overloaded function.

    This is a proxy object which stores a list of functions. When called,
    it calls the first of its functions which matches the given arguments."""

    def __init__(self):
        """Initialize a new overloaded function."""

        self.registry = list()

    def lookup(self, args, kwargs):
        """Return the first registered function
        that matches the given call-parameters."""

        for function in self.registry:
            fullargspec = getfullargspec(function)

            # Make sure that the function can handle
            # the given number of positional arguments
            if(
                len(args) <= len(fullargspec.args) or
                bool(fullargspec.varargs)
            ):

                # Make sure that the function can handle
                # the remaining keyword arguments
                remaining_args = fullargspec.args[len(args):]
                if(
                    frozenset(kwargs.keys()).issubset(remaining_args) or
                    bool(fullargspec.varkw)
                ):
                    return function

    def register(self, function):
        """Add a new function to the registry."""

        self.registry.append(function)

    def __call__(self, *args, **kwargs):
        """Call the first matching registered
        function with the given call parameters."""

        # Get the first function which can handle the given arguments
        function = self.lookup(args=args, kwargs=kwargs)

        # If no function can be found, raise an exception
        if not function:
            raise OverloadedFunctionError(
                "Failed to find matching function for given arguments: "
                "args={}, kwargs={}".format(args, kwargs)
            )

        # Evaluate the function and return the result
        return function(*args, **kwargs)

class Overloader(object):
    """A controller object which organizes OverloadedFunction by name."""

    def __init__(self):
        """Initialize a new OverloadedFunction controller."""

        self.registry = dict()

    def register(self, function):
        """Add a new function to the controller."""

        # Create a new OverloadedFunction for this
        # function name if one does not # already exists
        if function.__qualname__ not in self.registry.keys():
            self.registry[function.__qualname__] = OverloadedFunction()

        self.registry[function.__qualname__].register(function)

    def overload(self, function):
        """Decorator for registering a new function with
        the Overloader overloaded function controller."""

        # Register the new function with the controller
        self.register(function)

        # Handle the case of unbound functions
        if isfunction(function):
            def function_wrapper(*args, **kwargs):
                _function = self.registry[function.__qualname__]
                return _function(*args, **kwargs)
            return function_wrapper

        # Handle the case of bound functions
        if ismethod(function):
            def method_wrapper(_self, *args, **kwargs):
                _function = self.registry[function.__qualname__]
                return _function(self=_self, *args, **kwargs)
            return method_wrapper

Example

I wrote a short example script to test out the module. Here is the script:

#!/usr/bin/env python3
# coding: ascii
"""test_overload.py

Simple tests of the overload.py module.
"""

from overload import Overloader

# Instantiate a controller object
p = Overloader()

# Define several function with the same name "f"
@p.overload
def f():
    return "called: def f()"

@p.overload
def f(x):
    return "called: def f(x) with x={}".format(x)

@p.overload
def f(y):
    return "called: def f(y) with y={}".format(y)

@p.overload
def f(*args, **kwargs):
    return "called: def f(*args, **kwargs) with args={}, kwargs={}".format(args, kwargs)

# Call the overloaded function "f" and print the results
print(f())
print(f(1))
print(f(x=2))
print(f(y=3))
print(f(1, 2, 3, x=4, y=5, z=6))

# Define a class with overloaded methods
class MyClass(object):

    @p.overload
    def g(self, x=1):
        return "called: def MyClass.g(x) with x={}".format(x)

    @p.overload
    def g(self, x=1, y=2):
        return "called: def MyClass.g(x, y) with x={}, y={}".format(x, y)

# Instantiate an object
myobject = MyClass()

# Call the overloaded method
print(myobject.g())
print(myobject.g(1, 2))

When I run the test_overload.py script I get the following output:

called: def f()
called: def f(x) with x=1
called: def f(x) with x=2
called: def f(y) with y=3
called: def f(*args, **kwargs) with args=(1, 2, 3), kwargs={'x': 4, 'y': 5, 'z': 6}
called: def MyClass.g(x) with x=1
called: def MyClass.g(x, y) with x=1, y=2

Comments and Questions

Am I overlooking anything important? Are there any counterintuitive behaviors or unexpected errors that this implementation might lead to? Are there any easily implementable features that I could add? I thought about trying to check for function collision when registering a function, i.e. checking to see if there are multiple functions that could be called for the same arguments. Right now I'm resolving this issue my just relying on the order in which the functions are registered, but maybe there's a better way?

Answer 1

funtools.singledispatch()

Take a look at the code for functools.singledispatch(). It's different in that it dispatches based on the type of the first argument. Look at how it wraps the first function with an object, so a separate controller object isn't needed. It also has code to deal with method resolution order if the overloaded functions are at different places in the class inheritance heirarchy.

inspect.Signature

In Python callable signatures can be pretty complicated. Args can be positional, keyword, or both, or positional only, keyword only, and can have default values. Take a look at the inspect.Signature class. It represents the signature of a callable and has a bind() method that can check if some args match the signature. It can simplify your code and handle the complications.

from inspect import signature

class OverloadedFunction(object):
    """An overloaded function.

    This is a proxy object which stores a list of functions. When called,
    it calls the first of its functions which matches the given arguments."""

    def __init__(self):
        """Initialize a new overloaded function."""

        self.registry = list()

    def register(self, function):
        """Add a new function and it's signature to the registry."""

        self.registry.append(function, signature(function))

    def __call__(self, *args, **kwargs):
        """Call the first matching registered
        function with the given call parameters."""

        # Get the first function which can handle the given arguments
        for function, signature in self.registry:
            try:
                signature.bind(*args, **kwargs)
                return function(*args, **kwargs)

            except TypeError:
                # If no function can be found, raise an exception
                raise OverloadedFunctionError(
                    "Failed to find matching function for given arguments: "
                    "args={}, kwargs={}".format(args, kwargs)
                )