Python Export Decorator

Question

In Python, we use __all__ to explicitly denote the public API of a module or package. That looks like this:

__all__ = ['Foo', 'bar', 'BAZ_CONSTANT']

BAZ_CONSTANT = None

class Foo:
    pass

def bar():
    pass

... except that modules can go on for thousands of lines, and then it is not immediately apparent, without revisiting the heading of the file, which objects are for external use, and which are not.

We are also required to write the names of the objects twice for each of these objects. (At a minimum. Even more for unittests - and we write our tests first, don't we?)

What I want is a decorator that allows me to shift the declaration of the public API away from the heading and to the object and constant definitions, where we, in most cases, can only write the name to be exported once.

Here's a serviceable preliminary draft, targeted to Python 3.6+ - perhaps the simplest thing that could possibly work:

import sys

class ExportError(Exception):
    """Exception if redundant names attempted exporting"""

def export(obj, global_namespace: dict=None) -> 'obj arg':
    """export adds names of functions and classes via decorator
    and module level variables via call (with globals() argument)
    to __all__.
    Usage:
    from exportlib import export

    export("BAZ_CONSTANT", globals()) # I don't like globals() call...
    BAZ_CONSTANT = None

    @export
    class Foo: 
        pass

    @export
    def bar(): 
        pass
    """
    name = obj if isinstance(obj, str) else obj.__name__
    if global_namespace is None:
         try:
             global_namespace = obj.__globals__
         except AttributeError as error:
             raise AttributeError(
               f'no globals, try export({name!r}, globals())')
    __all__ = global_namespace.setdefault('__all__', [])
    if name in __all__: # Do we care if redundant exports? Assuming so:
        raise ExportError(f'{name} already exported')
    __all__.append(name)
    return obj

Ok, could probably be a little simpler with fewer features. I do like the documenting features of the annotations. I couldn't identify a more appropriate Exception to subclass or use directly than Exception.

I don't like the globals() call required when adding a module level variable that doesn't know its own name or module, but the only alternative, that I can think of, is to do what super does and climb a stackframe up and get the globals there, or pass in __all__ instead (which would require creating it explicitly). Stackframe climbing would preclude export from being wrapped/extended.

But I think this makes a sufficiently sophisticated starting point.

Answer 1

• Personally, I'd prefer to use @export() or @export(globals()), rather than manually perform the @ sugar.
• I don't like using __globals__, even if it is writeable.
• Honestly, I think the cleanest solution would be to abuse lists, and use a closure.
• I do however agree that inheriting from Exception, is the best parent to inherit from.

And so I'd personally use something more like:

class ExportError(Exception):
    """Exception if redundant names attempted exporting"""

def build_export(__all__=None):
    if __all__ is None:
        __all__ = []
    def export(obj):
        name = obj if isinstance(obj, str) else obj.__name__
        if name in __all__:
            raise ExportError(f'{name} already exported')
        __all__.append(name)
        return obj
    return __all__, export


__all__, export = build_export()


@export
class Foo:
    pass


@export
def bar():
    pass

Answer 2

If there's too much magic, I worry about static linters and new users not understanding the code, so I would want to support proper global variable assignment.

On the other hand, Peilonrayz's answer seems overly rigid - returning a tuple? I'd like those who would want minimalist usage to be able to have it.

First a disclaimer

I'd never submit this code for review at work or on Python-dev for inclusion in the standard library.

The existing mechanism is fine, even if it is redundant, and I think there's a benefit to declaring the API at the top instead of scattered throughout.

And even if my code works perfectly bug-free, its existence could lead to the easy introduction of further bugs.

Suggestion

How about a list subclass that implements __call__ to decorate functions/classes?

import inspect

class ExportError(Exception):
    """Exception if redundant names attempted exporting"""

class Export(list):
    __slots__ = 'global_namespace',
    def __init__(self, *obj, **globals_):
        self.global_namespace = inspect.currentframe().f_back.f_globals
        self.global_namespace.setdefault('__all__', self)
        self(*obj, **globals_)
    def __call__(self, *obj, **globals_):
        for obj_ in obj:
            if isinstance(obj_, str):
                for name in obj_.split():
                    self.append_name(name)
            else:
                self.append_name(obj_.__name__)
        self.extend(globals_)
        self.global_namespace.update(globals_)
        return obj[0] if obj else None
    def append_name(self, name):
        if name in self:
            raise ExportError(f'{name} already exported')
        self.append(name)

The downside of accessing the globals this way is that the __init__ can't be extended in a subclass. Plus it just seems hack-y.

Usage

Clean import (kinda funny to read out loud though):

from exportlib import Export

Clean assignment for the benefit of IDEs that understand __all__:

__all__ = export = Export()

We have the following minimalist usage supported - but it is potentially confusing to add to globals this way for static linting, like pyflakes - and even experienced users could be confused:

export = Export(
  BAZ_CONSTANT=None,
)

So we also support this slightly more verbose usage - which also supports annotations:

__all__ = export = Export('BAZ_CONSTANT', 'QUUX_CONSTANT')

BAZ_CONSTANT = None
QUUX_CONSTANT: None = None

Finally, we allow the namedtuple style single string API, splitting on the spaces:

__all__ = export = Export('BAZ_CONSTANT QUUX_CONSTANT')

BAZ_CONSTANT = None
QUUX_CONSTANT: None = None

And other objects get decorated and added to __all__:

@export
class Foo: 
    pass

@export
def bar(): 
    pass

Interactive Sessions

Here's an interactive session:

>>> __all__ = export = Export('BAZ_CONSTANT QUUX_CONSTANT')
>>> 
>>> BAZ_CONSTANT = None
>>> QUUX_CONSTANT: None = None
>>> 
>>> __all__
['BAZ_CONSTANT', 'QUUX_CONSTANT']
>>> 
>>> @export
... class Foo: 
...     pass
... 
>>> @export
... def bar(): 
...     pass
... 
>>> __all__
['BAZ_CONSTANT', 'QUUX_CONSTANT', 'Foo', 'bar']

Both style checkers and new users should find the code easy to understand.

Here's the Proof of Concept (POC) for the minimalist usage as well - if this got into the standard library (inconceivable), then this usage could get into common practice and style checkers would need to be updated:

>>> export = Export(
...   BAZ_CONSTANT=None,
... )
>>> BAZ_CONSTANT is None
True
>>> __all__
['BAZ_CONSTANT']
>>> @export
... class Foo: 
...     pass
... 
>>> @export
... def bar(): 
...     pass
... 
>>> __all__
['BAZ_CONSTANT', 'Foo', 'bar']

Final acceptance test

Finally, putting Export into exportlib.py and the above minimalist-style code into user.py, we test import * (which can be appropriately used to provision an API point.) If this works, this should fully test the globals lookup in the __init__:

$ python -i
Python 3.6.1 |Anaconda 4.4.0 (64-bit)| (default, May 11 2017, 13:09:58) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import user
>>> from user import *
>>> dir()
['BAZ_CONSTANT', 'Foo', '__annotations__', '__builtins__', '__doc__',
 '__loader__', '__name__', '__package__', '__spec__', 'bar', 'user']
>>> BAZ_CONSTANT is None
True

And it seems to work just fine.