14
\$\begingroup\$

I have a package directory pkg with several classes that I would like to build into a convenient dict property.

The structure of pkg/ looks like:

pkg/base.py:

class _MyBase(object):
    pass

pkg/foo.py:

from .base import _MyBase

class Foo(_MyBase):
     pass

And in pkg/__init__.py, it is a bit clunky, but once pkg is imported, a all_my_base_classes dict is built with a key of the class name, and value of the class object. The classes are all subclasses of pkg.base._MyBase.

import os
import sys
import pkgutil

import base
# I don't want to import foo, bar, or whatever other file is in pkg/

all_my_base_classes = {}
pkg_dir = os.path.dirname(__file__)
for (module_loader, name, ispkg) in pkgutil.iter_modules([pkg_dir]):
    exec('import ' + name)
    pkg_name = __name__ + '.' + name
    obj = sys.modules[pkg_name]
    for dir_name in dir(obj):
        if dir_name.startswith('_'):
            continue
        dir_obj = getattr(obj, dir_name)
        if issubclass(dir_obj, base._MyBase):
            all_my_base_classes[dir_name] = dir_obj

Running it from an interactive Python shell, one directory below pkg/:

>>> import pkg
>>> pkg.all_my_base_classes
{'Foo': <class 'pkg.foo.Foo'>}

So it works as expected, but pkg/__init__.py is pretty terrible looking. How can it be better?

\$\endgroup\$
2
  • \$\begingroup\$ Why not import the modules normally in __init__? \$\endgroup\$ Nov 19, 2014 at 9:47
  • \$\begingroup\$ Which version of python are you using? \$\endgroup\$
    – Nihathrael
    Nov 19, 2014 at 9:47

2 Answers 2

16
\$\begingroup\$

Since the classes are all subclasses of _MyBase, they can be accessed via _MyBase.__subclasses__() after they have been imported:

for (module_loader, name, ispkg) in pkgutil.iter_modules([pkg_dir]):
    importlib.import_module('.' + name, __package__)

all_my_base_classes = {cls.__name__: cls for cls in base._MyBase.__subclasses__()}

For importing the modules, I followed the advice of Nihathrael.

\$\endgroup\$
2
  • \$\begingroup\$ Cheers, very nice and concise answer! \$\endgroup\$
    – Nihathrael
    Nov 19, 2014 at 10:53
  • \$\begingroup\$ __subclasses__(), fantastic! I also found this answer very useful. \$\endgroup\$
    – Mike T
    Nov 19, 2014 at 22:13
4
\$\begingroup\$

Using exec() is generally regarded as bad practice (https://stackoverflow.com/questions/1933451/why-should-exec-and-eval-be-avoided). Since you are trying to import a module, you can do the same by using the importlib package.

Python 2: https://docs.python.org/2/library/importlib.html

Python 3: https://docs.python.org/3/library/importlib.html#module-importlib

Using it should clean up the rest of the code as well.

You might also be interested in this discussion for more possibilities regarding imports: https://stackoverflow.com/questions/1057431/loading-all-modules-in-a-folder-in-python

Edit 2: Removed my code example, as @janne-karila's solution is even shorter and more concise.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.