I'm working on an open-source test framework which needs to dynamically import Python source modules from a given directory.
I can't just use
__import__(), because it's a black box and I need to do some source-code-rewriting. On the other hand, I feel the PEP 302 Import Hooks to be overkill for what I'm trying to do (they seem to be designed with importing totally foreign file formats in mind), so I rolled my own:
import os import sys import types def import_module(dir_path, module_name): if module_name in sys.modules: return sys.modules[module_name] filename = resolve_filename(dir_path, module_name) with open(filename, 'r', encoding='utf-8') as f: source = f.read() # Here I do some AST munging code = compile(source, filename, 'exec') module = create_module_object(module_name, filename) exec(code, module.__dict__) sys.modules[module_name] = module return module def resolve_filename(dir_path, module_name): filename = os.path.join(dir_path, *module_name.split('.')) # I happen to know that the calling code will already have # determined whether it's a package or not if os.path.isdir(filename): filename = os.path.join(filename, '__init__.py') else: filename += '.py' return filename def create_module_object(module_name, filename): module = types.ModuleType(module_name) module.__file__ = filename if '__init__.py' in filename: module.__package__ = module_name module.__path__ = [os.path.dirname(filename)] else: if '.' in module_name: module.__package__, _ = module_name.rsplit('.', 1) else: module.__package__ = '' return module
To paraphrase Greenspun's Tenth Rule:
Any sufficiently complicated Python program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of the import system.
So, does my code do everything it's supposed to? I wrote my tests against an implementation that used
importlib.import_module() and then rewrote it as this one, which gives me some confidence, but I'm not sure whether I've missed any important steps.