Plugin Pattern for Generic Python Application

Question

Summary

I am experimenting with a plugin pattern for a generic Python application (not necessarily web, desktop, or console) which would allow packages dropped into a plugin folder to be used according to the contract they would need to follow. In my case, this contract is simply to have a function called do_plugin_stuff(). I'd like the pattern to make sense for a system that sells plugins like the plugin store in Wordpress.

Minimal Python plugin mechanism is a decent question (despite being 4 years old) with some very good discussion about Django (which I haven't used) and how it allows for a plugin to be installed anywhere via pip. I'd see that as a phase two, because it seems like a pip-based plugin pattern is (sweeping generalization probably not always true) most valuable for purely free (as in money) plugin store. If free (as in open source) plugins are sold for money in a store, if seems that pip would be a poor choice for installation because which people might pay for something they're about to get the source code for and use freely / redistribute, they might be unlikely to pay / donate for something they've already installed.

Project Structure

Code

It's also on GitHub under my same username (PaluMacil) and I made a release tag of v1.0.0 to freeze the the repo at the code shown below.

app/plugins/blog/__init__.py

def do_plugin_stuff():
    print("I'm a blog!")

app/plugins/toaster/__init__.py

def do_plugin_stuff():
    print("I'm a toaster!")

app/plugins/__init__.py

(empty)

app/__init__.py

from importlib import import_module
from os import path, listdir


def create_app():
    app = Application()
    plugin_dir = path.join(path.dirname(__file__), 'plugins')

    import_string_list = [''.join(['.plugins.', d]) for d
                          in listdir(plugin_dir)
                          if path.isdir(path.join(plugin_dir, d))
                          and not d.startswith('__')]

    print(str(len(import_string_list)) + " imports to do...")

    for import_string in import_string_list:
        module = import_module(import_string, __package__)
        app.plugins.update({module.__name__.split('.')[2]: module})

    print(str(len(app.plugins)) + " plugins in the app")
    return app


class Application:
    def __init__(self):
        self.plugins = {}

The line not d.startswith('__') eliminated my pychache dir from Pycharm.

run.py

from app import create_app
from pprint import PrettyPrinter


app = create_app()

app.plugins['toaster'].do_plugin_stuff()

printer = PrettyPrinter(indent=4)
printer.pprint(app.plugins.__repr__())

Points for Review

I'm new enough to Python (very new but coming from a decent C# background, and I read PEP8 before attempting this) that I've never written a Python 2 application. I think my method of importing requires Python 3.3 or 3.4, though I'm not certain. Commentary on this might be nice. Ways of making this code accessible to earlier versions of Python seem to be messy; they involve conditional imports and such, which are verbose and ugly. If there is a trick or two that would make my code better for different versions of Python with minimal cruft, that would be great to see.

Am I missing anything that makes my code much more verbose than it should be? For instance, I'm iterating twice through the directories--once to make a list of packages, and again to make my dictionary. Would it be cleaner to make both parts one loop? The one-loop alternative seems verbose, but there could be further improvements, perhaps:

# Alternative to current code which uses a single loop:
for d in listdir(plugin_dir):
    if path.isdir(path.join(plugin_dir, d)) and not d.startswith('__'):
        module = import_module(''.join(['.plugins.', d]), __package__)
        app.plugins.update({module.__name__.split('.')[2]: module})

Is module.__name__.split('.')[2] a fragile way to get the value for my plugin dictionary? Would [-1] be a better index to use on the result of the split?

I'm having trouble understanding why I might chose to use pkgutil.iter_modules instead of my approach, but I'm wondering if there might be some benefit. It seems to be based on importlib since Python 3.3 (PEP 302). Would the only difference be that I wouldn't pull in a folder that doesn't have an __init__.py inside it to make it a package?

Answer 1

You shouldn't call str on the int returned from len, instead use str.format.

"{} plugins in the app".format(len(app.plugins))

Format will coerce the int to a string implicitly and is neater to read.

Also you're calling repr backwards. The whole point of an object having a __repr__ function is that it allows an object to be passed to repr(). So you could change app.plugins.__repr__() to repr(app.plugins).