1
\$\begingroup\$

The div and add functions are located randomly in my package structure. I have a function with this signature:

def calc(x, y, func_name='add')

How do I implement this function?

My idea is to import all the modules, then use the dir function, like so:

def import_submodules(package, recursive=True):
    """ Import all submodules of a module, recursively, including subpackages

    :param package: package (name or actual module)
    :type package: str | module
    :rtype: dict[str, types.ModuleType]
    """
    if isinstance(package, str):
        package = importlib.import_module(package)
    results = {}
    for loader, name, is_pkg in pkgutil.walk_packages(package.__path__):
        full_name = package.__name__ + '.' + name
        print 'full_name =', full_name
        results[full_name] = importlib.import_module(full_name)
        if recursive and is_pkg:
            results.update(import_submodules(full_name))
    return results

def create_symbol_module_d(module_name_module_d, include=None, exclude=None):
    """ Create mapping of symbol to module

    :param module_name_module_d
    :type module_name_module_d: dict[str, types.ModuleType]
    :rtype: dict[str, types.*]
    """
    inv_res = {}
    for mod in module_name_module_d.itervalues():
        for sym in dir(mod):
            if include and sym in include:
                inv_res[sym] = mod
            elif exclude and sym not in exclude:
                inv_res[sym] = mod
            else:
                inv_res[sym] = mod
    return inv_res

Then I can just do:

sym2mod = create_symbol_module_d(import_submodules('package_name'))
def calc(x, y, func_name='add'):  return sym2mod[func_name](x, y)
\$\endgroup\$
2
  • \$\begingroup\$ Are you only adding custom computation for your custom type? Then all you might need is to implement some special methods. \$\endgroup\$ Oct 22 '15 at 9:52
  • \$\begingroup\$ Thanks, but this calc is just a simple example. My real use-case is for a pseudo plugin system. \$\endgroup\$
    – A T
    Oct 22 '15 at 10:02
1
\$\begingroup\$

Well I'm going to start by saying that your end result is confusing. sym2mod and create_symbol_module_d aren't clear names that indicate what's going on. In particular I don't know why this would be necessary, so it wouldn't occur to me that these are custom adding and dividing functions that are placed so erratically in a package that you can't even find them. Also there is already an operator module that contains these functions. If yours are more specialised or complex, then you should use different names to distinguish them and reduce confusion.

Which brings me to another problem, why are they in random places? It sounds like your problem entirely exists just because of the package structure. To me that means you ought to improve the actual structure rather than treating the symptom of the problem.

Anyway, onto your actual functions. The docstring for import_module is a little off. It says the import is recursive, but it isn't necessarily recursive, it's optionally so. I don't think you need to mention it since it's easy to see the arguments with introspection. ie.

>>> help(import_submodules)
Help on function import_submodules in module __main__:

import_submodules(package, recursive=True)
    Import all submodules of a module, recursively, including subpackages

The fact that your importer is also printing out messages could be quite a problem for people who might have a large package, and there's no way to turn it off! I'd add an argument like logging and default it to False. Only print if its been explicitly asked for. After all there's a dictionary record of the importing that can be inspected for what was imported.

create_symbol_module_d is quite confusing to me. It's a bad name, so is module_name_module_d. They're both long and communicate little. Then there's include and exclude. Are they meant to be lists of symbols like ['+', '-', '/']? You give no examples or indication. Then when I look at what they do... they do nothing! In every result of your if statements you end up just performing the same test. In the current function there's no need for if statements at all, let alone include and exclude. This is equivalent to your function:

def create_symbol_module_d(module_name_module_d, include=None, exclude=None):
    """ Create mapping of symbol to module

    :param module_name_module_d
    :type module_name_module_d: dict[str, types.ModuleType]
    :rtype: dict[str, types.*]
    """
    inv_res = {}
    for mod in module_name_module_d.itervalues():
        for sym in dir(mod):
            inv_res[sym] = mod
    return inv_res

You need better names and documentation for me to even be able to give feedback on it. I also don't know what inv_res is or means.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.