3
\$\begingroup\$
import nltk

req_modules = {'from nltk import punkt': 'punkt',
               'from nltk.corpus import stopwords': 'stopwords',
               'from nltk import pos_tag': 'averaged_perceptron_tagger',
               'from nltk import ne_chunk': 'maxent_ne_chunker'}

for m in req_modules:
    try:
        print("Trying: '%s'" % m)
        exec(m)    # try to import the package
        print("Success.")
    except (LookupError, ImportError):
        # if data not found (not already installed), download it
        print("Tried: %s. Resource '%s' was not available \
              and is being downloaded.\n" % (m, req_modules[m]))
        nltk.download(req_modules[m])
        exec(m)

I have some code I'm distributing to folks at work, and the lines above are at the top of the file. In order to run the code underneath (which harnesses several subpackages in nltk), the user needs to have downloaded these subpackages. Is this is safe/reliable way of doing dynamic module import? It has worked for me thus far.

\$\endgroup\$
3
\$\begingroup\$

I mean I'm not a fan of it since it's basically just a list of statements to execute, but then again, the solution via globals and importlib doesn't look that much better:

import importlib
import nltk

req_modules = {'nltk.punkt': 'punkt',
               'nltk.corpus.stopwords': 'stopwords',
               'nltk.pos_tag': 'averaged_perceptron_tagger',
               'nltk.ne_chunk': 'maxent_ne_chunker'}

def try_load(module, name):
    print("Trying to load: '%s'" % module)
    globals()[name] = importlib.import_module(module)
    print("Success.")

for module, name in req_modules.items():
    try:
        try_load(module, name)
    except (LookupError, ImportError):
        # if data not found (not already installed), download it
        print("Tried to load: '%s'. Resource '%s' was not available \
               and is being downloaded.\n" % (module, name))
        nltk.download(name)
        try_load(module, name)

N.b. I'd say use a list of module/name pairs instead to enforce the order to make diagnostics a bit easier (or load dependencies for one of the later modules first).

Now that I read it again, this seems alright if it's in a module and reusable, otherwise doesn't look like it's less safe than other options that involve nltk.download and it provides informative output while it gets things, so that's nice.

\$\endgroup\$
  • \$\begingroup\$ Why do you say "I'm not a fan because it's basically just a list of statements to execute?" \$\endgroup\$ – blacksite Nov 16 '16 at 20:07
  • \$\begingroup\$ Stringly typed, repetitive (from ... import ...) and it's using exec when it's not strictly necessary - you can create a more structured model by using regular functions, which is normally preferable to running exec on a string. Like I said, it's not that much of a difference. \$\endgroup\$ – ferada Nov 16 '16 at 20:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.