Python interactive console which automatically reload modules

Question

I just started to program Python. As my first application I implemented an interactive console which automatically python modules before a new code block is evaluated. Here is the code:

### module "module_reloading_console.py"
import sys
import code
import imp
import readline
import os

class ModuleReloadingConsole(code.InteractiveConsole):

    def __init__(self):
        super().__init__()
        readline.parse_and_bind("tab: complete")
        self.stored_modifier_times = {}
        self.check_modules_for_reload()

    def runcode(self, code):
        self.check_modules_for_reload()
        super().runcode(code)
        self.check_modules_for_reload() # maybe new modules are loaded

    def check_modules_for_reload(self):
        for module_name, module in sys.modules.items():
            if hasattr(module, '__file__'):
                module_modifier_time = os.path.getmtime(module.__file__)

                if module_name in self.stored_modifier_times:
                    if module_modifier_time > self.stored_modifier_times[module_name]:
                        imp.reload(module)
                    self.stored_modifier_times[module_name] = module_modifier_time
                else:
                    self.stored_modifier_times[module_name] = module_modifier_time

ModuleReloadingConsole().interact("Welcome to ModuleReloadingConsole")

Usage:

~: python3 module_reloading_console.py 
Welcome to ModuleReloadingConsole
>>> import test
>>> test.foo
23
>>> test.foo # in the meanwhile change foo to '42' in test.py
42

Because I am really new to Python I want to now which improvements you have for the above code. Are there common Python conventions I have not considered?

marginal note: this code recipe (automatically upgrade class instances on reload()) would be a good improvement. Maybe also this article about dynamically reloading a python code might improve the code...

Answer 1

Automatic tools

PEP8: I'd say your code's OK style wise, but this is what the checker said:

1:1: E266 too many leading '#' for block comment
19:40: E261 at least two spaces before inline comment
27:80: E501 line too long (86 > 79 characters)
29:80: E501 line too long (82 > 79 characters)
31:80: E501 line too long (82 > 79 characters)

Pylint: Apart for missing doc strings, the only thing it finds is that in your runcode method, you shadow a module name, code. It may be better to call you argument code_to_run or inp so that code always refers to the module. But, as the method's short, its not a big deal.

Human tool

Your check_modules_for_reload method looks like this:

def check_modules_for_reload(self):
    for module_name, module in sys.modules.items():
        if hasattr(module, '__file__'):
            module_modifier_time = os.path.getmtime(module.__file__)

            if module_name in self.stored_modifier_times:
                if module_modifier_time > self.stored_modifier_times[module_name]:
                    imp.reload(module)
                self.stored_modifier_times[module_name] = module_modifier_time
            else:
                self.stored_modifier_times[module_name] = module_modifier_time

Things to note:

• Your if statement checking if you have already seen the module contains the line:

  self.stored_modifier_times[module_name] = module_modifier_time
  

  As this occurs in both branches of the if statement, you can move it out, after the branching.

• Currently your code involves 4 indentation levels. You can reduce this by inverting the check_modules_for_reload logic and using continue.

The code now looks like:

def check_modules_for_reload(self):
    for module_name, module in sys.modules.items():
        if not hasattr(module, '__file__'):
            continue
        module_modifier_time = os.path.getmtime(module.__file__)

        if module_name in self.stored_modifier_times:
            if module_modifier_time > self.stored_modifier_times[module_name]:
                imp.reload(module)
        self.stored_modifier_times[module_name] = module_modifier_time

We see that the nested if could be reduced to:

if module_name in self.stored_modifier_times and module_modifier_time > self.stored_modifier_times[module_name]

That's long and horrid. But we can use dict.get combined with the fact module_modifier_time > module_modifier_time is False to produce:

if self.stored_modifier_times.get(module_name, module_modifier_time) < module_modifier_time

Which is slightly shorter. The code is now:

def check_modules_for_reload(self):
    for module_name, module in sys.modules.items():
        if not hasattr(module, '__file__'):
            continue

        module_modifier_time = os.path.getmtime(module.__file__)
        if self.stored_modifier_times.get(module_name, module_modifier_time) < module_modifier_time:
            imp.reload(module)
        self.stored_modifier_times[module_name] = module_modifier_time

Two lines shorter, two indentation levels and a bit more DRY. It might not be much, but the code was good to start with. Other things:

• os.getmtime may throw OSError, but assuming no ones deleting modules or doing weird things this should not matter.

• imp.reload could throw a whole horde of exceptions, crashing the shell. E.g.:

  >>> m.a
  Traceback (most recent call last):
  File "rl.py", line 32, in <module>
      ModuleReloadingConsole().interact("Welcome to ModuleReloadingConsole")
  File "/usr/lib/python3.4/code.py", line 234, in interact
      more = self.push(line)
  File "/usr/lib/python3.4/code.py", line 256, in push
      more = self.runsource(source, self.filename)
  File "/usr/lib/python3.4/code.py", line 74, in runsource
      self.runcode(code)
  File "rl.py", line 17, in runcode
      self.check_modules_for_reload()
  File "rl.py", line 28, in check_modules_for_reload
      imp.reload(module)
  File "/usr/lib/python3.4/imp.py", line 315, in reload
      return importlib.reload(module)
  File "/usr/lib/python3.4/importlib/__init__.py", line 149, in reload
      methods.exec(module)
  File "<frozen importlib._bootstrap>", line 1153, in exec
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1467, in exec_module
  File "<frozen importlib._bootstrap>", line 1572, in get_code
  File "<frozen importlib._bootstrap>", line 1532, in source_to_code
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "/home/matthew/m.py", line 1
      a=5svnls2 
              ^
  SyntaxError: invalid syntax
  

  After which the shell exits.