Using argparse module within cmd interface

Question

I've created an application that uses a cmd interface. It has multiple levels, and the number of available commands and their complexity is growing. As such, I need to generalise argument parsing - of the line parameter.

I like argparse - I've previously used it a few times for its 'intended' purpose - so I figured I'd try that. Unfortunately, it doesn't lend itself well to being used in 'unconventional' ways, as many design decisions are forced on you (e.g. sys.exit() is called on invalid arguments, etc.).

I looked around for solutions and found this. However, that solution relinquishes cmd's listing of available functions, and its help functionality. As well, it requires that all the available commands and their arguments be listed in one place (i.e. __init__), which makes it impractical when there are many commands.

I came up with a solution using the decorator below:

class WrapperCmdLineArgParser:
    def __init__(self, parser):
        """Init decorator with an argparse parser to be used in parsing cmd-line options"""
        self.parser = parser
        self.help_msg = ""

    def __call__(self, f):
        """Decorate 'f' to parse 'line' and pass options to decorated function"""
        if not self.parser:  # If no parser was passed to the decorator, get it from 'f'
            self.parser = f(None, None, None, True)

        def wrapped_f(*args):
            line = args[1].split()
            try:
                parsed = self.parser.parse_args(line)
            except SystemExit:
                return
            f(*args, parsed=parsed)

        wrapped_f.__doc__ = self.__get_help(self.parser)
        return wrapped_f

    @staticmethod
    def __get_help(parser):
        """Get and return help message from 'parser.print_help()'"""
        f = tempfile.SpooledTemporaryFile(max_size=2048)
        parser.print_help(file=f)
        f.seek(0)
        return f.read().rstrip()

It allows a command/function to be defined independently, taking in a third argument parsed, which will contain the return value of a successful parse_args(). There are 2 ways to define a function/command:

This allows one to pass the parser as an argument to the decorator. A simple way would be to define name mangled static variables, as below __test1_parser.

__test1_parser = argparse.ArgumentParser(prog="test1")
__test1_parser.add_argument('--foo', help="foo help")

@WrapperCmdLineArgParser(parser=__test1_parser)
def do_test1(self, line, parsed):
    print("Test1...")
    print(parsed)

In some cases, specially if the parser is complex, it may be preferred to define the parser within the function itself. This is possible as well. The function would have to take a 4th argument get_parser=False, and return the parser when True.

@WrapperCmdLineArgParser(parser=None)
def do_test2(self, line, parsed, get_parser=False):
    if get_parser:
        parser = argparse.ArgumentParser(prog="test2")
        parser.add_argument('--bar', help="bar help")
        return parser

    print("Test2...")
    print(parsed)

Regardless of the way in which they are defined, the functions will be listed under cmd's help

(Cmd) help

Documented commands (type help <topic>):
========================================
help  test1  test2

as well as have their own help messages - via help <func_name> or <func_name> -h.

(Cmd) help test1
usage: test1 [-h] [--foo FOO]

optional arguments:
  -h, --help  show this help message and exit
  --foo FOO   foo help
(Cmd)  test2 -h
usage: test2 [-h] [--bar BAR]

optional arguments:
  -h, --help  show this help message and exit
  --bar BAR   bar help

Furthermore, when parsing invalid arguments, the error message will be printed, and it will return to the prompt, without calling the given function.

(Cmd) test1 --unk
usage: test1 [-h] [--foo FOO]
test1: error: unrecognized arguments: --unk
(Cmd)

I think this is a decent solution that allows one to simply and independently define commands that use argparse, while maintaining and not clashing with any of cmd's functionality.

What are some possible drawbacks of this solution? What are some possible improvements, or obvious bugs? Is there a better solution using cmd and argparse? Do any other modules exist that provide the functionality I am trying to achieve? Which of the 2 ways above would be better?

Thus far I have noticed that on PyCharm's debugger console, when there are invalid arguments and SystemExit is raised, the error messages and the prompt sometimes overlap or are out of order. It doesn't happen on a regular terminal.

Answer 1

Not as much as a code review, but an architecture suggestion. Your code is fine, but I'm weary of the cleverness we tend to add when things grow more and more complex.

Unless you really want to run everything from a single .py module, I'd suggest breaking up the program into multiple pieces. Have your shared pieces inside a common library and split the different groups into different executables, a bit like git and apt do - by calling external utilities. This also allows other people to easily write add-ons for your utility.

Think of utility verb --be-awesome as just calling utility-verb --be-awesome under the hood and passing whatever command line parameters it got. As a useful convention, you may want to have parameters that affect the base utility that could be stripped out of the command line when called or just force them to be placed between the utility and the verb parts. All utility-verb commands should be able to rely on the shared functionality on the library.