6
\$\begingroup\$

When I show others how to use this module I have them make a PROJECT instance and then add other objects.

>>> from Abstracted import PROJECT

>>> project = PROJECT('bob', Y_factor=2, use_MQTT=True)
MQTT status: started
Yay, your new Project named "bob" has been created

They may do this from a terminal which does not show the docstring balloons the way IDLE and many other IDEs can. So instead I've found a way so that when they type the instance, a few lines only of helpful text appears. This is in contrast to help() which can generate dozens of lines and scroll their previous work right up off the top of the screen.

>>> project
PROJECT("bob")
  intantiate with p = PROJECT(name, Y_factor=None, use_MQTT=False)
  later use start_MQTT(), stop_MQTT(), and
  add THINGs with p.new_CARROT(name) or p.new_ONION(name, n_rings=None)
>>> carrot = project.new_CARROT('carrotte')
a CARROT was added!
>>> onion = project.new_ONION('onionne', n_rings=42)
an ONION was added!

Type the object, and a few lines with a few "main methods" and helpful hints shows up. This is the functionality that I am asking about.

>>> carrot
CARROT("carrotte")
  instantiate with c = CARROT(project, name)
  change thread with c.change_thread_value(new_value)

>>> onion
ONION("onionne"), n_rings=42
  instantiate with o = ONION(project, name, n_rings=None)
  increase onionrings with o.multiply(factor)

Question: Am I re-inventing the wheel? Is this recreating something that Python does naturally? I've only really learned about docstrings in the past 24 hours, so maybe there are better ways to do this?


Here is Abstracted.py, an abstracted version of a large module. All of the functionality I'm asking about is contained here.

import time, datetime, logging, socket, Queue
from threading import Thread, Lock, Event
#import paho.mqtt.client as mqtt 

important_lock = Lock()   # the rest of threading not shown

class PROJECT(object):
    """  intantiate with p = PROJECT(name, Y_factor=None, use_MQTT=False)
  later use start_MQTT(), stop_MQTT(), and
  add THINGs with p.new_CARROT(name, X) or p.new_ONION(name)"""
    def __init__(self, name, Y_factor=None, use_MQTT=False):

        self.name               = name
        self.use_MQTT           = use_MQTT
        self.things             = []
        if Y_factor == None:
            Y_factor = -1
        self.Y_factor           = Y_factor
        if self.use_MQTT:
            status = self.start_MQTT()
            print "MQTT status: {}".format(status)
        self.info = ('{self.__class__.__name__}("{self.name}")'
                .format(self=self))
        print 'Yay, your new Project named "{}" has been created'.format(self.name)

    def __repr__(self):
        return (self.info + '\n' + self.__doc__)

    def doc(self):
        print self.__doc__

    def start_MQTT(self, start_looping=False):
        """(start_looping=False) creates an MQTT client attribute and connects it"""
        # do stuff
        status = 'started'
        self.client = 'client instance'
        return status

    def stop_MQTT(self):
        """disconnects from MQTT"""
        # do stuff
        status = 'stopped'
        return status

    def new_ONION(self, name, n_rings=None):
        """def new_ONION"""

        onion = ONION(project=self, name=name, n_rings=n_rings)

        self.things.append(onion)
        print "an ONION was added!"
        return onion        

    def new_CARROT(self, name):
        """def new_CARROT"""

        carrot = CARROT(project=self, name=name)

        self.things.append(carrot)
        print "a CARROT was added!"
        return carrot

class THING(object):
    """THING!"""

    def __init__(self, project, name):
        """THING.__init__!"""

        self.project              = project
        self.name                 = name
        self.Y_factor             = self.project.Y_factor

    def __repr__(self):
        return (self.info + '\n' + self.__doc__)

class ONION(THING):
    """  instantiate with o = ONION(project, name, n_rings=None)
  increase onionrings with o.multiply(factor)"""
    def __init__(self, project, name, n_rings=None):
        """ONION.__init__!"""
        if n_rings==None:
            n_rings = 7
        self.n_rings = n_rings

        THING.__init__(self, project, name)

        self.info = ('{self.__class__.__name__}("{self.name}"), n_rings={self.n_rings}'
                .format(self=self))

    def multiply(self, factor):
        if type(factor) in (int, float):
            self.n_rings *= factor

class CARROT(THING):
    """  instantiate with c = CARROT(project, name)
  change thread with c.change_thread_value(new_value)"""
    def __init__(self, project, name):
        """CARROT.__init__!"""

        self.thread_value = 1

        THING.__init__(self, project, name)

        self.info = ('{self.__class__.__name__}("{self.name}")'
                .format(self=self))

    def change_thread_value(self, new_value):
        with important_lock:
            self.thread_value = new_value
\$\endgroup\$
  • \$\begingroup\$ Good question! Would print(onion.__doc__) provide the desired behavior? What about inspect.getdoc()? \$\endgroup\$ – alecxe Jan 2 at 16:41
  • \$\begingroup\$ @alecxe my power went off right after I posted, just got back on line... Those are particularly wordy for two-finger typers, and remembering that is a bit hard for python noobs, and by the time they finish typing it they may forget what it was they were hoping to do. I'm looking for a quick way, as easy as remembering help() but not actually help(). Thanks! \$\endgroup\$ – uhoh Jan 2 at 23:08
7
\$\begingroup\$

I'm not sure whether you're reinventing the wheel, but you are using the wrong tools for the job.

I've been through the phase where I think I'm doing something unique enough that I need to do something nonstandard. Every time I've done it, I inevitably regret it. Don't do it. It's not a good idea. Use tools very closely to the way they were intended to be used.

You seem to be trying to do basically two things.

A guidance system to aid developers

The first is a sort of "brief help" that prompts the users with details about the API.

You implement this with repr, but repr is absolutely the wrong tool for this job. repr is not intended to return help messages. repr is intended to be a quick and dirty introspection tool, showing just enough information for the programmer to identify the object they're looking at. You're reducing their ability to do so by misusing it, which is far and away something I would not advise for beginners.

To be honest, I wouldn't recommend building this into Python itself at all. Doing so is very nonstandard. Instead, I'd be more inclined to generate a set of HTML docs for them to use. pydoc is built in and is probably a good starting point. Sphinx is also well known and used. Python's wiki lists others to consider as well. Some of these will allow you to include documentation not just of the API, but also custom pages including examples and what have you. A set of HTML docs is likely to be infinitely more usable to your users since it doesn't require backspacing the command they're in the middle of typing.

Bottom line: I'd use the existing standard tools to provide your users with this kind of guidance, and the standard is to make it external to the code itself.

You might also want to take note of the dir command. dir(o) lists the names bound to o, which includes attributes, descriptors (similar to "properties" in other languages), and methods. This can be combined with pprint or pformat for easier to read output, and the if clause of list comprehensions can be used to filter out private names (i.e., [a for a in dir(o) if not a.startswith('_')]) or based on other conditions. I use this myself to explore an API when the documentation is poor. vars(o) may also be useful for viewing attributes.

If it absolutely must be accessible by code, I think I'd probably implement it as a new "protocol". Something like this:

class PROJECT(object):
    # Your other code
    @staticmethod
    def _mytips():
         return """
             PROJECT("bob")
               intantiate with p = PROJECT(name, Y_factor=None, use_MQTT=False)
               later use start_MQTT(), stop_MQTT(), and
               add THINGs with p.new_CARROT(name) or p.new_ONION(name, n_rings=None)
         """

Then in another module, you can leverage this _mytips() method:

import os

def tips(o):
    try:
        raw_tips = o._mytips()
        # Use standardized indentation
        return (os.linesep + '  ').join([l.strip() for l in raw_tips.strip().splitlines()])
    except AttributeError:
        raise ValueError('object does not support tips')

Then it can be used like this:

>>> tips(PROJECT)
PROJECT("bob")
  intantiate with p = PROJECT(name, Y_factor=None, use_MQTT=False)
  later use start_MQTT(), stop_MQTT(), and
  add THINGs with p.new_CARROT(name) or p.new_ONION(name, n_rings=None)
>>> p = PROJECT()
>>> tips(p)
PROJECT("bob")
  intantiate with p = PROJECT(name, Y_factor=None, use_MQTT=False)
  later use start_MQTT(), stop_MQTT(), and
  add THINGs with p.new_CARROT(name) or p.new_ONION(name, n_rings=None)

Calling the method _mytips gives the tips call a standard method to look for, and prefixing it with an underscore indicates it's not intended to be called directly. Note that you should not use something like __mytips__, as the magic method syntax is reserved for the language or the standard library. Making it static means you can use it on other the class directly or an instance. You still have to write the documentation yourself or generate it somehow. This is not particularly common, but at least this is in line with the standard "protocol pattern" Python uses for things that need to be implemented by lots of different classes/objects. Other devs will recognize what you're doing when they look at the tips code.

I'd still prefer an actual help page open in the browser, though, and don't really recommend it. You might consider loading the tips from an actual external document, giving you both with the same content.

Informational messages

The other thing you seem to want to implement is informational messages. The standard solution for this is logging. That is definitely what you're doing when you print the "added" messages. Using the logging module in those cases would look like this:

def new_carrot(self, name):
    """def new_carrot"""

    carrot = Carrot(project=self, name=name)

    self.things.append(carrot)
    logging.debug("a Carrot was added!")
    return carrot

But requires a tiny bit of set up:

import logging

logging.basicConfig(level=logging.DEBUG)

You can get fancier if you want more control over the format, but I'm not going to get into all those details here.

This also gives your users the option of disabling these messages by cranking up the message level:

logging.getLogger().setLevel(logging.INFO)

Other concerns

Resource management

I notice your PROJECT class starts up something in its initializer:

status = self.start_MQTT()

and then has a stop method as well.

The Python standard for managing resources is context managers. Context managers are a fantastic little tool that will make your code much simpler. If you convert PROJECT to one, then callers will have much less boilerplate managing it:

with PROJECT("myproj", use_MQTT=True) as p:
    p.new_CARROT("carrot 1")

Yes, that block of code is correct. There is no need to call stop_MQTT, and that's because the with block invokes the method to release it automatically:

class PROJECT(object):
    def __init__(self, name, Y_factor=None, use_MQTT=False):
        # Everything EXCEPT starting the server

    def __enter__(self):
        if self.use_MQTT:
            # do stuff
            status = 'started'
            self.client = 'client instance'
            logging.debug("MQTT status: {}".format(status))

    def __exit__(self, type, value, traceback):
        if self.status == 'started':
            # do stuff to stop server
            status = 'stopped'
        return status

    # Other methods

Some details may not exactly line up with what you need to do, but you have pseudocode here anyway. Much simpler for your callers.

Use super

You have this call:

THING.__init__(self, project, name)

Don't do that. Use super:

super(ONION, self).__init__(project, name)

Naming and formatting standards

Please read PEP8 for naming conventions. Right off the bat, I notice that PROJECT would be Project and Y_factor should be yfactor. (One might argue that y_factor is also acceptable, but PEP8 notes that underscores should be used "to improve readability". Putting it after just a y doesn't really add any readability.)

Conclusion

The bottom line here is that Python has a lot of norms and standards, and your code doesn't conform to them. This isn't just a matter of being pedantic. Python's norms are built around practical concerns and avoiding real world problems. When you use Python the way it was intended to be used, it will make solving your problems easier.

If you want a sort of intro to the idea of writing "Pythonic" code and why it's important and a practical example, check out Raymond Hettinger's Beyond PEP8.

\$\endgroup\$
  • 1
    \$\begingroup\$ +n! This is an excellent answer and what I needed, thank you! You are right about logger, elsewhere I've echoed the logged events to the screen with a second handler using logging.getLogger('').addHandler(console) and that may be what I need here. I've used context manager for the threading lock (and files) but never realized it could be used this way, nice! I have the manual stop method for other reasons (example: if you turn off WiFi to save batteries, then MQTT's looping/pinging starts complaining) but for final shut down, what you've shown is great. \$\endgroup\$ – uhoh Jan 3 at 6:09
  • \$\begingroup\$ Super looks super, I will dig in and read about that now. Again, thank you for taking the time for a thorough review! \$\endgroup\$ – uhoh Jan 3 at 6:11
  • 1
    \$\begingroup\$ @uhoh I added some notes about dir and vars you may find useful. \$\endgroup\$ – jpmc26 Jan 3 at 23:40
  • \$\begingroup\$ yep, thanks. Maybe what I'm really looking for is something that could look like and work like a Built-in function, and be called pretty_help(class), concise_help(class) or even local_hints(class). Something that returns a few concise lines of main methods, hints, or other things that are what users are most likely to need, and because of compactness still leave your current train of thought on the screen, even if the screen is small. I'm a heavy python user but will never be, or think like you steel-trap-minded developers, and there are a lot of us who could use some simple hints. \$\endgroup\$ – uhoh Jan 4 at 3:03
  • 1
    \$\begingroup\$ @uhoh I'm not really opposed to the idea; I can see the usefulness.Jupyter/IPython do something similar, after all. It's just that such a thing does not exist out of the box, and hijacking functionality that's supposed to do something else will hurt you more that it will help in the long run. I read documentation all day long, and I find myself using help pages way more than any kind of command line help. That said, I've made one more addition: designing your own protocol method. It's not my preference and I don't really encourage it, but at least it doesn't conflict with existing things. \$\endgroup\$ – jpmc26 Jan 4 at 4:25
8
\$\begingroup\$

Disclaimer

Now that the OP has clarified that they intend to use this as a teaching tool, my answer has lost relevance for their use-case. It can still be taken as general advice for most Python modules.

I do agree with @jpmc26 to a certain extent, specifically that this code itself as currently written should not be used as an example for new learners of Python. I am certainly not advocating teaching students bad Python style. But I think that this could be a useful learning aid within the context the OP provides.

Original answer

I realize this is one of the central premises of your program, but printing help docstrings during normal use of the program is highly unusual behavior and likely to be annoying to users of your program. For example, a user might already know how to use your API, and doesn't need to be told how every time they invoke it; it would just clutter up the command line. Besides, you're solving a problem that doesn't really need to be solved: a class docstring may scroll the screen as described, but function and method docstrings do not, and the Python REPL interpreter already allows you to short circuit the scrolling.

Instead, I would recommend using more descriptive and helpful docstrings. For example, informing the user of the class calling conventions in isolation is not useful; explaining what the class's intended use is, and what the various parameters mean would be more useful. PEP 257 provides guidance on how to write good docstrings.

\$\endgroup\$
  • \$\begingroup\$ The first sentence of my question makes it clear that this is an exercise that I am leading them through, rather than a module for general release. In this case Python is a tool, but not the topic. Getting short, helpful docstrings will not be annoying to them, but instead helpful. I would have mentioned this right away, but my power went out right after posting (I usually babysit my questions). In any event, I'll take your answer to be that you don't feel there is a better way to do this (type an object and receive hints how to use it). \$\endgroup\$ – uhoh Jan 2 at 23:15
  • \$\begingroup\$ @uhoh My answer was premised on the assumption that this was code intended to be a general Python module for distribution, not a teaching tool. I read the first sentence, but I was genuinely confused as to who the intended audience was. Perhaps adding a sentence or two that provides the additional context, and editing your question title would help you receive answers that are closer to what you're looking for; my answer is by no means a comprehensive analysis of your code. \$\endgroup\$ – Graham Jan 2 at 23:35
  • \$\begingroup\$ It's unfortunate the power went out and I couldn't stay with my post. I've just gotten home now after spending the night in a McDonalds writing with a pen in a notebook (it's been a while, surprised I remembered how!). I'll update the question in a few hours. Until then let's assume that the behavior indicated really is the behavior needed, and I'm checking if there is a simpler way to do this, rather than asking "is this a good idea". Thanks! \$\endgroup\$ – uhoh Jan 2 at 23:41
  • \$\begingroup\$ "Now that the OP has clarified that they intend to use this as a teaching tool" NO! It is vital that education conforms to the norms of the language and teaches students to adhere to them. The students will be far better served by the teacher using the correct, existing technology for the task they are trying to perform. \$\endgroup\$ – jpmc26 Jan 2 at 23:46
  • \$\begingroup\$ @jpmc26 I think it depends on what level of education the students are at, and the general context. If the OP wants to show their students how to instantiate objects and how objects work in Python, then this might be a useful tool. But, there are probably better tools. And I do agree that if this code is used, the underlying code should not be shown to students as an example. \$\endgroup\$ – Graham Jan 2 at 23:53
3
\$\begingroup\$

It is difficult to add anything to the existing great and detailed answers, but, to follow up your comment that .__doc__ might be to difficult to remember for newcomers, you could then wrap it around your custom function:

def doc(obj):
    """Prints a docstring of a given object."""
    print(obj.__doc__)

Usage:

In [2]: class CARROT:
   ...:     """  instantiate with c = CARROT(project, name)
   ...:   change thread with c.change_thread_value(new_value)"""
   ...:     

In [3]: carrot = CARROT()

In [4]: doc(carrot)
  instantiate with c = CARROT(project, name)
  change thread with c.change_thread_value(new_value)

Jupyter Notebooks?

As a side note, you may switch to using Jupyter notebooks that may actually be a good learning environment for newcomers and would let them use the standard built-in help() in the Jupyter cells.

There are also built-in shortcuts, like shift+TAB to access a help popup:

enter image description here

\$\endgroup\$
  • 1
    \$\begingroup\$ I'd forgotten about Jupyter, thanks! You might want to add a "me too" answer there (or not). I won't have control over what people use, and this question is for those who live 100% inside a terminal. "They may do this from a terminal which does not show the docstring balloons the way IDLE and many other IDEs can. So instead I've found a way..." \$\endgroup\$ – uhoh Jan 3 at 6:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.