0
\$\begingroup\$

I would like to learn Python packaging and testing best practices.

I wanted to gain experience with packaging and testing, and also play around with context managers, so I made a contextmanager that can indent stdout with minimal work on the callee's part. E.g.,

>>> print('hello')
hello
>>> with Indent():
...     print('hello')
    hello

I've structured my source like this:

.
├── README.md
├── setup.py
├── src
│   └── indent
│       └── __init__.py
└── tests
    ├── __init__.py
    └── test_indent.py

README.md can be empty, my setup.py looks like

from setuptools import setup, find_packages

with open('README.md', 'r') as readme:
    long_description = readme.read()

setup(
    name='indent',
    version='1.0.0',
    package_dir={'': 'src'},
    packages=find_packages(where='src'),
    description='Tools for indenting text',
    long_description=long_description,
    long_description_content_type='text/markdown',
    url='https://github.com/myarcana/indent.py/',
    classifiers=[
        'Development Status :: 5 - Production/Stable',
        'Programming Language :: Python :: 3 :: Only'
    ],
    python_requires='>=3.7, <4',
    project_urls={
        'Bug Reports': 'https://github.com/myarcana/indent.py/issues',
        'Source': 'https://github.com/myarcana/indent.py/'
    }
)

tests/__init__.py is empty.

The library I've written is in src/indent/__init__.py:

import re
import sys

from io import TextIOBase
from types import SimpleNamespace


class LastCharTracker:
    r'''Wraps a text stream to track the last character written to the stream'''
    def __init__(self, stream: TextIOBase):
        if stream.readable(): # try to find the last character that has already been written
            try:
                stream_position = stream.tell()
                stream.seek(stream_position - 1)
                self.last_character = stream.read(1)
                stream.seek(stream_position)
            except:
                self.last_character = None
        else:
            self.last_character = None
        for attr in filter(lambda x: not x.startswith('_'), dir(stream)):
            if attr == 'write':
                def wrapped_attr(s: str):
                    try:
                        self.last_character = s[-1]
                    except IndexError:
                        pass
                    return stream.write(s)
            elif attr == 'writelines':
                def wrapped_attr(lines: list[str]):
                    try:
                        lines = [line for line in lines if line]
                        last_line = lines[-1]
                        self.last_character = last_line[-1]
                    except IndexError:
                        pass
                    return stream.writelines(lines)
            else:
                wrapped_attr = getattr(stream, attr)
            setattr(self, attr, wrapped_attr)


class Indent:
    r'''Context manager that indents lines written to sys.stdout

    Constructor arguments:
    :param amount: the width of the indent in spaces
    :param string: the string to indent with. If set, amount should not be set.
    '''
    def __init__(self, amount: int=4, *, string: str=None):
        self.indent_string = string if string is not None else ' ' * amount

    def _write(self, s: str):
        if self._tracked_stdout.last_character in ('\n', None) and s:
            s = self.indent_string + s
        s = re.sub(r'\n(.)', r'\n' + self.indent_string + r'\1', s)
        return self._tracked_stdout.write(s)

    def _writelines(self, lines: list[str]):
        indented_lines = []
        for i, line in enumerate(lines):
            if any(lines[i + 1:]): # more characters to come
                newlines_for_indents = (r'\n', r'\n' + self.indent_string)
            else: # this is the final non-empty line, don't indent after any trailing newline
                newlines_for_indents = (r'\n(.)', r'\n' + self.indent_string + r'\1')
            indented_lines.append(re.sub(*newlines_for_indents, line))
        if self._tracked_stdout.last_character in ('\n', None):
            try:
                i, first__line = next((i, line) for i, line in enumerate(indented_lines) if line)
                indented_lines[i] = self.indent_string + first_line
            except StopIteration:
                pass
        return self._tracked_stdout.writelines(indented_lines)

    def __enter__(self):
        self._tracked_stdout = sys.stdout if isinstance(sys.stdout, LastCharTracker) else LastCharTracker(sys.stdout)
        self._original_stdout = sys.stdout
        indenting_io = SimpleNamespace()
        for attr in filter(lambda x: not x.startswith('_'), dir(self._tracked_stdout)):
            if attr == 'write':
                setattr(indenting_io, attr, self._write)
            elif attr == 'writelines':
                setattr(indenting_io, attr, self._writelines)
            else:
                setattr(indenting_io, attr, getattr(self._tracked_stdout, attr))
        sys.stdout = LastCharTracker(indenting_io)

    def __exit__(self, *args, **kwargs):
        sys.stdout = self._original_stdout


ORIGINAL_STDOUT = sys.stdout
sys.stdout = LastCharTracker(sys.stdout)

And I've manually written tests for it in tests/test_indent.py:

import io
import sys

from contextlib import redirect_stdout
from unittest import TestCase, main

from indent import Indent


def recursively_indent(string: str, levels: int, *args, **kwargs):
    if levels > 0:
        with Indent(*args, **kwargs):
            recursively_indent(string, levels - 1, *args, **kwargs)
    else:
        print(string)


Tree = dict[str, 'Tree']
def print_tree(tree: Tree):
    if not tree:
        return
    markers = [('├── ', '│   '), ('└── ', '    ')]
    children = list(tree.items())
    for key, subtree in children:
        is_last_child = (key, subtree) == children[-1]
        key_indent, subtree_indent = markers[is_last_child]
        with Indent(string=key_indent):
            print(key)
        with Indent(string=subtree_indent):
            print_tree(subtree)


class TestIndent(TestCase):
    def testPrintNoIndent(self):
        output = io.StringIO()
        with redirect_stdout(output):
            print('hello')
        self.assertEqual(output.getvalue(), 'hello\n')

    def testOneLevel(self):
        output = io.StringIO()
        with redirect_stdout(output):
            with Indent():
                print('hello')
        self.assertEqual(output.getvalue(), '    hello\n')

    def testStaticallyNested(self):
        blockstack_limit = 20
        for n in range(blockstack_limit + 1):
            code = ''
            for j in range(n):
                code += ' ' * j + 'with Indent():\n'
            code += ' ' * n + 'print("hello")'
            output = io.StringIO()
            with redirect_stdout(output):
                exec(code)
            self.assertEqual(output.getvalue(), ' ' * 4 * n + 'hello\n')

    def testRecursivelyNested(self):
        levels = -1
        with self.assertRaises(RecursionError):
            while True:
                levels += 1
                output = io.StringIO()
                with redirect_stdout(output):
                    recursively_indent('hello', levels)
                self.assertEqual(output.getvalue(), ' ' * 4 * levels + 'hello\n')
        self.assertTrue(levels > 100) # make sure recursion limit not hit instantly

    def testTrailingNewline(self):
        output = io.StringIO()
        with redirect_stdout(output):
            with Indent():
                print('hello\n')
        self.assertEqual(output.getvalue(), '    hello\n    \n')

    def testLeadingNewline(self):
        output = io.StringIO()
        with redirect_stdout(output):
            with Indent():
                print('\nhello')
        self.assertEqual(output.getvalue(), '    \n    hello\n')

    def testInterspersedNewlines(self):
        output = io.StringIO()
        with redirect_stdout(output):
            with Indent():
                print('one', 't\nwo', 'thr\nee', 'four\n', sep='')
        self.assertEqual(output.getvalue(), '''    onet
    wothr
    eefour
    \n''')

    def testSmartJoinsStream(self):
        output = io.StringIO()
        with redirect_stdout(output):
            print('hello', end='')
            with Indent():
                print('no indent')
                print('indent')
        self.assertEqual(output.getvalue(), 'hellono indent\n    indent\n')

    def testEmptyStringWrites(self):
        output = io.StringIO()
        with redirect_stdout(output):
            with Indent():
                print('', '', '', 'one', '', 't\nwo', 'thr\nee', '', '', 'four\n', '', sep='')
        self.assertEqual(output.getvalue(), '''    onet
    wothr
    eefour
    \n''')
        output = io.StringIO()
        with redirect_stdout(output):
            with Indent():
                print(end='')
                print('', end='')
        self.assertEqual(output.getvalue(), '')
        output = io.StringIO()
        with redirect_stdout(output):
            print('hello', '', sep='', end='')
            with Indent():
                print('', end='')
                print('no indent')
                print('indent')
        self.assertEqual(output.getvalue(), 'hellono indent\n    indent\n')

    def testIndentString(self):
        output = io.StringIO()
        with redirect_stdout(output):
            with Indent(string='> '):
                print()
        self.assertEqual(output.getvalue(), '> \n')

    def testNestedBlockquotes(self):
        blockstack_limit = 20
        for n in range(blockstack_limit + 1):
            code = ''
            for j in range(n):
                code += ' ' * j + 'with Indent(string="> "):\n'
            code += ' ' * n + 'print("hello")'
            output = io.StringIO()
            with redirect_stdout(output):
                exec(code)
            self.assertEqual(output.getvalue(), '> ' * n + 'hello\n')

    def testNestedMarkdown(self):
        output = io.StringIO()
        with redirect_stdout(output):
            with Indent(string='> '):
                print('Blockquoted.')
                print()
                with Indent(string='    '):
                    print('code')
                    print('more code')
                print()
                print('End of blockquote.')
        self.assertEqual(output.getvalue(), '''> Blockquoted.
> ''''''
>     code
>     more code
> ''''''
> End of blockquote.\n''')

    def testPrintsTree(self):
        tree = {'alpha':{}, 'beta': {'beta.alpha':{}, 'beta.beta':{}}, 'charlie': {'charlie.alpha':{}, 'charlie.beta':{'charlie.beta.alpha':{}}, 'charlie.charlie':{}}, 'delta':{}}
        output = io.StringIO()
        with redirect_stdout(output):
            print('.')
            print_tree(tree)
        self.assertEqual(output.getvalue(), '''.
├── alpha
├── beta
│   ├── beta.alpha
│   └── beta.beta
├── charlie
│   ├── charlie.alpha
│   ├── charlie.beta
│   │   └── charlie.beta.alpha
│   └── charlie.charlie
└── delta\n''')


if __name__ == '__main__':
    main()

I made a virtual environment for the development of this package, and from the project root directory I installed it: pip install --editable .. Now I can run the tests, even during development, using python -m unittest discover.

My concerns:

  • As you can see, there is a lot of repeated boilerplate for the sake of redirecting and capturing stdout in test_indent.py.

  • Is my test suite well-written, comprehensive enough, etc.? How should tests be written or generated?

  • Has my package-development workflow been sensible? What could be a better way to develop inside of a package structure?

  • What do you think of import indent reassigning stdout? The point of this, and LastCharTracker in general, is so that

    print('one', end='')
    with Indent():
        print('two')
    

    produces

    onetwo
    

    not

    one    two
    

    I thought it would be more expected that only the beginnings of lines in the stream are indented.

  • What do you think of my API's user-friendliness and behaviour decisions?

  • Creating dummy I/Os using SimpleNamespaces and reflection.

Please give me feedback and critique wherever else could be helpful to me too.

\$\endgroup\$
2
  • 1
    \$\begingroup\$ You've written tests for a hypothetical situation, but have you applied the project to code where you actually required it? \$\endgroup\$
    – Mast
    Nov 21 at 15:27
  • \$\begingroup\$ @Mast only insofar as the tests can be thought of as that project, especially e.g. the print_tree function. I've not used this library for anything "real-world". \$\endgroup\$ Nov 21 at 16:01

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy