3
\$\begingroup\$

I'm refactoring a monster 176-line function into something more sensible and more testable. The function as it stands fails the 'one thing and thing well' test by doing many things:

  • Indexing through a data structure
  • Compiling a str that represents state
  • Defining a file path to write that string into
  • Checking if the file path already exists and backing it up if so
  • Writing the string to filepath

I'm influenced by Gary Bernhardt's talk about FunctionalCore, ImperativeShell and recognise that the first 3 activities above could be functionalCore, and the last two having I/O have to be out of that core. I'm also exploring Python coroutines so of course I've digested David Beasley's talks.

I have mocked up activity_1, 2, 3 with the function below. The root_fpath is set to my home path for testing.

import random
def write_files(writer=write_w()):
'''mock source generator
files `file_{10..19}` have contents `content_{10..19}`; At random that
content is substituted by content at random selected from {10..19}; this
sporadic random substitution mocks existing file with
different content (i.e. that the content of file_x has changed)'''
    for i in xrange(10,20):
        fpath = '{}/file_0{}'.format(root_fpath, i)
        content = 'content_0{}'.format(i)
        print (fpath, content)
        if random.randint(0,1):
            content = 'content_0{}'.format(random.randint(10,19))
            print('random content:',content)
            writer.send((fpath, content))                         
        else:                                  
            writer.send((fpath, content))
    writer.close()

This function uses the writer.send() method to send filepath and content to the co-routine writer, write_w, defined and initialised by the functions below:

def coroutine(f):
'''initializes f as coroutine by calling f.send(None)'''
    def init(*args, **kwargs):
        cor = f(*args, **kwargs)
        cor.send(None)
        return cor
    return init

@coroutine
def write_w():
'''fpath content writer
'''
    while True:
        fpath, content = (yield)
        with open(fpath, 'w') as wFH:
            wFH.write(content)

So far these do activities 1,2,3,5; the backup of any existing file is done using the functions below:

def isnew_fpathcontent(fpath,content):
'''determines if fpath:content combination is new'''
    if not os.path.isfile(fpath):
        return True
    with open(fpath) as rFH:
        redlines = rFH.readlines()
        if not redlines == [content]:                                                 
           print('existing content:', redlines)
           return True
        else:                                                                                
            print('existing content:', redlines)
            print('not new fpathcontent') #tmp#
            return False

@coroutine
def update(writer=None):
'''writer wrapper forwarding new fpath:content combinations'''
    while True:
        fpath, content = (yield)
        if isnew_fpathcontent(fpath, content):
            print('new fpathcontent')   #tmp#
            # need a BU call here
            writer.send((fpath, content))

These functions can be called to do all activities 1..5 using write_files(writer=update(writer=write_w())).

The outcome is -- despite the developmental bloat from a lot of print stmts -- way shorter than the original, has nicely separated the original concerns into individual functions, which will be (I haven't written tests yet; too early in this learning process for TDD) way more testable than the original.

And I have some questions:

  1. Does this look sensible?
  2. Is my writer.close() call at the and of the write_files() mock source adequate? Have I missed something co-routine~y which will prove a liability? (I'm deep in learning process here)
  3. Is there a recommended docstring practice for co-routine functions? Google has not yielded anything very useful. How to document e.g. co-routine write_w() or update()? Their interfaces suggest no arguments; it gets these by the yield expression; is there an accepted docstring terminology to use? (params: None at call-time; receives (fpath,content) via yield?) And for the return from update()? `returns: (fpath,content); tuple, sent to writer co-routine?
\$\endgroup\$
  • \$\begingroup\$ Just to make sure: does this work as intended? \$\endgroup\$ – Mast Nov 5 '17 at 20:51
  • \$\begingroup\$ it does indeed work as intended \$\endgroup\$ – iusctx Nov 5 '17 at 20:57
  • 3
    \$\begingroup\$ Please ensure that the code you posted is indented as intended — particularly the docstrings. \$\endgroup\$ – 200_success Nov 6 '17 at 6:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.