1
\$\begingroup\$

I often have a need to log my progress when doing a repetitive task. Logging allows me to continue where I left off if execution is halted (for example, perhaps my VPS server was rebooted by my host).

My current working solution is a bit ad hock. Any input on improving this would be appreciated:

class RepeatingTask(object):
    def __init__(self, first, logfile):
        self.current = first
        self.logfile = logfile

    def page(self):
        self.process_request()
        self.increment()
        self.log_progress()

    def process_request(self):
        # do processing here... 
        pass

    def increment(self):
        # set self.current = next item for processing 
        # when items are exhausted, set self.current=None
        pass

    def log_progress(self):
        if self.current is None:
            os.remove(self.logfile)
        else:
            with open(self.logfile, 'w') as logfile:
                logfile.write(json.dumps([["current","key-xyz"],
                                          [self.current,self.val_abc]]))

    def resume(self):
        try:
            with open(self.logfile, 'r') as logfile:
                content = logfile.read()
        except IOError:
            return False
        content = json.loads(content)
        content = dict(zip(content[0],content[1]))
        self.current = content['current']
        self.val_abc = content['key-xyz']
        return True


if __name__ == "__main__":
    task_object = RepeatingTask("first_item_to_process","/path/to/logfile")
    if task_object.resume():
        print "Resuming progress, current:", task_object.current
    else:
        print "Starting freash, current:", task_object.current
    while task_object.current is not None:
        task_object.page()
        # do some work
        print "Next:", task_object.current
\$\endgroup\$
  • \$\begingroup\$ Using docstrings (or more accurately, trying to use them but failing to explain what your code is trying to do) would probably make it more obvious why your approach is bad. "If the implementation is hard to explain, it's a bad idea" — PEP20 \$\endgroup\$ – msw Aug 6 '14 at 21:45
2
\$\begingroup\$

You may want to consider using the standard Python logging module to log in the information, and just keep the parser.

import logging

logger = logging.getLogger('repeatingApp')
hdlr = logging.FileHandler('/path/to/logfile')
hdlr.setFormatter(logging.Formatter('%(message)s'))
logger.addHandler(hdlr) 
logger.setLevel(logging.WARNING)

class RepeatingTask(object):
    def page(self):
        self.process_request()
        self.increment()
        logger.info(json.dumps([["current","key-xyz"],
                                [self.current,self.val_abc]]))
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.