2
\$\begingroup\$

I have coded multiple variations on this and would like to make it generic, to avoid coding any more variations.

Comments and suggested improvements, please, (typos also), and what about that ToDo ?

I often want to parse a text file and split it into chunks, generally those lines between two lines containing known keywords.

E.g.

  • zero or more lines of text
  • A start
  • zero or more lines of text -A end
  • zero or more lines of text
  • B start
  • zero or more lines of text
  • B end
  • zero or more lines of text
  • B start
  • zero or more lines of text
  • C end
  • zero or more lines of text

Note that “zero or more”.

Also, note that some sections are optional. For instance, knowing that there are no lines between sections, to get the A related text, I can specify a start as any line containing “A start” and an end as any line containing “A end”. BUT, since B and C are optional, my end for A is any one of “A end”, “B end” or “C end”.

And sometimes I know that I am on the start line (it may not even contain a keyword).

More detail in the header comment (where it belongs). Please help me to improve this code.

# +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
# Given the lines and current line number, this function reads
# from the current line (or the first following line containing the start string,
# returning empty string if not found) and returns all lines between
# the line after that and the first line containing any of the end strings
# (or end of lines) - not including the line with the terminating string.
#
# @param [in] lines                 - list of strings
#
# @param [in] lineNumber            - current index to lines parameter
#
# @param [in] searcrhForStartString - Bool; set to False if currently positioned
#                                     at the start line, and there is no start
#                                     string to search for
#
# @param [in] startStrings          - list of strings (even if there is only one)
#                                     If param searcrhForStartString == True
#                                     then search param lines until one contains
#                                     one of these lines. Returned lines will start
#                                     at the next line, skipping the keyword
#                                     (start at current line if param
#                                      searcrhForStartString is False).
#
#                                     Will return an empty list if no match is found
# @param [in] endStrings            - a list (even if there is only one) of strings.
#                                     Search until a line containing one of these is found
#                                     (or end of lines) and return all lines
#                                     from the first until the line prior to this
#                                     (do not include keyword line)
#
# @param [in] errorOnEndOfLines     - if True, will return an empty list
#                                     if no end string is found
#                                     before the end of param lines
# @return                           - list of strings
#
# ToDo: consider (a) parameter(s) to indicate that lines must start with,
#       or be equal to, a search string, rather than just containing?
#       That would make this even more generic, but would require (an) extra parameter(s)
#
def GetLinesBetween(lines,lineNumber, searcrhForStartString, startStrings, endStrings, errorOnEndOfLines):
    try:
        result = []     # emmpty list
        line = lines[lineNumber]

        if searcrhForStartString:
            while not any(keyword in line for keyword in startStrings):
                lineNumber += 1
                if lineNumber == len(lines):
                    if errorOnEndOfLines:
                        result.clear()
                    return  result
                line = lines[lineNumber]

            #serach string matched, if requested, so advance one line to skip it
            lineNumber += 1
            line = lines[lineNumber]

        # Line number now indexes the first line of what we want to return
        while lineNumber < len(lines):
            if any(keyword in line for keyword in endStrings):
                return result

            result.append(line)
            lineNumber += 1
            if not lineNumber == len(lines):
                line = lines[lineNumber]

        if lineNumber == len(lines):
            if errorOnEndOfLines:
                result.clear()
            return result

    #-----------------------
    except Exception as err:
        print('Exception:')
        exc_type, exc_value, exc_tb = sys.exc_info()
        traceback.print_exception(exc_type, exc_value, exc_tb)
        sys.exit(0)
\$\endgroup\$
  • 4
    \$\begingroup\$ The function specification itself is rather complicated, and could probably be improved, if you included the code that calls this function. Including some sample inputs and outputs would help clarify the expected behavior as well. \$\endgroup\$ – 200_success Feb 20 at 10:29
2
\$\begingroup\$

This is some interesting code, thanks for posting it here. Good work Mawg.

Nitpicking

# +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
# Given the lines and current line number, this function reads
# from the current line (or the first following line containing the start string,

Please get rid of banners like this. If you are using an IDE such as PyCharm it can generate documentation comments for you. Current python industry standard regarding documentation is PEP-287.

  1. PEP-287
  2. Related SO Question
lines,lineNumber,

Use consistent spacing. I personally like to use black to format my code. But you can use anything that formats to PEP8.

GetLinesBetween, lineNumber, ...

Please use correct PEP8 style such as get_lines_between and line_number?

#-----------------------
except Exception as err:
    print('Exception:')
    exc_type, exc_value, exc_tb = sys.exc_info()

This is sloppy exception handling.

  • If you must use a catch-all: Maybe use a higher level method that captures exceptions and change this to try_get_lines_between or _get_lines_between as a more readable alternative. Why: separation of error handling and application logic - It is up to you to decide if this is best course of action.
  • You can also create your own exceptions if that is more suitable to you. You can for example raise them instead of returning empty list if it is more suitable.
  • Use a logger instead of print to log exceptions. logger.exception is more suitable for this.
  • Instead of sys.exit(0) - maybe exit with non zero value so you know that there was an error.

emmpty, serach

Please use a spell checker plugin in your IDE.

# ToDo: consider (a) parameter(s) to indicate that lines must start with,
#       or be equal to, a search string, rather than just containing?
#       That would make this even more generic, but would require (an) extra parameter(s)
#

People sometimes forget to do to-dos. Use a software like JIRA, Github Projects to track your tasks and remove them from code. It's OK to create tasks for yourself.

How would I design this better?

  • Use generators instead of creating lists, so you can create pipeline of functions and can even read multiple files, and also save some memory.
  • Maybe use Aho-Corasick algorithm (Trie based) to match against multiple strings.
  • Sensible defaults - there are lot of parameters for this function. If you add default values then API would be easier to use.
\$\endgroup\$
  • \$\begingroup\$ Thanks very much for the feedback. 1) "generate documentation comments for you" - wilco 2) "Use consistent spacing" - I thought that I did, but will check 3) "use correct PEP8 style" for variable/function names - alas, several decades of embedded C programming have ingrained this, so I am unlikely to change :-( 4) "This is sloppy exception handling ... •Use a higher level methods that captures exceptions and a try_get_lines_between method" - I don't see what that would buy me, just shoving it upwards into a wrapper; what am I missing? ---> \$\endgroup\$ – Mawg Feb 20 at 14:46
  • \$\begingroup\$ ---> 5) "Use a logger instead of print to log exceptions" - sounds good. Thanks for all of your feedback. Btw, 6) what do you think of the ToDo? and 7) the function does not have to return lines as it never changes it (I just did my own review :-) \$\endgroup\$ – Mawg Feb 20 at 14:47
  • 1
    \$\begingroup\$ @Mawg there are various ways to do your todo. You can for example pass lambda functions as arguments (default to in), pass lambda line, keyword: line.startswith(keyword) when you want to handle starts with. Flag functions however become more complex overtime so be warned. \$\endgroup\$ – 422_unprocessable_entity Feb 20 at 16:11
  • 1
    \$\begingroup\$ I've added more points to answer as well. \$\endgroup\$ – 422_unprocessable_entity Feb 20 at 16:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.