Python code to get all lines between two keyword lines in a file (with exceptions)

Question

I have coded multiple variations on this and would like to make it generic, to avoid coding any more variations.

Comments and suggested improvements, please, (typos also), and what about that ToDo ?

I often want to parse a text file and split it into chunks, generally those lines between two lines containing known keywords.

E.g.

• zero or more lines of text
• A start
• zero or more lines of text
• A end
• zero or more lines of text
• B start
• zero or more lines of text
• B end
• zero or more lines of text
• C start
• zero or more lines of text
• C end
• zero or more lines of text

Note that “zero or more”.

Also, note that some sections are optional. For instance, knowing that there are no lines between sections, to get the A related text, I can specify a start as any line containing “A start” and an end as any line containing “A end”. BUT, since B and C are optional, my end for A is any one of “A end”, “B end” or “C end”.

And sometimes I know that I am on the start line (it may not even contain a keyword).

More detail in the header comment (where it belongs). Please help me to improve this code.

# +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
# Given the lines and current line number, this function reads
# from the current line (or the first following line containing the start string,
# returning empty string if not found) and returns all lines between
# the line after that and the first line containing any of the end strings
# (or end of lines) - not including the line with the terminating string.
#
# @param [in] lines                 - list of strings
#
# @param [in] lineNumber            - current index to lines parameter
#
# @param [in] searcrhForStartString - Bool; set to False if currently positioned
#                                     at the start line, and there is no start
#                                     string to search for
#
# @param [in] startStrings          - list of strings (even if there is only one)
#                                     If param searcrhForStartString == True
#                                     then search param lines until one contains
#                                     one of these lines. Returned lines will start
#                                     at the next line, skipping the keyword
#                                     (start at current line if param
#                                      searcrhForStartString is False).
#
#                                     Will return an empty list if no match is found
# @param [in] endStrings            - a list (even if there is only one) of strings.
#                                     Search until a line containing one of these is found
#                                     (or end of lines) and return all lines
#                                     from the first until the line prior to this
#                                     (do not include keyword line)
#
# @param [in] errorOnEndOfLines     - if True, will return an empty list
#                                     if no end string is found
#                                     before the end of param lines
# @return                           - list of strings
#
# ToDo: consider (a) parameter(s) to indicate that lines must start with,
#       or be equal to, a search string, rather than just containing?
#       That would make this even more generic, but would require (an) extra parameter(s)
#
def GetLinesBetween(lines,lineNumber, searcrhForStartString, startStrings, endStrings, errorOnEndOfLines):
    try:
        result = []     # emmpty list
        line = lines[lineNumber]

        if searcrhForStartString:
            while not any(keyword in line for keyword in startStrings):
                lineNumber += 1
                if lineNumber == len(lines):
                    if errorOnEndOfLines:
                        result.clear()
                    return  result
                line = lines[lineNumber]

            #serach string matched, if requested, so advance one line to skip it
            lineNumber += 1
            line = lines[lineNumber]

        # Line number now indexes the first line of what we want to return
        while lineNumber < len(lines):
            if any(keyword in line for keyword in endStrings):
                return result

            result.append(line)
            lineNumber += 1
            if not lineNumber == len(lines):
                line = lines[lineNumber]

        if lineNumber == len(lines):
            if errorOnEndOfLines:
                result.clear()
            return result

    #-----------------------
    except Exception as err:
        print('Exception:')
        exc_type, exc_value, exc_tb = sys.exc_info()
        traceback.print_exception(exc_type, exc_value, exc_tb)
        sys.exit(0)

Answer 1

This is some interesting code, thanks for posting it here. Good work Mawg.

Nitpicking

# +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
# Given the lines and current line number, this function reads
# from the current line (or the first following line containing the start string,

Please get rid of banners like this. If you are using an IDE such as PyCharm it can generate documentation comments for you. Current python industry standard regarding documentation is PEP-287.

1. PEP-287
2. Related SO Question

lines,lineNumber,

Use consistent spacing. I personally like to use black to format my code. But you can use anything that formats to PEP8.

GetLinesBetween, lineNumber, ...

Please use correct PEP8 style such as get_lines_between and line_number?

#-----------------------
except Exception as err:
    print('Exception:')
    exc_type, exc_value, exc_tb = sys.exc_info()

This is sloppy exception handling.

• If you must use a catch-all: Maybe use a higher level method that captures exceptions and change this to try_get_lines_between or _get_lines_between as a more readable alternative. Why: separation of error handling and application logic - It is up to you to decide if this is best course of action.
• You can also create your own exceptions if that is more suitable to you. You can for example raise them instead of returning empty list if it is more suitable.
• Use a logger instead of print to log exceptions. logger.exception is more suitable for this.
• Instead of sys.exit(0) - maybe exit with non zero value so you know that there was an error.

emmpty, serach

Please use a spell checker plugin in your IDE.

# ToDo: consider (a) parameter(s) to indicate that lines must start with,
#       or be equal to, a search string, rather than just containing?
#       That would make this even more generic, but would require (an) extra parameter(s)
#

People sometimes forget to do to-dos. Use a software like JIRA, Github Projects to track your tasks and remove them from code. It's OK to create tasks for yourself.

How would I design this better?

• Use generators instead of creating lists, so you can create pipeline of functions and can even read multiple files, and also save some memory.
• Maybe use Aho-Corasick algorithm (Trie based) to match against multiple strings.
• Sensible defaults - there are lot of parameters for this function. If you add default values then API would be easier to use.