3
\$\begingroup\$

I have various texts that are inconsistent in how paragraphs are formatted. Some texts use the <P> </P> tags to specify a paragraph, while others use two break tags (<br><br>). Sometimes the user forgets to close a <p> tag too. Also, a tag could contain whitespace. I need to "normalise" these texts so that they all specify paragraphs in the same way. Below is some Python code I wrote to achieve this goal. The code scans the input for the start or end of a paragraph and puts each paragraph it finds into an array, which I can then output in whatever format the business wants. I wrote it in Python, but this is not the language the actual solution will be implemented in. The language is so obscure and old that there is no point in posting it here. This language doesn't even have regular expressions, which is why I do not use regular expressions in the Python code. The plan is to translate this python code into the obscure old language we use at work. The array PARAGRAPH_DELIMITERS contains all of the strings that specify the the start or end of a paragraph.

PARAGRAPH_DELIMITERS = ["<p>","</p>","<br><br>","<br></br><br></br>", "<br/><br/>"]

class Input:
    def __init__ (self,html):
        self.pos = 0
        self.html = html
        self.length = len(html)

    def get_next_char(self):
        return self.html[self.pos]
    
 
def get_paragraphs(input):
    paragraphs = []
    paragraph = ""
    while(input.pos < input.length):
        if new_paragraph(input):
            # We found a new paragraph, add it to the list of paragraphs.
            if paragraph != "":
                paragraphs.append(paragraph)
                paragraph = ""
        else:
            paragraph += input.get_next_char()
            input.pos += 1
     
    if paragraph != "":
         # We found a new paragraph, add it to the list of paragraphs.
        paragraphs.append(paragraph)

    return paragraphs

def new_paragraph(input):
    if input.get_next_char() != "<":
        return False

    lookahead_pos = input.pos
    new_paragraph_found = False
    potential_paragraph_delimiter = ""
    while(lookahead_pos < input.length):
        if input.html[lookahead_pos] == " ":
             # Skip whitespace.
            lookahead_pos += 1
        else:
            potential_paragraph_delimiter += input.html[lookahead_pos].lower()
            match_possible = False
            for i in range(len(PARAGRAPH_DELIMITERS)):
                if PARAGRAPH_DELIMITERS[i].startswith(potential_paragraph_delimiter):
                    match_possible = True
                    if len(potential_paragraph_delimiter) == len(PARAGRAPH_DELIMITERS[i]):
                        new_paragraph_found = True
            if not match_possible:
                break;
            if new_paragraph_found:
                break;
            lookahead_pos += 1

    if new_paragraph_found:
        input.pos = lookahead_pos + 1
    return new_paragraph_found



html = "<p>hello this is paragraph 1<P>hello this is paragraph 2</p><p    >hellow this is paragraph 3<br><br>this is paragraph 4"
input = Input(html)
paragraphs = get_paragraphs(input)
print(paragraphs)
\$\endgroup\$
5
  • \$\begingroup\$ If you are planning on writing the algorithm in another language, it might be better if you post your code in that language. Using python you could fairly easily solve the problem you have given (besides using regex), using str.split (based on the sample data you have provided). And this functionality might or might not be available in the language you are planning to use to write the algorithm \$\endgroup\$ Jun 20, 2021 at 8:44
  • \$\begingroup\$ str.split won't work because the paragraph delimiters could contain spaces. Am I mistaken? Posting the other language is pointless as it's used by fewer than 100 people worldwide. I'm more interested in the algorithm than python best practices. \$\endgroup\$ Jun 20, 2021 at 8:45
  • \$\begingroup\$ If delimiters can contain whitespace, I would encourage you to add that wrinkle to the provided example html. It's a surprising and crucial fact: you should emphasize it, and the best way to do that is by embedding it in the test case everyone will use. \$\endgroup\$
    – FMc
    Jun 20, 2021 at 23:40
  • \$\begingroup\$ Also, when you say whitespace is allowed in delimiters, do you mean it fully: for example, what about < b r >< b r >? \$\endgroup\$
    – FMc
    Jun 20, 2021 at 23:45
  • \$\begingroup\$ Yes, a whitespace can occur anywhere in a delimiter. The users who create these texts don't really know what they are doing and they enter all sorts of rubbish. \$\endgroup\$ Jun 20, 2021 at 23:52

1 Answer 1

1
\$\begingroup\$

You seem to be interested in algorithmic alternatives more than a typical Python best-practices review. Here's one approach that seems simpler to me: just one function, and no need for an Input class wrapping the input text. Like your solution, this one does build up the delimiter character-by-character, for the reason you articulate in your comments.

def get_paragraphs(text, delimiters):
    # Positions (current and limit), paragraphs (current and all),
    # and delimiters (current target and viable candidates).
    i = 0
    limit = len(text)
    para = ''
    paragraphs = []
    target = ''
    candidates = list(delimiters)
    while i < limit:
        # Get character and just continue on spaces.
        c = text[i]
        i += 1
        if c == ' ':
            para += c
            continue
        # Add non-space to delimiter-target, counting remaining candidates:
        # - Exact match: reset to bypass the matched delimiter.
        # - Opening angle and no candidates: reset target.
        # - Otherwise, just add character to current paragraph.
        target += c.lower()
        candidates = [d for d in candidates if d.startswith(target)]
        n = len(candidates)
        if n == 1 and candidates[0] == target:
            para = ''
            target = ''
            candidates = list(delimiters)
        elif n == 0 and c == '<':
            paragraphs.append(para)
            para = c
            target = c
            candidates = list(delimiters)
        else:
            para += c
    if para:
        paragraphs.append(para)
    return paragraphs
\$\endgroup\$
2
  • \$\begingroup\$ Thank you for your comments. I think you are overlooking something though. The reason I go character by character on the current delimiter is that the current delimiter could contain whitespace. Your solution would fail if the delimiter looked liked this <{white space}p> \$\endgroup\$ Jun 20, 2021 at 23:06
  • \$\begingroup\$ @user1235598399 Revised in light of my clearer understanding of the situation. \$\endgroup\$
    – FMc
    Jun 21, 2021 at 20:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.