2
\$\begingroup\$
from itertools import cycle

# Integers represent Chinese characters
main_text = "234458674935346547521456767378878473562561423"
text_with_commentary = "2344586749215432678652353465475214561655413164653413216576737887847356152352561423"

iter_main = iter(main_text)
iter_comm = iter(text_with_commentary)

marked_up = []
x = next(iter_main)
y = next(iter_comm)
i = 0
j = 0

def commentary_start():
    global x, y, i, j, marked_up

    while x == y and j < len(text_with_commentary):
        try:
            marked_up.append(y)
            x = next(iter_main)
            y = next(iter_comm)
            i += 1
            j += 1
        except: return

    marked_up.append("{{")
            
    return marked_up

def commentary_end():
    global x, y, i, j, marked_up

    while x != y:
        marked_up.append(y)
        y = next(iter_comm)
        j += 1
        
    if main_text[i+1] == text_with_commentary[j+1] and \
    main_text[i+2] == text_with_commentary[j+2]:
        marked_up.append("}}")
        
    else: 
        marked_up.append(y)
        y = next(iter_comm)
        j += 1
        commentary_end()
    
    return marked_up

def mark_up_commentary():
    global marked_up
    
    while j < len(text_with_commentary):
        try:
            commentary_start()
            commentary_end()
        except:
            break
            
    marked_up = "".join(marked_up)
    
    return marked_up



mark_up_commentary()

# Expected result: "2344586749{{215432678652}}35346547521456{{16554131646534132165}}76737887847356{{15235}}2561423"

The above code compares two sets of strings, character by character, to markup commentary text that is interspersed between the main text.

The code makes use of the iterators iter_main and inter_comm to check if the next character in the commented text is the same as the next character of the main text, adding double braces to mark up commentary text that deviates from the main text at the relevant positions.

I would like to seek help to improve the code in these respects:

  1. See if the code can be made more concise and efficient.
  2. The current method is actually quite hard-coded and mechanical, which makes little room for minor deviations in the texts, which is actually unavoidable in the real world. I would hence like to know whether fuzzy text-comparison tools that make use of concepts such as Levenshtein distance can be implemented to make the code more robust and flexible.
  3. Are there good libraries to recommend for the task?
\$\endgroup\$
3
\$\begingroup\$

Are there good libraries to recommend for the task?

Yes, and it's built-in: read about difflib.

I would hence like to know whether fuzzy text-comparison tools [...] can be implemented

Off-topic for CodeReview.

Avoid globals, use difflib, and this becomes

from difflib import SequenceMatcher
from typing import Iterable


def diff(main_text: str, text_with_commentary: str) -> Iterable[str]:
    matcher = SequenceMatcher(a=main_text, b=text_with_commentary, autojunk=False)
    for op, a0, a1, b0, b1 in matcher.get_opcodes():
        right = text_with_commentary[b0: b1]
        if op == 'equal':
            yield right
        elif op == 'insert':
            yield '{' + right + '}'
        else:
            raise ValueError(f"{op} indicates malformed text")


def test():
    actual = ''.join(
        diff(
            main_text=(
                "2344586749"
                "35346547521456"
                "76737887847356"
                "2561423"
            ),
            text_with_commentary=(
                "2344586749"
                "215432678652"
                "35346547521456"
                "16554131646534132165"
                "76737887847356"
                "15235"
                "2561423"
            ),
        )
    )

    expected = (
        "2344586749"
        "{215432678652}"
        "35346547521456"
        "{16554131646534132165}"
        "76737887847356"
        "{15235}"
        "2561423"
    )
    assert actual == expected
\$\endgroup\$
8
  • \$\begingroup\$ Could you explain the logic of the code to me? This is the first time I've come across a generator function. Seems like I am supposed to use it with ''.join([i for i in diff(main_text, text_with_commentary)]). \$\endgroup\$
    – Sati
    May 31 at 14:17
  • 1
    \$\begingroup\$ Close, but it would just be ''.join(diff(....)). It's just yielding strings that the user can choose to do with it what they like - join, or otherwise. \$\endgroup\$
    – Reinderien
    May 31 at 14:24
  • \$\begingroup\$ What does Iterable[str] at the function annotation do? Is it there just to indicate that this function returns an iterable string? \$\endgroup\$
    – Sati
    May 31 at 14:51
  • \$\begingroup\$ An iterable of strings - in other words, if you for x in diff(), x will be a string on each iteration. \$\endgroup\$
    – Reinderien
    May 31 at 14:55
  • 1
    \$\begingroup\$ SequenceMatcher(None, a, b) would return a tuple that describes how string a transcribes to string b. op is the tag, while a0, a1 and b0, b1 are the position indicators of the stretch of matching or mismatching strings between a and b. Is this understanding correct? \$\endgroup\$
    – Sati
    Jun 1 at 4:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.