Find and replace strings in the first column of data

Question

Not a professional coder, but my current hospital position is allowing me to code some workarounds for our abysmal EHR system. The following is part of my Python project to visualize laboratory data.

The following code snippet works, but I haven't figured out a way with list comprehensions or np.where to streamline it and potentially improve performance. I haven't found any posts yet to answer this, including among those suggested during question submission.

This code tests tuple[0] in a list of tuples (test_and_value) comprising (lab test names, test values), searching for misspelled lab test names and replacing them with standardized lab test names.

Common misspellings and their replacement terms live in another list of tuples (chem_fix) to preserve the order in which they are checked. Some word fragments of misspellings are used in lieu of full words, therefore testing cannot be a simple == statement.

Final product is a list of tuples, test_and_value_f, with correctly spelled lab tests.

Pertinent code excerpts:

chem_fix = [('A/G RATIO', 'A:G RATIO'), ('ALK', 'ALKALINE PHOSPHATASE'), ('ALT', 'ALT'), ('AST', 'AST'), ('BILI', 'BILIRUBIN,TOTAL'), ('BLIL', 'BILIRUBIN,TOTAL'), ('BUN/CREAT', 'BUN/CREAT RATIO'), ('BUN', 'BLOOD UREA NITROGEN'), ('CARBON', 'CO2'), ('GLOB', 'GLOBULIN'), ('RANDOM', 'GLUCOSE'), ('PROTEIN', 'TOTAL PROTEIN')]
fix_terms = [x[0] for x in chem_fix]

test_and_value_f = []
replaced = False
for lab_test, value in test_and_value:
    for count, term_needing_fix in enumerate(fix_terms):
        if term_needing_fix in lab_test:
            test_and_value_f.append((chem_fix[count][1], value))
            replaced = True
            break
    if replaced == False:
        test_and_value_f.append((lab_test, value))
    else:
        replaced = False

Sample test_and_value input: [('GLUCOSE', '77'), ('BUN', '14'), ('CREATININE', '1.4'), ('CALCULATED BUN/CREAT', '10'), ('SODIUM', '142'), ('POTASSIUM', '3.7'), ('CHLORIDE', '100'), ('CARBON DIOXIDE', '30'), ('CALCIUM', '8.9'), ('PROTEIN, TOTAL', '6.5'), ('ALBUMIN', '3.4'), ('CALCULATED GLOBIN', '3.1'), ('CALCULATED A/G RATIO', '1.1'), ('BILIRUBIN, TOTAL', '0.7'), ('ALKALINE PHOSPHATASE', '59'), ('SGOT (AST)', '3')]

Sample test_and_value_f output: [('GLUCOSE', '77'), ('BLOOD UREA NITROGEN', '14'), ('CREATININE', '1.4'), ('BUN/CREAT RATIO', '10'), ('SODIUM', '142'), ('POTASSIUM', '3.7'), ('CHLORIDE', '100'), ('CO2', '30'), ('CALCIUM', '8.9'), ('TOTAL PROTEIN', '6.5'), ('ALBUMIN', '3.4'), ('GLOBULIN', '3.1'), ('A:G RATIO', '1.1'), ('BILIRUBIN,TOTAL', '0.7'), ('ALKALINE PHOSPHATASE', '59'), ('AST', '3')]

Is there a more efficient way to do this?

Answer 1

You could use a dictionary instead of a list of tuples for chem_fix, and then you could get rid of fix_terms. Then you could use search from the re module in the list comprehension and then look up any matches in the dictionary. And you can use walrus operator to store whatever is returned from search since first you have to make sure it’s not None. Here’s an example of that:

# there’s already a function called “compile”
# “import re” used so it doesn’t get overwritten
pattern = re.compile('|'.join(chem_fix))
pattern_search = pattern.search # method will be looked up faster
test_and_value_f = [
    (
        chem_fix[m.group(0)]
        if (m := pattern_search(lab_test)) is not None
        else lab_test,
        value
    )
    for lab_test, value in test_and_value
]