2
\$\begingroup\$

Problem

I have a list of around ~1000 honorifics, see below for a sample.

Given an input string of a name, for example "her majesty queen elizabeth windsor", the function should return "elizabeth windsor". If there is no honorific present at the start of the name (to simplify the problem), the function should simple return the name itself (e.g. elizabeth windsor -> elizabeth windsor).

I have pretty intense latency constraints, so need to optimise this code as much as possible.

Working solution

Here is my working solution, there are some additional constraints to reduce false positives (for example lance is both an honorific and a first name), see the unit tests:

def strip_honorific(source: str, honorifics: List[str]) -> str:
    source_tokens = source.split()
    if len(source_tokens) > 2:
        for honorific in honorifics:
            if source.startswith(f"{honorific} "):
                stripped_source = source[len(honorific) + 1 :]
                if len(stripped_source.split()) > 1:
                    return stripped_source
    return source

Unit tests

def test_honorifics():
    assert strip_honorific(source="her majesty queen elizabeth windsor", honorifics = honorifics) == "elizabeth windsor"
    assert strip_honorific(source="elizabeth windsor", honorifics = honorifics) == "elizabeth windsor"
    assert strip_honorific(source="mrs elizabeth windsor", honorifics = honorifics) == "elizabeth windsor"
    assert strip_honorific(source="mrselizabeth windsor", honorifics = honorifics) == "mrselizabeth windsor"
    assert strip_honorific(source="mrselizabeth windsor", honorifics = honorifics) == "mrselizabeth windsor"
    assert strip_honorific(source="her majesty queen", honorifics = honorifics) == "her majesty queen"
    assert strip_honorific(source="her majesty queen elizabeth", honorifics = honorifics) == "her majesty queen elizabeth"
    assert strip_honorific(source="kapitan fred", honorifics = honorifics) == "kapitan fred"
    
    
test_honorifics()

Benchmark

For a basic benchmark, I've used the below list of honorifics (minus the ellipses).

source_lst = [
    "her majesty queen elizabeth windsor",
    "mr fred wilson",
    "the rt hon nolan borak",
    "his most eminent highness simon smithson", 
    "kapteinis jurijs jakovļevs", 
    "miss nancy garland",
    "missnancy garland",
]


times = []
for _ in range(1000):
    for source in source_lst:
        t0 = time.time()
        strip_honorific(source=source, honorifics = honorifics)
        times.append(time.time() - t0)

print(f"Mean time: {sum(times)/ len(times)}s") # Mean time: 5.11584963117327e-06s

Honorifics list

honorifics = [
    "mr", 
    "mrs", 
    "the hon",
    "the hon dr",
    "the hon lady",
    "the hon lord",
    "the hon mrs",
    "the hon sir",
    "the honourable",
    "the rt hon",
    "her majesty queen",
    "his majesty king",
    "vina",
    "flottiljamiral",
    "superintendent",
    "rabbi",
    "diraja",
    "domnul",
    "kindralleitnant",
    "countess",
    "pan",
    "khatib",
    "zur",
    "vice",
    "don",
    "flotiles",
    "dipl",
    "his most eminent highness", 
    ...
    "the reverend", 
    "archbishop", 
    "sheik", 
    "shaikh", 
    "the rt hon lord", 
    "la tres honorable"
    "ekselence", 
    "kapteinis", 
    "kapitan", 
    "excellenza"
    "mr", 
    "mrs", 
    "miss"
] 
\$\endgroup\$
3
  • \$\begingroup\$ You have a bug. Check the output of strip_honorific("the hon dr smith", honorifics) \$\endgroup\$
    – C.Nivs
    Nov 7, 2022 at 21:58
  • \$\begingroup\$ If you want something like this in production, it may be worth exploring the re regular expressions module. \$\endgroup\$
    – Josiah
    Nov 7, 2022 at 23:38
  • \$\begingroup\$ What’s the output for “m bison”? \$\endgroup\$
    – Davislor
    Nov 8, 2022 at 10:17

1 Answer 1

2
\$\begingroup\$

Combine all the honorifics into a regular expression so they can all be tested at once rather that testing them one-by-one. Sort the honorifics longest to shortest

import re

honorifics.sort(key=len, reverse=True)
all_honorifics = '|'.join(f'{h} ' for h in honorifics)

honorifics_regex = re.compile(rf"\A(?:{all_honorifics})")

def strip_honorific(source, honorifics_regex):
    match = honorifics_regex.search(source)
    if match:
        stripped_source = source[match.end():]
        if len(stripped_source.split()) > 1:
            return stripped_source
        
    return source

Use the timeit module from the standard library:

import timeit

code = "for source in source_lst: name = strip_honorific(source, regex)"

timeit.timeit(code, globals=globals())

The regex version runs in about 5ms, compared to 20ms for the original code.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.