3
\$\begingroup\$

I am trying to find a way to optimize the code runtime. Improvements I have implemented include using dictionaries, removing redundancies and using str.translate.

from collections import namedtuple
import string

import jellyfish as jf
import Levenshtein as l #conda install -c conda-forge python-levenshtein

#instantitiate trans table
words2numbers = {
    '1' : 'one ',
    '2' : 'two ',
    '3' : 'three ',
    '4' : 'four ',
    '5' : 'five ',
    '6' : 'six ',
    '7' : 'seven ',
    '8' : 'eight ',
    '9' : 'nine ',
    '0' : 'zero '
    }

string_punc = {key: None for key in string.punctuation}
replace_with = str.maketrans({**words2numbers, **string_punc})
String = namedtuple('String','encode lev ori')

def norm_func(s):
    '''
    Normalize number characters and punctuations

    '123' -- > 'one two three'
    '.?1 12 gray' --> 'one two gray'
    '''
    s = s.translate(replace_with)
    return s.lower()

def encode_phonetic(s): #should return normalize string here
    '''
    Tag sentence by tokenizing string and returns soundex via jellyfish.
    Creates a dictionary that stores the normalized string and key value pairs
    for each tokenized string.

    '123 washington blvd' --> 
    '{'T000': 'two',
    'B413': 'blvd',
    'T600': 'three',
    'W252': 'washington',
    'O500': 'one',
    'KEY': 'one two three  washington blvd'}'
    '''
    norm = norm_func(s)
    norm_set = set(norm.split())
    norm_dict = {jf.soundex(i):i for i in norm_set}
    norm_dict['KEY'] = norm
    return norm_dict

def bstring(s,o):
    '''
    Compares both 2 strings based on soundex differences and string distance in a namedtuple.
    encode: difference in soundex encoding
    lev: difference in string distance
    ori: holds original string

    bstring('123 washington blvd','123 washington boulevard') -->
    String(encode={'B413': 'blvd'}, lev=5, ori=('123 washington blvd', '123 washington boulevard'))    
    '''
    s_dict = encode_phonetic(s)
    o_dict = encode_phonetic(o)
    e_d = {**s_dict, **o_dict}
    e_diff = {k:e_d[k] for k in set(s_dict) - set(o_dict)}

    dist = l.distance(s_dict['KEY'], o_dict['KEY'])
    return String(e_diff, dist, (s,o))

I also tried timing the functions and I understand that a big part of the runtime depends on the string length, which I am leaving out at the moment.

import timeit

print(bstring('123 washington blvrd','123 washington boulevard'))
print(timeit.timeit(lambda: bstring('123 washington blvd','123 washington boulevard'), number= 1000000))
>>String(encode={}, lev=4, ori=('123 washington blvrd', '123 washington boulevard'))
>>47.34113038036594

Is there anything I to improve the code runtime from the data model perspective?

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.