Reduce run time of NLP approximate matching code

Question

The code below matches a list of features to a large corpus and returns the sub-query match with a score above 80. The challenge is the list of features on the full data-set is > 5,000 and comparing to multiple documents. Therefore is taking too long to work using the fuzzywuzzy package.

Per the Spyder profiler, the bottlenecks are at:

if(fuzz.ratio(wordtocompare,feature.lower())> match) and _find_and_load_unlocked

Would vectorizing the code in its current form help or is there a faster approximate matcher that accounts for matching a sub-query (information extraction) of text compared to a defined list? Has anyone had success using polyleven and port the results back into Python?

import pandas as pd
from fuzzywuzzy import fuzz
import re

document = """If you're shopping within the Toyota family, the Highlander offers appreciably more space than the RAV4, both in terms of cargo capacity and its extra row of seats. It also has a deeper, more accessible space than what's in the 4Runner.
That said, the Highlander is one of the smallest three-row crossovers available. Apart from the Kia Sorento and maybe the Mazda CX-9, you're going to find more cargo capacity and passenger space in the Highlander's competitors. That's especially true in the third row. The second row slides a bit more to grant extra legroom now, but the third row remains awfully close to the floor, and it won't be long before your growing kids will feel cramped and claustrophobic in the way-back. Full-size teens and adults will be flat-out grumpy.
That said, the Highlander's smaller size might be just right for many buyers who appreciate its more manageable dimensions when parking or maneuvering in tight spots. Plus, if you only need that third row for occasional use and just a little more space than what a RAV4 provides, it really won't matter that the Highlander can't match its competitors' jumbo size.
We expect pricing for the 2020 Highlander to be announced closer to its on-sale date in December 2019, with the Hybrid arriving in February 2020. Specifically, it should correspond with our first test drive opportunity, likely in November. We do have a pretty comprehensive features breakdown, however, which you can see below.
Standard equipment on the Highlander L includes 18-inch alloy wheels, three-zone automatic climate control, accident avoidance tech features (see safety section below), full-speed adaptive cruise control, LED headlights, rear privacy glass, proximity entry and push-button start, an eight-way power driver seat and the 8-inch touchscreen. The LE additions include a power liftgate, blind-spot warning, LED foglamps, and a leather-wrapped steering wheel.
The XLE additions include automatic headlights, roof rails, a sunroof, heated front seats, driver power lumbar, a four-way power passenger seat, SofTex vinyl upholstery, second-row sunshades and an auto-dimming rearview mirror.
The Limited additions include 20-inch wheels, a handsfree power liftgate, upgraded LED headlights, a cargo cover, driver memory settings, ventilated front seats, leather upholstery, integrated navigation and a JBL sound system upgrade.
The Platinum additions include adaptive and self-leveling headlights, automatic wipers, a panoramic sunroof bird's-eye parking camera, a head-up display, a digital rearview mirror camera, perforated leather upholstery, heated second-row seats and a 12.3-inch touchscreen.
"""
features =["steering","touch screen","LED headlight"]

def findcarfeatures(features, document, match=80):
    result=[]
    for feature in features:
        lenfeature = len(feature.split(" "))
        word_tokens = nltk.word_tokenize(document)
        #filterd_word_tokens = [w for w in word_tokens if not w in stop_words]
        for i in range (len(word_tokens)-lenfeature+1):
            wordtocompare = ""
            j=0
            for j in range(i, i+lenfeature):
                if re.search(r'[,!?{}\[\]\"\"\'\']',word_tokens[j]):
                    break
                wordtocompare = wordtocompare+" "+word_tokens[j].lower()
            wordtocompare.strip()
            if not wordtocompare=="":
                if(fuzz.ratio(wordtocompare,feature.lower())> match):
                    result.append([wordtocompare,feature,i,j])
    return result

findcarfeatures(features,document)

Out[90]: 
[[' steering', 'steering', 353, 353],
 [' touchscreen .', 'touch screen', 334, 335],
 [' touchscreen .', 'touch screen', 474, 475],
 [' led headlights', 'LED headlight', 313, 314],
 [' headlights', 'LED headlight', 314, 315],
 [' headlights', 'LED headlight', 361, 362],
 [' led headlights', 'LED headlight', 408, 409],
 [' headlights', 'LED headlight', 409, 410],
 [' headlights', 'LED headlight', 442, 443]]

Answer 1

Minor perf improvements

These are unlikely to impact your performance in a material way, but they are performance improvements nonetheless:

re.search(r'[,!?{}\[\]\"\"\'\']',word_tokens[j])

recompiles the regex every time. re.compile() outside of your loops so that this does not happen.

Repeated concatenation such as this:

wordtocompare = wordtocompare+" "+word_tokens[j].lower()

can be a problem; strings in Python are immutable, so this is recreating a new string instance every time the concatenation is done. To avoid this, consider using StringIO or join a generator.

Other improvements

if not wordtocompare=="":

should be

if word_to_compare != "":

Also, wordtocompare.strip() is not being assigned to anything so it does not have any effect, currently.