10
\$\begingroup\$

I have a django app with search powered by haystack. The general search goes very quickly, but I need to filter the results by their model type. It works but is very slow. I've never really worked on optimizing code so I don't necessarily know the tools to use.

def search(request):
    if 'q' in request.GET:
        form = SearchForm(request.GET)
    else:
        redirect(reverse('index'))

    results = form.search()
    result_list = []

    # all search indexes defined
    asset_classes = get_subclasses(Asset) #returns all models (10 or so)
    for item_type in asset_classes:
        model_name = item_type._meta.model_name
        subquery = results.filter(item_model_name=model_name) #I _think_ this is the slow step
        num_results = subquery.count()
        verbose_name = item_type._meta.verbose_name
        result_table = getattr(tables, model_name.capitalize() +
                               'Table', tables.ItemTable)(subquery)
        RequestConfig(request,
                      paginate={"per_page": 10}
                      ).configure(result_table)
        result_list.append((verbose_name,
                            num_results,
                            result_table))
    return render(request, 'assets/generic_search.html',
                  {'result_list': result_list})

EDIT: So, I discovered the timeit module, and I found two bottlenecks: The RequestConfig took about 500 ms each time, so I moved that to an if-block so it only happens when there's a table in the first place (which I should have done anyway). However, the num_results=subquery.count() also takes about 600 ms, which seems really long and I don't see any way to speed it up.

\$\endgroup\$
1
\$\begingroup\$

I suggest collecting all the item_model_names from the search results, counting them, and forming an index.

Then during the for loop, instead of querying the database, just look up the count.

This solution has the benefit that it removes about 10 database calls, at the cost of some up-front processing.

source

def make_name_index(queryset):
    import collections
    name_index = collections.Counter()
    name_index.update(
        queryset.values_list(
            'item_model_name', flat=True,
        )
    )
    return name_index


name_index = make_name_index( form.search() )

# in for loop:
num_results = name_index[model_name]
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.