I am building a backend APi for which I need to extract query filters from request URL of following pattern:

http://api.com/v1/users?filters=filters=(name like 'Milk' and name eq 'Eggs') and age ge 2.55`

The URL can contain complex filters like: like 'Milk' and name eq 'mil') and age ge 2.55 and age ge 2.55 and (name like 'Milk' and name eq 'Eggs') and age ge 2.55

What I have done so far is:

import re

"""Dated: 13-12-2016
Parsing utility for EntityManager. Defines set of rules to parse filters      
string received in GET request URL.

class Parser():

"""Contains utility to parse precedence groups, simple filters i.e without grouping. This class assumes to receive
one kind of boolean logical operator at a certain level. If you wish to use 2 difference operator try enclosing 
one of them in a precedence group
Regular Expression:
    - '\((.*?)\)' (Parse precedence groups)
    -  '(((?:(and|or)\s+)*\((%s)\))(?:\s+(and|or))*)' (match groups within query string and identify pre and post logical operators)
    -   '\s+(and|or)\s+' (match logical operator in simple statement)
Example Filter String: 
    - filters = (name like Milk or name eq mil) and age ge 2 and age ge 2 and (name like Milk or name eq Eggs) and age ge  2
    - filters = (name like Milk or name eq mil) and (name like Milk or name eq Eggs) 
    - filters=name like Milk

REGEX_FILTER_GROUPS = r'\((.*?)\)'
REGEX_EXPRESSION_PARSER = r'(((?:(and|or)\s+)*\((%s)\))(?:\s+(and|or))*)'
REGEX_LOGICAL_OPERATORS = r'\s+(and|or)\s+'

def parse_filters(filter_str):

    """Accepts a string and returns list of filters
        - filter_str (str): Filter string received in request URL.
        - [{},{}], [{},{}]: first list are precedence filters, second list is simple filters   
    r = re.compile(Parser.REGEX_FILTER_GROUPS)
    iterator = r.finditer(filter_str)
    match_list = list(iterator)

    if len(match_list) > 0:

        filters, ranges = Parser.parse_priority_group_list(match_list, filter_str)
        priority_filters = list()

        for key in Parser.parse_filter_statement_list(filters):

        remaining_str, logical_operator = Parser.parse_remaining_filters(filter_str, filters, ranges)

        if remaining_str.startswith('(') and remaining_str.endswith(')'):
            remaining_filters_dict = {'expr': None, 'op': logical_operator}

            remaining_filters,op = Parser.parse_filter_statement(remaining_str)

        if op is None and logical_operator is not None:
            op = logical_operator

        remaining_filters_dict = {'expr': remaining_filters, 'op': op}
        return priority_filters, remaining_filters_dict

        simple_filters,op = Parser.parse_filter_statement(filter_str)
        return None, {'expr': simple_filters, 'op': op}

def parse_priority_group_list(match_list, query_string):

    """Parse precedence filters in filter string
        - match_list (str): list of precedence group match against reegx.
        - query_string (list): filter string

        - list, list(tuple(),): first list are actual precedence filters, second is start, end ranges of each filter.
    filters, group_boundries, group_filters, operators = list(), list(), list(), set()
    priority_filters, filter_boundaries = zip(*[(match.group(1), match.span()) for match in match_list])
    return priority_filters, filter_boundaries

def parse_remaining_filters(query_string_dict, filters, group_boundries):

    """Parse simple filters in filter string
        - query_string_dict (str): Filter string received in request URL.
        - filters (list): list of precedence filters.
        - group_boundries (list of tuples): start-end range of each precedence filters.
        - str, str: first value is str representation of simple filters, second is  logical operator in between   
    remaining_filters, logical_operator = '', ''
    remaining_str = query_string_dict

    for count, key in enumerate(filters):
        operators = set()
        match_string = list()

        match_string = re.search(Parser.REGEX_EXPRESSION_PARSER % key, query_string_dict, flags = re.I).groups()

        expr_with_both_operator = match_string[0]
        expr_with_pre_operator = match_string[1]
        pre_operator = match_string[2]
        post_operator = match_string[4]

        logical_operator = pre_operator

        if group_boundries[count][1] <= len(query_string_dict) and group_boundries[count][0] != 0:
            string_to_match = expr_with_pre_operator
            string_to_match = expr_with_both_operator

        if pre_operator is None:
            print("Group At Start")

        if post_operator is None:
            print("Group At End")

        remaining_str =  remaining_str.replace(string_to_match,'')

    remaining_str = re.sub(r'\s+', ' ', remaining_str)
    remaining_str = remaining_str.strip()

    return remaining_str, logical_operator

def parse_filter_statement_list(expr):

    """Parse list of filters to individual statement
        - expr (list): Filters list

        - list : list of parsed filters, empty in case no filters found   
    filters_list = list()

    if isinstance(expr, list) or isinstance(expr, tuple):   

        for key in expr:

            filter,op = Parser.parse_filter_statement(key)
            filters_list.append({'expr': filter, 'op': op})

    return filters_list

def parse_filter_statement(expr):

    """Parse individual filter individual statement
        - expr (list): Filters statement

        - list, str : list of parsed filters, operator in between   
    operator,base_filters = set(),list()
    operator = set([op for op in Parser.parse_logical_operator(expr)])

    if bool(operator):

        bool_operator = next(iter(operator))
        base_filters = expr.split(bool_operator)


        bool_operator = None

    return base_filters,bool_operator

def parse_logical_operator(str):

    """Identify logical boolean operator between filters
        - str (str): Filters statement

        - list: list of operators found   
    op_list = list()
    op_list = re.findall(Parser.REGEX_LOGICAL_OPERATORS, str, flags = re.I)
    return op_list

After that I remove extra and's from the expression and append remaining filters to the list and return the list.


Is this approach good to parse filter like these? Any suggestions/comments?

  • \$\begingroup\$ Your code has indentation issues so it can't work. It is also stripped off of too many context, leaving us at guessing what it is achieving. I am voting to close as off-topic for hypothetical code. \$\endgroup\$ Commented Dec 5, 2016 at 19:31
  • \$\begingroup\$ I have fixed indentation issues. I don't understand where the context is stripped off, i have mentioned comments for every line of code. The input is the second URL string mentioned in question and output of each line is there in comments. If you wish to see the whole code you can: github.com/sohaibfarooqi/stargate/blob/master/stargate/… \$\endgroup\$ Commented Dec 6, 2016 at 5:52
  • \$\begingroup\$ Your first two lines contains ubdefined symbols, and you say you removed the last part of the function. There is also self used only once that makes me wonder about why this would be a method rather than a simple function... Why don't you post your whole file instead of a link to it? \$\endgroup\$ Commented Dec 6, 2016 at 7:30
  • \$\begingroup\$ First line defines a regex which is mentioned in comment right next to that line. second line also specifies its value in comment above it. self was a mistake which i corrected. I will post the whole function. \$\endgroup\$ Commented Dec 6, 2016 at 7:37
  • \$\begingroup\$ Please check i have updated the code. \$\endgroup\$ Commented Dec 13, 2016 at 12:07


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.