# Converting text to dataset parameters: Union of an object and the category to which it belongs

You need to open the parentheses combined category and the object that belongs to it. The combination of categories means that we have a category and its objects, for example genre (rock, pop, hip-hop) needs to be converted to genre_rock, genre_pop, genre_hip-hop. At the input and output of the code is a string, just during operation it is converted into a list. The purpose of the code is to clear one of the dataset parameters to work with the recommender system.

Question:
What parts of the code can be optimized so that it works faster + does not look ugly?

Example:

1. 'Category (first ,second (1,2)) -> 'Category_first,Category_second_1,Category_second_2'

2. 'Category (first ,second)' -> 'Category_first,Category_second'

Code:

def i_par_find(text, i_par_l):
l_r = 0
i_par_r = i_par_l
for s in text[i_par_r:]:
if s == '(':
l_r += 1
elif s == ')':
l_r -= 1
i_par_r += 1
if l_r == 0:
return i_par_l, i_par_r
​
def glue_list(text, i_par_l, i_par_r, i_comma):
word_list = re.sub(r'[()]', '',
text[i_par_l:i_par_r]).replace(' ', '_').split(",")
word_list = [w.strip('_') for w in word_list]
category = text[i_comma:i_par_l].strip()
list_word = []
for w in word_list:
if w != '':
list_word.append(f'{category}_{w}')
return ",".join(list_word)
​
def list_w(text):
# If there is no comma, the category search starts at index 0.
i_comma = 0
# Index
i = 0
​
while i < len(text) - 1:
# Select a specific character in the text.
i += 1
s = text[i]
​
# Step 1: save the last comma before the parenthesis
if s == ',': i_comma = i
​
# Step 2: Check for "("
if s == '(':
# Step 3: Look up the index of the leading "(" and closing ")", nested parentheses are ignored.
i_par_l, i_par_r = i_par_find(text, i)
​
# If a comma is inside brackets, then its index is set to zero.
if i_comma >= i_par_l and i_comma <= i_par_r:
i_comma = 0
​
# If there is "(" inside the found slice, then we use recursion
if '(' in text[i_par_l + 1:i_par_r - 1]:
text = text[:i_par_l + 1] + list_w(
text[i_par_l + 1:i_par_r - 1]) + text[i_par_r - 1:]
​
# If processing text doesn't start at 0, add 1 to index to capture comma.
if i_comma != 0: i_comma += 1

​            # Adding a comma depending on the position of the processed area in the text
c_or_n = ''
if i_par_r <= len(text) and text[i_par_r - 1] == ',': c_or_n = ','
​
# Step 4: Connect parameters and categories, with text before and after the treated area.
text = text[:i_comma] + glue_list(
text, i_par_l, i_par_r, i_comma) + c_or_n + text[i_par_r:]
​
text = re.sub(r'[()]', '', text)
return text

list_w(text)


It looks like you are trying to parse the string character by character, mutating the text as you go along.

def list_w(text):
...
i = 0
​
while i < len(text) - 1:
i += 1
s = text[i]
...
text = text[:i_par_l + 1] + list_w(
text[i_par_l + 1:i_par_r - 1]) + text[i_par_r - 1:]
​        ...
text = re.sub(r'[()]', '', text)
return text


A clearer algorithm could be to work from the inside out. With 'Category (first ,second (1,2))', find the innermost category pattern second (1,2) and replace that with second_1, second_2, producing 'Category (first ,second_1, second_2)'. Since there are more parenthesis, repeat, which produces the desired result.

Finding the innermost category pattern starts with looking for text which does not contain parenthesis [^()]+, but is surrounded by them, $$...$$. We also need to find a "word" before that, possibly with some trailing space: \b \w+ \s*.

To that regex, we'll add capturing groups (...) to extract the term before the parenthesis and well as inside. \b(\w+)\s*$$([^()]+)$$

Now all that is left to do it split on any commas in the second group, strip off spaces, and add the first group as a prefix to each term. Join the result with commas, and replace what what originally matched. Repeat until there are no more parenthesis.

import re

def categories(s):
def expand(m):
prefix, suffixes = m.groups()
return ",".join(f"{prefix}_{suffix}"
for suffix in map(str.strip, suffixes.split(",")))

while '(' in s:
s = re.sub(r"\b(\w+)\s*$$([^()]+)$$", expand, s)

return s

s = 'Category (first ,second (1,2))'

print(categories(s))

• Thank you very much, I'll go learn regular expressions Jul 30 at 6:49

I'd suggest that you are solving this at the wrong level. Trying to do it character-by-character makes it much more difficult than it needs to be. You are already using the re module to replace '(' and ')' with spaces. Why not use it to "tokenize" the input string (You could also use the tokenize module).

The input string is made up of names, parenthesis, and commas. This regex would break return a list of these pieces:

tokens = re.findall(r"\s*(\w+|[(,)])", text)


Then we can operate with those pieces to assemble the categories.

def flatten(source):
# This is the prefix of the current subcategory if any.
# It is a string of names joined by '_'.
path = ''

tokens = pat.findall(source)
while tokens:
# pick off the next token
token, *tokens = tokens

if token.isalnum():
name = token

# If there is an '(' then there must be subcategories. So, add
# the current name to the "path"
elif token == '(':
path = f"{path}_{name}" if path else name

# A ',' ends the current subcategory. Yield the entire flattened
# category name. The 'if name:' guard prevents yielding multiple
# copies of a subcategory when there are ")," and "))"
elif token == ',':
if name:
yield f"{path}_{name}"
name = ''

# a ')' ends the list of subcategories. Yield the flattened
# category name. Also remove the parent from the 'path'. Same
# guard as for the ',' above.
elif token == ')':
if name:
yield f"{path}_{name}"

path,_,_ = path.rpartition('_')
name = ''


Use it like so:

text = "Category (first (alpha, bravo), second (1))"
print(','.join(flatten(text)))


Output:

Category_first_alpha,Category_first_bravo,Category_second_1

• Thanks a lot, it's a pity that I can't mark two solutions at once. Jul 30 at 6:59