# Python function to strip inline repr'd unicode strings

I have to write a python function that takes a string that contains raw unicode strings (e.g. "u'hello' there") and transform it into a string that strips the u'' identifiers from it. For example:

strip_inline_unicode("u'hello' there")  # -> 'hello there'


Without getting into too much detail as to why I have to do this, sufficed to say I cannot simply replace the logic that generates these strings, and I have to strip these because it is being returned as human-readable output to a user.

Constraints:

• Strings can have "malformed" unicode identifiers which should be handled correctly (e.g. "hello u'there" -> "hello there")
• Strings can have empty unicode identifiers (e.g. "hello u''there" -> "hello there")
• Strings will never have "nested" unicode identifiers (e.g. 'u"u\'foo\'"')
• Quotes will always be single-quotes (so never u"<stuff>")

Here is what I came up with:

def strip_inline_unicode(stupid_string):
"""Takes a string that looks like "u'hello' there" and returns
"hello there" """

in_unicode = False
pos = 0

new_str = ''

while (pos < len(stupid_string)):
if pos + 1 >= len(stupid_string):
if in_unicode and stupid_string[-1] == "'":
new_str += stupid_string[pos:-1]
else:
new_str += stupid_string[pos:]
break

cur = stupid_string[pos]
nxt = stupid_string[pos + 1]
if cur == 'u' and nxt == "'" and not in_unicode:
in_unicode = True
pos += 1
elif in_unicode and cur == "'":
in_unicode = False
else:
new_str += cur

pos += 1

return new_str


When I plug it into the interpreter it seems to work correctly:

In [12]: strip_inline_unicode("u'hello' there")
Out[12]: 'hello there'

In [14]: strip_inline_unicode("hello there")
Out[14]: 'hello there'

In [15]: strip_inline_unicode("hello u'there")
Out[15]: 'hello there'

In [16]: strip_inline_unicode("hello u''there")
Out[16]: 'hello there'

In [17]: strip_inline_unicode("au'b'")
Out[17]: 'ab'

In [18]: strip_inline_unicode("u'abc'")
Out[18]: 'abc'


However, I am by no means a Python expert, and it seems like I could accomplish something similar in a simpler and more robust manner, perhaps by using regexes. I was hoping to get some feedback on the implementation and maybe simplify it/make it better.

re.sub(r"u'([^']*)'?", r'\1', string)

1. A literal u'.
2. Then anything that is not a ' zero or more times: [^']*.
3. Store that for later retrieval: ([^']*).
4. End with an optional '.
• Infinitely better. I missed the part in the re.sub docs that mentions that backreferences can be used to specify replacements. Really cool stuff. Thanks so much for your help! Mar 19 '15 at 0:52