Python function to strip inline repr'd unicode strings

Question

I have to write a python function that takes a string that contains raw unicode strings (e.g. "u'hello' there") and transform it into a string that strips the u'' identifiers from it. For example:

strip_inline_unicode("u'hello' there")  # -> 'hello there'

Without getting into too much detail as to why I have to do this, sufficed to say I cannot simply replace the logic that generates these strings, and I have to strip these because it is being returned as human-readable output to a user.

Constraints:

• Strings can have "malformed" unicode identifiers which should be handled correctly (e.g. "hello u'there" -> "hello there")
• Strings can have empty unicode identifiers (e.g. "hello u''there" -> "hello there")
• Strings will never have "nested" unicode identifiers (e.g. 'u"u\'foo\'"')
• Quotes will always be single-quotes (so never u"<stuff>")

Here is what I came up with:

def strip_inline_unicode(stupid_string):
    """Takes a string that looks like "u'hello' there" and returns
    "hello there" """

    in_unicode = False
    pos = 0

    new_str = ''

    while (pos < len(stupid_string)):
        if pos + 1 >= len(stupid_string):
            if in_unicode and stupid_string[-1] == "'":
                new_str += stupid_string[pos:-1]
            else:
                new_str += stupid_string[pos:]
            break

        cur = stupid_string[pos]
        nxt = stupid_string[pos + 1]
        if cur == 'u' and nxt == "'" and not in_unicode:
            in_unicode = True
            pos += 1
        elif in_unicode and cur == "'":
            in_unicode = False
        else:
            new_str += cur

        pos += 1

    return new_str

When I plug it into the interpreter it seems to work correctly:

In [12]: strip_inline_unicode("u'hello' there")
Out[12]: 'hello there'

In [14]: strip_inline_unicode("hello there")
Out[14]: 'hello there'

In [15]: strip_inline_unicode("hello u'there")
Out[15]: 'hello there'    

In [16]: strip_inline_unicode("hello u''there")
Out[16]: 'hello there'

In [17]: strip_inline_unicode("au'b'")
Out[17]: 'ab'

In [18]: strip_inline_unicode("u'abc'")
Out[18]: 'abc'

However, I am by no means a Python expert, and it seems like I could accomplish something similar in a simpler and more robust manner, perhaps by using regexes. I was hoping to get some feedback on the implementation and maybe simplify it/make it better.

Answer 1

Yes, regexps were my first thought:

re.sub(r"u'([^']*)'?", r'\1', string)

Dissected:

1. A literal u'.
2. Then anything that is not a ' zero or more times: [^']*.
3. Store that for later retrieval: ([^']*).
4. End with an optional '.