Universal string conversion

Question

Below there is a function eating all strings and convert it to Unicode. It works fine. But is it the best way to do so? Is there maybe already an existing function for it?

The reason was a German Windows version which prints German ä ö ü - error-messages which are retrieved from popen/communicate etc. There must have been someone before me with the same issue.

# python2.7
def all_eating_unicode_converter(input_string):
    test1 = 'äöü'
    test2 = b'\xe4\xf6\xfc'
    test3 = u'äöü'
    """
    Converts every input to unicode. tested with testcases abouve!
    :param input_string: may be a string or unicode or bytes.
    :return: returns unicode
    """

    l.debug("type: %s\n message:%s" % (type(input_string), input_string))
    if type(input_string) is not unicode:
        # i don't need to decode unicode. because it already is!
        output_string = input_string.decode("utf-8")  # converts bytes to unicode
    else:
        output_string = input_string
    return output_string

Answer 1

First, you should not be using Python 2 anymore, if at all possible. It will reach end of support at the end of the year.

In Python 3 your code would just be:

def all_eating_unicode_converter(input_string):
    """
    Converts every input to unicode.
    :param input_string: may be a string or unicode or bytes.
    :return: returns unicode

    Tested with: 'äöü', b'\xc3\xa4\xc3\xb6\xc3\xbc', u'äöü'
    """
    if isinstance(input_string, bytes):
        return input_string.decode("utf-8")
    return input_string

Note that I used isinstance instead of type, to also allow derived types of bytes to work properly.

I also shortened the logic by not insisting on having a single return from the function. Here it is clear enough what is happening, and I am in general a fan of early returns.

Docstrings only work when they are actually right after the function header, so I moved the testcases inside the docstring.

In addition, one reason the logging module is so powerful is that it allows you to write log messages which are costly to print and never print them. In other words the replacement of the placeholders is only performed if the message is actually being printed, i.e. if the log level is high enough. So you should always do this:

l.debug("type: %s\n message:%s",  type(input_string), input_string)