# Advanced formatting with argument passing

I want to customise how my objects are printed using the .format() function.

I have two problems to make that happen in clear and concise way. But since the second one stems from the first one I'm presenting both here.

Formatting array

Passing options to .format() in array seems to me to be awfully messy if I want to match default behaviour with __repr__:

import binascii

def format_data(data, formatstr):
if 'x' in formatstr:
return binascii.b2a_hex(data).decode('ascii')
else:
return repr(data)

class InsideA(object):
def __repr__(self):
return "InsideA()"

def __format__(self, _):
return "InsideA()"

class InsideB(object):
def __init__(self, data):
self.data = data

def __repr__(self):
data = format_data(self.data, 'x')
return "InsideB(data={0})".format(data)

def __format__(self, formatstr):
data = format_data(self.data, formatstr)
return "InsideB(data={0})".format(data)

if __name__ == "__main__":
a = InsideA()
b = InsideB(bytearray(b'test\xff\xff'))

array = [a, b]

print("what I want:")
print(array)
print("")

print("doesn't work:")
try:
print("{0:x}".format(array))
except Exception as e:  # catch just to continue execution
print(e)
print("")

print("Leaves ' around elements:")
print(["{0:x}".format(elem) for elem in array])
print("")

print("Works but seems overly complex:")
print("[" + ", ".join("{0:x}".format(elem) for elem in array) + "]")


Produces:

What I want:

[InsideA(), InsideB(data=74657374ffff)]


Doesn't work:

non-empty format string passed to object.__format__


Leaves ' around elements:

['InsideA()', 'InsideB(data=74657374ffff)']


Works but seems overly complex:

[InsideA(), InsideB(data=74657374ffff)]


Formatting with non-uniform objects

The problem only gets compounded if the array elements don't always implement the format function, requiring use of try except clauses:

from __future__ import print_function
import binascii

def format_data(data, formatstr):
if 'x' in formatstr:
return binascii.b2a_hex(data).decode('ascii')
else:
return repr(data)

class InsideA(object):
def __repr__(self):
return "InsideA()"

class InsideB(object):
def __init__(self, data):
self.data = data

def __repr__(self):
data = format_data(self.data, 'x')
return "InsideB(data={0})".format(data)

def __format__(self, formatstr):
data = format_data(self.data, formatstr)
return "InsideB(data={0})".format(data)

def format_array(array, formatstr):
if array is None:
return "None"
else:
str_array = []
for elem in array:
try:
str_array += ["{0:{1}}".format(elem, formatstr)]
except (TypeError, ValueError):  # Py2 and Py3 compat
str_array += [repr(elem)]
return "[" + ", ".join(str_array) + "]"

if __name__ == "__main__":
a = InsideA()
b = InsideB(bytearray(b'test\xff\xff'))

array = [a, b]

print("Automatic:")
print(array)
print("")

print("With format:")
print(format_array(array, "x"))


Prints:

Automatic:

[InsideA(), InsideB(data=74657374ffff)]


With format:

[InsideA(), InsideB(data=74657374ffff)]


This code in format_array() seems to me to be too complex for what it does. Do I don't know of some syntax that makes this a breeze or is it the best Python can do?

(Yes, I know that I can implement __repr__ as just a call to format(), leaving that in for readability of example.)

• It appears that the code is not behaving the way you want it to, so this is a "How can I …?" question that should be asked on Stack Overflow. – 200_success Dec 13 '15 at 15:44
• @200_success: It is behaving as I want it to. I'm asking how I can make it simpler or more readable. The code definitely is not "broken" - both examples run and produce the expected output. – Hubert Kario Dec 13 '15 at 15:47

Stripping down my rather longish answer to the minimum you have two options as I see it:

• Implement a subclass of list with a dedicated __format__ option
• Use a function like format_array()

## Refactored format_array()

Your format_array() does the work but has the following issues related to doing it the Pythonic way:

• Explicit if None ... else instead of ternary – You could use return "list" if array else "None", to simplify away the if else structure
• Remove anti-pattern of list = []; for ...: list.append() – This can be replaced with a list comprehension, i.e [something(e) for e in array]. However to get to use it in your case we need to use an extra inner function to handle the try .. except clause
• Avoid using + for string comprehension – Most people like to avoid using string concatenation with pluss, and in this case you could use '[{}]'.format( ... ).

Compensating for these we get the following code:

def format_array(array, formatstr):
"""Ensure that format is called for each of the array elements."""

def format_or_repr(elem):
try:
return format(elem, formatstr)
except (TypeError, ValueError, AttributeError):
return repr(elem)

return '[{}]'.format(', '.join(map(format_or_repr, array))) if array else 'None'


Now you can do print('{}'.format(format_array(array, "x"))).

## Subclassing the list

Furthermore to allow for really neat print code, we could opt for subclassing the list with a specific __format__ or __str__ code:

class MyList(list):
def __new__(cls, data=None):
obj = super(MyList, cls).__new__(cls, data)
return obj

def __str__(self):
return 'sM: MyList({})'.format(', '.join(map(str, self)))

def __format__(self, formatstr):
return 'fM: MyList({})'.format(format_array(self, formatstr))

return MyList(list(self) + list(other))


Based on code from ""Replace str method on list object in Python". This does however require that you create your list using this class, but it does allow for a very neat syntax of print('{:x}'.format(my_list)).

This code still uses the format_array() function, but could have reimplemented it as an inner part of the function. To avoid code bloat in this bare minimum I kept it as a function call.

## Test code and output

Here is the test code:

def main():
a = InsideA()
b = InsideB(bytearray(b'test\xff\xff'))

array = [a, b]
my_list = MyList([a, b])

print('plain array:           {}'.format(array))
print('format_array:          {}'.format(format_array(array, "x")))

print('\nUsing plain my_list:   {}'.format(my_list))
print('Using myList and :x:   {:x}'.format(my_list))

if __name__ == "__main__":
main()


And here is the output:

plain array:           [InsideA(), InsideB(data=bytearray(b'test\xff\xff'))]
format_array:          [InsideA(), InsideB(data=74657374ffff)]

Using plain my_list:   MyList([InsideA(), InsideB(data=bytearray(b'test\xff\xff'))])
Using myList and :x:   MyList([InsideA(), InsideB(data=74657374ffff)])


You could of course opt to not include the MyList(...) in the returned text of the subclassing options.

• Nice, I knew that it was possible with list comprehension or iterators in format_array(). Though I will probably use maping = (format_or_repr(i) for i in array) instead of the map(). – Hubert Kario Dec 14 '15 at 11:16

list always uses repr on it's elements. When you str or repr it. And internally list is doing roughly the equivalent of what you are doing in format_array.

The only way to make this work is to use something like format_array, or subclass list. But they will both be doing the same thing, just depends on how much sugar you want.

Your code is nice, readable and follows PEP8. As this is tending towards example code, it's not in it's native environment, it's hard to comment distinctively on it.

But format_data can be merged with the class.

class InsideB(object):
def __init__(self, data):
self.data = data

def __repr__(self):
data = self._data(True)
return "InsideB(data={0})".format(data)

def __format__(self, formatstr):
data = self._data('x' in formatstr)
return "InsideB(data={0})".format(data)

def _data(self, as_hex):
if as_hex:
return binascii.b2a_hex(self.data).decode('ascii')
else:
return repr(self.data)


You can then add an option to use 'hex' or repr as the output type to the class. With the option to reverting to the original after a use. Obviously it depends on your usage.

class InsideB(object):
def __init__(self, data, as_hex=False, revert_hex=False):
self.data = data
self._as_hex = as_hex
self.output_type = as_hex
self._revert_hex = revert_hex

def _data(self, as_hex):
if as_hex or self.output_type:
data = binascii.b2a_hex(self.data).decode('ascii')
else:
data = repr(self.data)

if self._revert_hex:
self.output_type = self._as_hex

return data


Then you would need to make a function that maps over all the items in the list, changes output_type to what you want, and you'll be in the same position as you are now. But with a bit more sugar.

If you choose to subclass list, and to use str in __str__, then you'll get output which is just wrong.

>>> print('{!s}, {!s}'.format('a', 'b'))
a, b
>>> print('{!r}, {!r}'.format('a', 'b'))
'a', 'b'


No matter how I look at it you wont be able to change the output 'nicely'. Unless you think something more robust to map(lambda i: i.output_type = True, array) is nice.

Bottom line you have to manually implement this, as repr is hard-coded.

• "Bottom line you have to manually implement this," that's what I was afraid of – Hubert Kario Dec 13 '15 at 20:55
• @HubertKario It could be worse, like ~10 line helper function isn't that bad... – Peilonrayz Dec 13 '15 at 21:07

There are two major point I would like for you to focus on:

• Python uses repr on each of the elements in a list (as Joe Wallis point out in his answer), and I will examplify this and show a way around it
• Convert to using __str__ instead of __format__

Edit: It turns out the question is really about how to apply multiple different formatting options on to each of the list elements. For specifics on this, see Addendum on end of answer.

## Convert to __str__

In the following code I've converted your __format__ into a __str__ function instead. The reason for this is to adhere better to the standards used when printing and formatting different objects automatically. This approach I hope will convince you is an easier approach than doing the __format__ path.

I've also included the __repr__ class, and added a prefix to both function so that we can easily see which one is used throughout the examples.

Here is the first part of the code:

import binascii

class InsideA(object):
def __repr__(self):
return "rA: InsideA()"

def __str__(self):
return "sA: InsideA()"

class InsideB(object):
def __init__(self, data):
self.data = data

def __repr__(self):
return "rB: InsideB(data={0!r})".format(self.data)

def __str__(self):
return 'sB: {}'.format(binascii.b2a_hex(self.data).decode('ascii'))


There is nothing magic happening here other than if you do print(b) it according to your example produce "sB: 74657374fff" (with my added prefix).

## Subclassing list to avoid repr on elements

As suggested in "Replace str method on list object in Python", one can subclass a list to get it to call the __str__ method on each element. The accepted answer shows a version which:

This solution works without a wrapper. And works if you join two lists by add. Any operation that modify the list itself will work as expected. Only functions that return a copy of the list like: sorted, reversed will return the native python list which is fine. sort and reverse on the other hand operate on the list itself and will keep the type.

Here is this class updated to use str.format():

class MyList(list):
def __new__(cls, data=None):
obj = super(MyList, cls).__new__(cls, data)
return obj

def __str__(self):
return 'sM: MyList({})'.format(', '.join(map(str, self)))

return MyList(list(self) + list(other))


## Example output and explanations

To display the difference of the various options, run the following code:

def strify(list, join_text=', ', join_brackets='[]'):
"""Returns join of str(list element) with join_txt, surrounded by join_brackets."""
return '{}{}{}'.format(join_brackets[0],
join_text.join(str(elem) for elem in list),
join_brackets[1])

def main():
a = InsideA()
b = InsideB(bytearray(b'test\xff\xff'))

array = [a, b]

print('\n== Working directly on array ==')
print('Just array:   {}'.format(array))
print('[Joining]:    [{}]'.format(', '.join('{!s}'.format(elem) for elem in array)))
print('map(str):     {}'.format(map(str, array)))
print('[join & map]: [{}]'.format(', '.join(map(str, array))))
print('Using !s:     {!s}'.format(array))
print('Using !r:     {!r}'.format(array))
print('strify():     {}'.format(strify(array)))

print('\n== Working directly on *array ==')
print('Using 2 !s:   {!s} and {!s}'.format(*array))
print('Using 2 !r:   {!r} and {!r}'.format(*array))

print('\n== Using MyList class on my_list ==')
my_list = MyList([a, b])

print('Just my_list: {}'.format(my_list))
print('Using !s:     {!s}'.format(my_list))
print('Using !r:     {!r}'.format(my_list))

if __name__ == "__main__":
main()


Which produces this output:

== Working directly on array ==
Just array:   [rA: InsideA(), rB: InsideB(data=bytearray(b'test\xff\xff'))]
[Joining]:    [sA: InsideA(), sB: InsideB(data=74657374ffff)]
map(str):     ['sA: InsideA()', 'sB: InsideB(data=74657374ffff)']
[join & map]: [sA: InsideA(), sB: InsideB(data=74657374ffff)]
Using !s:     [rA: InsideA(), rB: InsideB(data=bytearray(b'test\xff\xff'))]
Using !r:     [rA: InsideA(), rB: InsideB(data=bytearray(b'test\xff\xff'))]
strify():     [sA: InsideA(), sB: InsideB(data=74657374ffff)]

== Working directly on *array ==
Using 2 !s:   sA: InsideA() and sB: InsideB(data=74657374ffff)
Using 2 !r:   rA: InsideA() and rB: InsideB(data=bytearray(b'test\xff\xff'))

== Using MyList class on my_list ==
Just my_list: sM: MyList(sA: InsideA(), sB: InsideB(data=74657374ffff))
Using !s:     sM: MyList(sA: InsideA(), sB: InsideB(data=74657374ffff))
Using !r:     [rA: InsideA(), rB: InsideB(data=bytearray(b'test\xff\xff'))]


As can be seen if you call print(array) it clearly uses repr on each of the elements, which you don't want. And specifying the conversion by !s or !r doesn't help either, as that only applies to the list it self and not the specific element.

However if you loop on the elements, and specify !s we get the wanted output. Next step would then be to do map but that introduces the quotes again. Combining map and join gets correct output, but the code isn't nice anymore, so then we can just as well create a function for it, aka strify() as given in above main() (or a variant of the original format.

An alternative to get the wanted output is to specify a formatting options for each element in the array, and then use *array. This has the obvious downside that you need to know how many elements you have and specify each of them in the format string. Not good! However you could force the str on each of the elements using map(str, array) which is possibly the nicest and easiest approach.

Lastly, you could use the subclassing of list with the caveats given above regarding sorted and reversed which would revert to native Python lists, and thusly using repr. But if using the MyList for creation, and it's not modified back to a native list, you get the proper variant when using !s and !r and you do the simple print(myList).

## Conclusion

In both my suggested solutions I would use the __str__ on the subclasses, as this helps when printing single elements as well as when used in the various other solutions.

As Python by default uses repr on each of the elements when printing a list, your best option is either to use a subclassed list which calls the str on each element, or to have shorter and clean code use a dedicated function like strify, which in the end defaults to joining str'ified version of each element.

## Addendum: To allow for multiple various format options

It turns out, see comments, that OP wants to have an option to vary the output of the formatting for each element, and then it could be viable to use the __format__ option. In that case I would simplify the format_array() to the following:

def format_array(array, formatstr, join_text=', ', join_brackets='[]'):
"""Ensure that format is called for each of the array elements."""

def format_or_repr(elem):
try:
return format(elem, formatstr)
except (TypeError, ValueError, AttributeError):
return repr(elem)

return '{}{}{}'.format(join_brackets[0],
join_text.join(format_or_repr(elem) for elem in array),
join_brackets[1]) if array else 'None'

#    return '[{}]'.format(', '.join(format_or_repr(e) for e in array)) if array else 'None'


The commented version at bottom, is if one don't want to have the default parameters of join_text and join_brackets. The gist of the simplification is simply to use a helper function to avoid exception handling, and a join(...) if array else 'None' to compress the situation if function is called without a list.

Now you can use your original format_data and it will still default nicely to using repr on those not having implemented the __format__ method.

Another version of the same using join and map, and direct index referencing of join_bracket follows:

def format_array_v2(array, formatstr, join_text=', ', join_brackets='[]'):
"""Ensure that format is called for each of the array elements."""

def format_or_repr(elem):
try:
return format(elem, formatstr)
except (TypeError, ValueError, AttributeError):
return repr(elem)

return '{0[0]}{1}{0[1]}'.format(
join_brackets,
join_text.join(map(format_or_repr, array))
) if array else 'None'

#    return '[{}]'.format(', '.join(map(format_or_repr, array))) if array else 'None'


And finally some test-code:

print("\n== Using format extension ==")
print('Just format(b, "x"):     {}'.format(format(b, "x")))
print('format_array:            {}'.format(format_array(array, "x")))
print('format_array(None):      {}'.format(format_array(None, "x")))
print('format_array_v2:         {}'.format(format_array_v2(array, "x")))
print('format_array_v2(None):   {}'.format(format_array_v2(None, "x")))


And corresponding output:

== Using format extension ==
Just format(b, "x"):     fB: InsideB(data=74657374ffff)
format_array:            [rA: InsideA(), fB: InsideB(data=74657374ffff)]
format_array(None):      None
format_array_v2:         [rA: InsideA(), fB: InsideB(data=74657374ffff)]
format_array_v2(None):   None

• It's a impressive answer, but I know that array uses repr() on elements and that's not the problem . I'm asking how to make format_array simpler because sometimes I need to encode bytearrays as hex strings and sometimes as repr() of bytearray. You can't do that with just repr or str you need to use format() parametrisation. – Hubert Kario Dec 13 '15 at 20:25
• I was looking for more of a general solution, as while for bytearrays I have just 2 options, for ints I have 4. It's really not the problem to do str() or repr()`, it's how to pass format string to array elements cleanly. – Hubert Kario Dec 13 '15 at 20:41
• @holroy I don't mean to be rude, but isn't your addendum a more complex version of OP's code? – Peilonrayz Dec 13 '15 at 21:35
• @JoeWallis, Depends on how you see it, I guess. It removes the test against None in explicit if's and adds a bit of functionality regarding choice of join_text and brackets. It also removes an anti-pattern of explicit list comprehension. And finally it doesn't string concatenation which many doesn't like. – holroy Dec 13 '15 at 21:38