10
\$\begingroup\$

I have a Python function w/ specified functionality, and two proposed approaches - which do you prefer, and why? (performance, simplicity, clarity, etc) (both use import random)


PROBLEM: Need a plain Python function that accepts an arbitrary number of iterables (tuples, lists, dictionaries), and returns them shuffled in the same order:

a = (1, 2, {3: 4}, 5)
b = [(5,6), [7,8], [9,0], [1,2]]
c = {'arrow': 5, 'knee': 'guard', 0: ('x',2)}

x, y, z = magic(a, b, c)
print(x, y, z, sep='\n')
# ({3: 4}, 1, 2)
# [[9, 0], (5, 6), [7, 8]]
# {0: ('x', 2), 'arrow': 5, 'knee': 'guard'}

The function must:

  1. Return iterables shuffled in the same order (see above)
  2. Accept any number of iterables
  3. Preserve iterables types
  4. Support nested iterables of any depth and type
  5. Not shuffle nested elements themselves (eg. [7,8] above doesn't become [8,7])
  6. Return iterables with length of shortest iterable's length w/o raising error (see above)
  7. Return shuffled iterables in the same order they're passed in (see above)

NOTE: requires Python 3.7. Can work with Python 3.6- if excluding dicts, as dicts aren't ordered.


SOLUTION 1:

def ordered_shuffle(*args):
    args_types = [type(arg) for arg in args]                               # [1]
    _args      = [arg if type(arg)!=dict else arg.items() for arg in args] # [2]
    args_split = [arg for arg in zip(*_args)]                              # [3]
    args_shuffled = random.sample(args_split, len(args_split))             # [4]
    args_shuffled = map(tuple, zip(*args_shuffled))                        # [5]
    return [args_types[i](arg) for i, arg in enumerate(args_shuffled)]     # [6]

Explanation: img


SOLUTION 2:

def shuffle_containers(*args):
    min_length = min(map(len, args))
    idx = list(range(min_length))
    random.shuffle(idx)
    results = []
    for arg in args:
        if isinstance(arg, list):
            results.append([arg[i] for i in idx])
        elif isinstance(arg, tuple):
            results.append(tuple(arg[i] for i in idx))
        elif isinstance(arg, dict):
            items = list(arg.items())
            results.append(dict(items[i] for i in idx))
        else:
            raise ValueError(
                "Encountered", type(arg),
                "expecting only list, dict, or tuple"
            )
    return results
\$\endgroup\$
  • \$\begingroup\$ Heads up: the edit you reverted has been made by a diamond moderator here to clarify the extent of the code up for review in your question. I strongly advise you to not roll it back. Thanks! \$\endgroup\$ – Vogel612 Oct 2 at 15:57
  • \$\begingroup\$ @Vogel612 It is up for review - it resulted from someone improving Sol 1, so someone could also improve GZ0 mod or comment on its performance/readability \$\endgroup\$ – OverLordGoldDragon Oct 2 at 16:10
  • \$\begingroup\$ if you want to put it up for review, please understand that it's not allowed on this site to update the question in a way that invalidates existing answers. See also what you may and may not do after receiving answers \$\endgroup\$ – Vogel612 Oct 2 at 16:12
7
\$\begingroup\$

While I do agree with others that Solution 2 is more readable with some improvements, there are also a few improvements that can be done on Solution 1.

  • It is unnecessary to construct lists from iterables (e.g., generator expressions) when all that is needed is an iterable. For example,

    _args      = [arg if type(arg)!=dict else arg.items() for arg in args]
    args_split = [arg for arg in zip(*_args)]
    

    Here, the unpacking operator * works on arbitrary iterables. So one can just do

    _args      = (arg if type(arg)!=dict else arg.items() for arg in args)
    args_split = [arg for arg in zip(*_args)]
    

    The parantheses keep the generator expressions without actually materializing them into lists.

  • It is better to use isinstance(arg, cls) rather than type(arg) == cls

  • Unpacking an iterable into a list can be done using list(iterable), which is more efficient than a list comprehension [arg for arg in iterable] that uses an explicit for-loop.
  • This expression

    [args_types[i](arg) for i, arg in enumerate(args_shuffled)]
    

    can be rewritten using zip to avoid the need of indices:

    [cls(arg) for cls, arg in zip(args_types, args_shuffled)]
    

Following is an improved version of Solution 1

def ordered_shuffle(*args):
    arg_types = map(type, args)
    arg_elements = (arg.items() if isinstance(arg, dict) else arg for arg in args)
    zipped_args = list(zip(*arg_elements))
    random.shuffle(zipped_args)
    return [cls(elements) for cls, elements in zip(arg_types, zip(*zipped_args))]
\$\endgroup\$
  • \$\begingroup\$ A shorter version, maybe shortest, for anyone interested; 42 chars shorter than Gloweye's lambda \$\endgroup\$ – OverLordGoldDragon Oct 2 at 4:48
  • 1
    \$\begingroup\$ @OverLordGoldDragon For pure golfing there are more tricks that can be applied. Here is an example of what regular golfed code would look like. \$\endgroup\$ – GZ0 Oct 2 at 5:01
  • \$\begingroup\$ Selected as the answer, but other answers are valuable also - selection rationale in my answer. Thanks for your solution \$\endgroup\$ – OverLordGoldDragon Oct 3 at 19:19
11
\$\begingroup\$

functools.singledispatch

functools library includes the singledispatch() decorator. It lets you provide a generic function, but provide special cases based on the type of the first argument.

import functools
import random

@functools.singledispatch
def shuffle(arg, order):
    """this is the generic shuffle function"""

    lst = list(arg)
    return type(arg)(lst[i] for i in order)


@shuffle.register(dict)
def _(arg, order):
    """this is shuffle() specialized to handle dicts"""

    item = list(arg.items())
    return dict(item[i] for i in order)


def ordered_shuffle(*args):
    min_length = min(map(len, args))

    indices = random.sample(range(min_length), min_length)

    return [shuffle(arg, indices) for arg in args]

Usage:

a = (1, 2, {3: 4}, 5)
b = [(5,6), [7,8], [9,0], [1,2]]
c = {'arrow': 5, 'knee': 'guard', 0: ('x',2)}

ordered_shuffle(a, b, c)

Output:

[({3: 4}, 1, 2),
 [[9, 0], (5, 6), [7, 8]],
 {0: ('x', 2), 'arrow': 5, 'knee': 'guard'}]
\$\endgroup\$
  • \$\begingroup\$ Interesting alternative - wonder how much fancy Python's hoarding that I've never seen. It is, however, slowest compared w/ others according to my benchmark (see updated answer) - but to be fair, for most uses, it's entirely negligible \$\endgroup\$ – OverLordGoldDragon Oct 2 at 3:49
  • \$\begingroup\$ @Vogel612 Read the rules, fair enough - I'm willing to post removed edits in an answer \$\endgroup\$ – OverLordGoldDragon Oct 2 at 20:25
7
\$\begingroup\$

I think your solution 2 is heading the right direction here. What I consider it's advantages over solution 1:

  1. It is much more readable
  2. It clearly shows that every member of *args is treated the same.

You might want to generic-ify it a bit more to handle more types. For example, the following has a good chance of also handling custom container types:

import collections
# -- Other code ---
        if isinstance(arg, collections.abc.Mapping):
            items = list(arg.items())
            results.append(type(arg)(items[i] for i in idx))
        else:
            results.append(type(arg)(arg[i] for i in idx))

Which will inspect the type of the iterable and feed it an iterator in an attempt to create a new one. This version here will handle lists and tuples the same as your does. If any custom container type supports a __init__(self, *args) constructor to fill itself, it'll also work.

To be honest, those will probably be very rare. I've never seen one. But this is a very easy generic to support, and it's already worth it because you have the same code handling tuples and lists, in my opinion.

Keep in mind you need at least python 3.6 for dictionaries to have a stable insertion ordering.

Readability > Shortness.

Unless you're planning on visiting Codegolf, don't do this. But if you do, you can shorten your functions a lot more, like this:

f = lambda *a:[type(y[i])(b) for i,bin enumerate(map(list, zip(*random.sample([y for y in 
    zip(*[x if type(x)!=dict else x.items() for x in a])], min(len(z) for z in a)))))]

I hope this example makes it abundantly clear why readability is important. If not, try and figure out if there's a bug in here.

\$\endgroup\$
  • 2
    \$\begingroup\$ Readability doesn't have much to do with "short" once you're above amateur programmer. The readability means that in solution 2, I can see what happens easily, and just as easily understand what happens. It requires no backtracking, no re-reads. I've read solution 1 ten times, and without looking up documentation on random.sample, I wouldn't be sure what it does. I've read it three times just for this comment to make sure I really got what happened. \$\endgroup\$ – Gloweye Oct 1 at 19:07
  • 1
    \$\begingroup\$ Yes, preference matters a lot in cases of readability. List comprehensions and related things are totally awesome tools in python (and I use them a lot), but they can kill readability just as fast as code golf does. \$\endgroup\$ – Gloweye Oct 1 at 19:37
  • 1
    \$\begingroup\$ Done for the question, but I have the same readability concerns about that answer that I raised here. \$\endgroup\$ – Gloweye Oct 1 at 20:05
  • 1
    \$\begingroup\$ Checking for isinstance(arg, collections.abc.Mapping) might be more appropriate, so that the code will work for any mapping even if it's not a subclass of dict. \$\endgroup\$ – Ilmari Karonen Oct 2 at 9:39
  • 1
    \$\begingroup\$ Good one. Added in. \$\endgroup\$ – Gloweye Oct 2 at 9:45
0
\$\begingroup\$

Below is an answer highlight and performance comparison of various solutions. Note: the differences should be negligible in practice - if otherwise, timing variance is substantial, and all are ~on par (per benchmark)


SOLUTION 1 GZ0-Mod+:

def ordered_shuffle(*args):
   zipped_args = list(zip(*(a.items() if isinstance(a, dict) else a for a in args)))
   random.shuffle(zipped_args)
   return [cls(elems) for cls, elems in zip(map(type, args), zip(*zipped_args))] 

UPDATE: Ran a performance benchmark - results for iterations = 1e7, and a, b, c = tuple, list, dict of length 10, nested, respectively:

  • #1: Solution 1 GZ0-Mod+   114.2 sec, 71.2% faster than slowest
  • #2: Solution 2   ---------- 131.9 sec, 48.0% faster than slowest
  • #3: Solution 1   ---------- 155.7 sec, 25.4% faster than slowest
  • #4: Gloweye's lambda  --- 160.5 sec, 21.7% faster than slowest
  • #5: RootTwo's functools -- 195.3 sec

Selected answer rationale:

  • Best minimalism, acceptable readability
  • Uses plain Python - no high-level libraries; easier to debug & understand
  • Well-explained answer, describing pitfalls of original approach & how it's remedied
  • Best performance (a bonus, not major factor)
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.