2
\$\begingroup\$

I'm implementing deep_get functionality to look inside arbitrarily nested Python 2.7 objects. Primarily for further logging.

This turned out to have surprising amount of quirks. Here's what I ended up with, would appreciate the feedback as I probably missed a few more things.

# coding=utf-8
from __future__ import unicode_literals
import collections

_default_stub = object()


def deep_get(obj, path, default=_default_stub, separator='.'):
    """Gets arbitrarily nested attribute or item value.

    Args:
        obj: Object to search in.
        path (str, hashable, iterable of hashables): Arbitrarily nested path in obj hierarchy.
        default: Default value. When provided it is returned if the path doesn't exist.
            Otherwise the call raises a LookupError.
        separator: String to split path by.

    Returns:
        Value at path.

    Raises:
        LookupError: If object at path doesn't exist.

    Examples:
        >>> deep_get({'a': 1}, 'a')
        1

        >>> deep_get({'a': 1}, 'b')
        LookupError: {'a': 1} has no element at 'b'

        >>> deep_get(['a', 'b', 'c'], -1)
        'c'

        >>> deep_get({'a': [{'b': [1, 2, 3]}, 'some string']}, 'a.0.b')
        [1, 2, 3]

        >>> class A(object):
        >>>     def __init__(self):
        >>>         self.x = self
        >>>         self.y = {'a': 10}
        >>>
        >>> deep_get(A(), 'x.x.x.x.x.x.y.a')
        10

        >>> deep_get({'a.b': {'c': 1}}, 'a.b.c')
        LookupError: {'a.b': {'c': 1}} has no element at 'a'

        >>> deep_get({'a.b': {'Привет': 1}}, ['a.b', 'Привет'])
        1

        >>> deep_get({'a.b': {'Привет': 1}}, 'a.b/Привет', separator='/')
        1

    """
    if isinstance(path, basestring):
        attributes = path.split(separator)
    elif isinstance(path, collections.Iterable):
        attributes = path
    else:
        attributes = [path]

    for i in attributes:
        try:
            success = False
            # 1. access as attr
            try:
                obj = getattr(obj, i)
                success = True
            except (AttributeError, TypeError, UnicodeEncodeError):
                # 2. access as dict index
                try:
                    obj = obj[i]
                    success = True
                except (TypeError, AttributeError, IndexError, KeyError):
                    # 3. access as list index
                    try:
                        obj = obj[int(i)]
                        success = True
                    except (TypeError, AttributeError, IndexError, KeyError,
                            UnicodeEncodeError, ValueError):
                        pass

            if not success:
                msg = "{obj} has no element at '{i}'".format(obj=obj, i=i)
                raise LookupError(msg.encode('utf8'))

        except Exception:
            if _default_stub != default:
                return default
            raise

    return obj

Update: Revised version with history https://gist.github.com/groz/f1838404d48971cc145609c226fdc6a2

\$\endgroup\$
2
\$\begingroup\$

Docstring

Your docstring is nice and descriptive. However, you have examples that look like doctests but they fail as doctests. One problem is that outputs contain Unicode strings, which must be written as u'abc'. Another problem is that expected exceptions have to be indicated like this. Also, the definition of class A needs to be written using ... continuation lines.

Implementation

Flag variables are usually a bad idea. You don't need success at all here.

You want to try three lookup methods until one succeeds. It could be done more elegantly using iteration rather than nesting.

# coding=utf-8
from __future__ import unicode_literals
import collections
import operator

_default_stub = object()


def deep_get(obj, path, default=_default_stub, separator='.'):
    """Gets arbitrarily nested attribute or item value.

    Args:
        obj: Object to search in.
        path (str, hashable, iterable of hashables): Arbitrarily nested path in obj hierarchy.
        default: Default value. When provided it is returned if the path doesn't exist.
            Otherwise the call raises a LookupError.
        separator: String to split path by.

    Returns:
        Value at path.

    Raises:
        LookupError: If object at path doesn't exist.

    Examples:
        >>> deep_get({'a': 1}, 'a')
        1

        >>> deep_get({'a': 1}, 'b')
        Traceback (most recent call last):
            ...
        LookupError: {u'a': 1} has no element at 'b'

        >>> deep_get(['a', 'b', 'c'], -1)
        u'c'

        >>> deep_get({'a': [{'b': [1, 2, 3]}, 'some string']}, 'a.0.b')
        [1, 2, 3]

        >>> class A(object):
        ...     def __init__(self):
        ...         self.x = self
        ...         self.y = {'a': 10}
        ...
        >>> deep_get(A(), 'x.x.x.x.x.x.y.a')
        10

        >>> deep_get({'a.b': {'c': 1}}, 'a.b.c')
        Traceback (most recent call last):
            ...
        LookupError: {u'a.b': {u'c': 1}} has no element at 'a'

        >>> deep_get({'a.b': {'Привет': 1}}, ['a.b', 'Привет'])
        1

        >>> deep_get({'a.b': {'Привет': 1}}, 'a.b/Привет', separator='/')
        1

    """
    if isinstance(path, basestring):
        attributes = path.split(separator)
    elif isinstance(path, collections.Iterable):
        attributes = path
    else:
        attributes = [path]

    LOOKUPS = [getattr, operator.getitem, lambda obj, i: obj[int(i)]]
    try:
        for i in attributes:
            for lookup in LOOKUPS:
                try:
                    obj = lookup(obj, i)
                    break
                except (TypeError, AttributeError, IndexError, KeyError,
                        UnicodeEncodeError, ValueError):
                    pass
            else:
                msg = "{obj} has no element at '{i}'".format(obj=obj, i=i)
                raise LookupError(msg.encode('utf8'))
    except Exception:
        if _default_stub != default:
            return default
        raise
    return obj

Consider requiring path to be an iterable that is already split by the caller.

\$\endgroup\$
  • \$\begingroup\$ Nice. Thanks for lots of useful info! I need to remember to re-run code in docstring when I'm updating the code. What's the name convention for LOOKUPS, thought the uppercased constants should be defined at the module level. What do you think about _default_stub trick? Doesn't look very elegant to me, any alternatives? \$\endgroup\$ – Grozz Aug 28 '16 at 9:56
  • \$\begingroup\$ Also that refactoring combining lookups does not exactly preserve the exception logic of corresponding accessors, which swallow only AttributeError and the like from the accessors in case default is provided. \$\endgroup\$ – Grozz Aug 28 '16 at 11:48
  • \$\begingroup\$ Updated the code accordingly. Not sure what's the policy here regarding updates. \$\endgroup\$ – Grozz Aug 28 '16 at 14:02
  • 1
    \$\begingroup\$ Don't update the question based on advice from answers. What you may and may not do after receiving answers. \$\endgroup\$ – 200_success Aug 28 '16 at 14:06
  • \$\begingroup\$ Understood. Moved it to gist. \$\endgroup\$ – Grozz Aug 28 '16 at 14:49
-1
\$\begingroup\$

Recursion. It would make the code smaller, easier to understand, way easier to implement as well. What you are trying to do is one of the tasks recursion is very good at.

How can I build a recursive function in python?

Basics of Recursion in Python

\$\endgroup\$
  • \$\begingroup\$ Would really like to know why I get downvoted \$\endgroup\$ – RetroCode Aug 27 '16 at 23:10
  • 2
    \$\begingroup\$ I'm not the downvoter. However, unless you can demonstrate that recursion actually results in improved code, it's not really useful advice. \$\endgroup\$ – 200_success Aug 27 '16 at 23:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.