2
\$\begingroup\$

The goal is to write a decorator that allows you to define "indexer proxies", which define properties with __getindex__ instead of normal method calls, like loc and iloc in Pandas dataframes.

To that end I ended up with a descriptor constructing an inner class with the transformed __getindex__, an instance of which is then constructed in __get__:

from functools import wraps

class locator:
    def __init__(self, getter):
        class LocatorProxy:
            def __init__(self, obj):
                self.obj = obj
            
            @wraps(getter)
            def __getitem__(self, key):
                return getter(self.obj, key)

        self.proxy_class = LocatorProxy

    def __get__(self, obj, objtype=None):
        return self.proxy_class(obj)

Example usage:

In [41]: class Foo:
    ...:     def __init__(self, **vals):
    ...:         self.vals = vals
    ...:     @locator
    ...:     def loc(self, key):
    ...:         """Return important stuff"""
    ...:         return self.vals[key]
    ...: 

In [42]: Foo(x = 1, y = 10).loc["y"]
Out[42]: 10

In [43]: Foo(x = 1, y = 10).loc["z"]
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-43-d91ef97fd1fe> in <module>
----> 1 Foo(x = 1, y = 10).loc["z"]

<ipython-input-40-ca29fed22d1f> in __getitem__(self, key)
      9             @wraps(getter)
     10             def __getitem__(self, key):
---> 11                 return getter(self.obj, key)
     12 
     13         self.proxy_class = LocatorProxy

<ipython-input-41-fb4c4b7f731c> in loc(self, key)
      5     def loc(self, key):
      6         """Return important stuff"""
----> 7         return self.vals[key]
      8 

KeyError: 'z'

In [44]: Foo(x = 1, y = 10).loc["x"]
Out[44]: 1

I'd like to know whether this is an idiomatic usage of descriptors. Is this the right way to avoid unnecessary allocations/definitions of the inner class as much as possible? Does @wraps do the right thing? Can I do anything useful with __set_name__?

Only the getters are needed in my case, so no .setter thing like property has it is necessary for now, although it would be nice to see how it's done.

\$\endgroup\$

1 Answer 1

2
\$\begingroup\$

Is this the right way to avoid unnecessary allocations/definitions of the inner class as much as possible?

This smells like premature optimization. If you actually care about every microsecond, you shouldn't be doing any of this, and should just have regular class methods.

That said, no, your approach doesn't avoid as many allocations as it could. Rather than one allocation per __get__, you can restructure your code to have one allocation per @locator decorator instance.

Does @wraps do the right thing?

Probably, but it isn't strictly needed.

The following suggested code has no nested class definitions, no class definitions in closure scope, no @wraps calls, and attempts to brow-beat mypy into understanding your types. This last part is only partially successful.

from typing import Any, Callable, Generic, Optional, Type, TypeVar

InstanceType = Any  # mypy is too stupid to infer this type as a generic
KeyType = TypeVar('KeyType')
ValueType = TypeVar('ValueType')
LocatorCallback = Callable[[InstanceType, KeyType], ValueType]


class LocatorProxy(Generic[KeyType, ValueType]):
    __slots__ = ('method', 'instance')

    def __init__(self, method: 'LocatorCallback'):
        self.method = method
        self.instance: Optional[InstanceType] = None

    def __getitem__(self, item: KeyType) -> ValueType:
        assert self.instance is not None
        return self.method(self.instance, item)


class locator(Generic[KeyType, ValueType]):
    __slots__ = ('proxy_class',)

    def __init__(self, method: LocatorCallback) -> None:
        self.proxy_class = LocatorProxy[KeyType, ValueType](method)

    def __get__(
        self,
        instance: InstanceType,
        owner: Optional[Type[InstanceType]] = None,
    ) -> LocatorProxy:
        self.proxy_class.instance = instance
        return self.proxy_class


class Foo:
    def __init__(self, **vals: int) -> None:
        self.vals = vals

    loc: LocatorProxy[str, int]

    @locator  # type: ignore  # you really need to shoehorn this thing in
    def loc(self, key: str) -> int:
        return self.vals[key]


def test() -> None:
    foo = Foo(x=1, y=10)
    l = foo.loc
    assert l["y"] == 10

    try:
        l["z"]
        raise AssertionError()
    except KeyError:
        pass


if __name__ == '__main__':
    test()
\$\endgroup\$
2
  • \$\begingroup\$ Is the apparently superfluous loc: LocatorProxy[str, int] in Foo also there only to satisfy mypy? \$\endgroup\$ Oct 30, 2021 at 9:35
  • \$\begingroup\$ @phipsgabler Basically yes. In the end it is handy to have this even if you don't use mypy - for example, PyCharm can then successfully statically analyse your code, meaningfully offer member dropdowns, etc. \$\endgroup\$
    – Reinderien
    Oct 30, 2021 at 14:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.