This question began as an off-topic answer to this question, but the code here serves a different goal.
I wrote the following class for the purpose of populating a dict on demand from an iterator. The intent of the iterator passed to the constructor is that it could alternatively be passed to dict
, which would consume the entire iterator in its constructor; an instance of this class consumes the iterator just far enough to locate a requested item. Such an iterator would be similar in spirit to a return value from dict.items
.
class LazyDict(dict):
""" A dict built on demand from an iterator """
def __init__(self, iterator):
super().__init__()
self.iterator = iterator
def __getitem__(self, item):
while not self.get(item):
try:
(key, value) = next(self.iterator)
self[key] = value
except StopIteration:
raise AttributeError
return super().__getitem__(item)
def __contains__(self, item):
try:
self[item] # pylint: disable=pointless-statement
return True
except AttributeError:
return False
Here is my calling code (with other details of the Directory
code omitted; if more of that is needed in this context I can provide it):
class Directory(object):
@Lazy
def hash(self):
""" Lazy dict mapping entry names to entries """
return LazyDict((self.name(entry), entry) for entry in self.readdir())
def __contains__(self, name):
return name in self.hash # pylint: disable=unsupported-membership-test
def __getitem__(self, name):
return self.hash[name] # pylint: disable=unsubscriptable-object
The code for Lazy
is equivalent to lazy_property
in this answer.
As I write this, it occurs to me that perhaps Directory
should itself be a subclass of LazyDict
rather than containing a LazyDict
(that might be a better way to stifle those pylint
warnings). Whether that seems right might be one specific question to fall out of this. Upon further investigation, here is an alternate version of the calling code:
class Directory(LazyDict):
def __init__(self, mem, size):
super().__init__((self.name(entry), entry) for entry in self.readdir())
self.mem = mem
self.size = size
As before, some details of Directory
are omitted (the implementations of name
and readdir
are not included for either version), but this version of the code inherits __getitem__
and __contains__
from LazyDict
rather than overriding them. The most visible difference between the two versions of the calling code is due to Directory
inheriting __repr__
from dict
rather than object
.
Another specific question might be the role of keys
and values
and items
in LazyDict
. The methods inherited from dict
reveal how much of the iterator has been consumed; convincing them to reveal all items is one AttributeError
away (I can attempt to fetch self[None]
within Directory
to fully populate the cache). My inclination is to limit on-demand operations to __getitem__
and __contains__
, but opinions on that point are welcome.
Suggestions for other ways to approach this are also welcome.
Directory
class mentioned here is part of a module which lists contents of asquashfs
image.Directory.__getitem__
is used for directory lookups. If onlyreaddir
is called on a directory (to list all of its entries), there is no need to populate the cache used for lookups within that directory. In a large directory, the time saved by not populating that cache unnecessarily might be significant. \$\endgroup\$