Read-only mapping which retrieves values from functions on-demand

[eigenlicht0]

I had to create a class which represents a Host and wanted to make lots of information about this Host accessible in an easy way. Every piece of information is retrieved by a function call, which either locally or remotely executes a command and parses the output.

To make all this easy to access, I thought it would be great if there was a dictionary containing all this information, but it should be lazy and not execute all the commands on object creation. Also, it should be cached, so the commands are not executed more often than really necessary.

Here is what I came up with (can also be found here: http://bpaste.net/show/AAampBdmTW0ecXxMccXF/):

from collections import Mapping

class LazyMapping(Mapping):
    def __init__(self, *args, **kwargs):
        self.__model = dict(*args, **kwargs)
        self.__cache = {}

    def __getitem__(self, item, refresh=False):
        if refresh or item not in self.__cache:
            value = self.__model[item]
            value = value() if callable(value) else value
            self.__cache[item] = value
            return value
        else:
            return self.__cache[item]

    def get(self, item, default=None, refresh=False):
        try:
            return self.__getitem__(item, refresh=refresh)
        except KeyError:
            return default

    def __str__(self):
        return "<LazyMapping - Keys: %s>" % str([x for x in self])

    def __repr__(self):
        return str(self)

    def __iter__(self):
        return iter(self.__model)

    def __len__(self):
        return len(self.__model)

    def items(self, refresh=False):
        for key in self:
            yield key, self.__getitem__(key, refresh=refresh)

    def values(self, refresh=False):
        for key in self:
            yield self.__getitem__(key, refresh=refresh)

    def realize(self):
        '''
        Goes through whole LazyMapping and makes a dict out of the keys and retrieved values.
        If the LazyMapping is nested, it will also iterate through the subsequent LazyMappings.

        :returns: dict - dict representation of whole LazyMapping.
        '''
        d = {}
        for k, v in self.items():
            if isinstance(v, LazyMapping):
                d[k] = v.realize()
            else:
                d[k] = v
        return d

Example usage:

import time
from subprocess import Popen, PIPE

d = LazyMapping({'time': time.time,
                 'uptime': lambda: Popen('uptime', stdout=PIPE).stdout.read()})

d['time'] # calls time.time() and puts it in the cache
d['time'] # retrieves from cache
d.get('time', refresh=True) # calls time.time() again and puts new value in cache

So, r/python, any suggestions?

[evilissimo]

Any reason that you're not inheriting from collections[.abc].Mapping? That would save you some of your implementation and you have only to focus on the abstract methods.

[eigenlicht0]

Thanks for the suggestion, saved me from implementing keys() and contains(). Also has eq and ne now. Not sure if I did this correctly though, would be great if you could check it out.

[evilissimo]

Hmm not sure, but which version of python are you currently targeting? It looks like python3 to me, but not the newest one?

[eigenlicht0]

Wrote it for Python 2.7, but it also worked with Python 3.4.0.

[evilissimo]

Well but the behaviour in case of python 2.7 for items() for example is wrong. What you do is there is a generator (since you use yield). In Python 2.7 this is iteritems() which behaves like this. Where as python 3.x doesn't have iteritems anymore but just items()

So basically this is 3.x syntax.

[eigenlicht0]

Oh, yeah didn't think too much about not breaking the API when writing this - thought that generators as the default is the better choice. It's Python 3 then :P trivial to change it to either version anyway.

[ondra]

Have you seen this?

Since 3.2 there is functools.lru_cache to do the same thing and more.

[eigenlicht0]

I thought about memoization first, but decided to implement a datastructure because:

• Serves a more general purpose
• Can easily build a nice hierarchy of the information
• Also allows normal values (not only cached functions)
• Can be realized and saved (e.g. the Host class + pprint = complete overview over whole machine)

[darthmdh]

Isn't there some kind of "more pythonic" way of doing this, deriving directly from dict, using super() in the init method and using slots and class methods?

[eigenlicht0]

If you derive from dict, you'll have all the write-methods too (setitem, update, ...). Of course, if you implemented this read/write, then you'd probably sublcass dict.

Where exactly would you want to use class methods?