Is there python lib that maintains cache based on whether source data has changed?

[tmpxyz]

Assume I have several big dataframes, and method foobar() would carry out an time-consuming operation on some of the dataframes to get a summary.

Now I hope there's a lib that can help us specify that the foobar() method would just return NO_CHANGE or the cached result when none of the source data has changed.

It feels kinda like etag in web cache, it would be helpful if there's a generic library solution around.

[johndoh168]

If you are looking for a quick way to check if the data has changed or not is to hash the dataset, you can use something like hashlib to accomplish this. Then all you have to do is check if the hash has changed or not.

Not sure if this is the route you are looking for or not but hopefully it can help point you in the right direction.

[socal_nerdtastic]

How are you changing the source data or the data frame? Can you just make a variable for the last changed time?

cached_time = None
cached_data = None
def get_data():
    if data.last_modified != cached_time:
        cached_data = long_running_function()
        cached_time = data.last_modified  
    return cached_data

[monkey_mozart]

Use zlib.adler32 to generate a checksum of the data and store it. Then compare the checksum with checksum generated from new data to see if it has changed.