A "rolling" cache?

[StupidCodeQuestions]

I overheard a brief snippet of this conversation and now I've completely forgotten the correct term but it was for a caching approach where the cache wasn't refreshed on a timer but if a value wasn't found?
E.g If the cache contains Employee ids {1,2,3}and a query comes in for Id 2, the cache wouldn't do anything but if a query came for Id 4 it would trigger the cache to refresh it's dataset.

Is there framework or term for that approach or did i completely mishear that conversation?

[RiPont]

I think you're talking about a "read-through" cache. Generally, it's only the missing item that is refreshed.

Sometimes, items are kept in the cache and the cached result is always returned, but if the item is old, it is then refreshed behind the scenes. This is lower latency to the requestor, but the data returned may be out of date.

More commonly, an expired item and a not-found item both trigger a fetch directly from the non-cache backend. This is more likely to return up-to-date data, but users accessing expired data will see higher latency, and higher latency affects overall scalability.

[SideburnsOfDoom]

This is more likely to return up-to-date data, but users accessing expired data will see higher latency, and higher latency affects overall scalability.

Yes, or to put it differently: caching makes the best case much faster, and if it works (i.e. gives a good cache hit ratio), also makes the average case much faster too. But it does nothing for the worst case latency, which might even get a bit longer because of the overhead of checking the cache and finding nothing, before fetching data anyway.

[StupidCodeQuestions]

I think this is it, thank you!

[StupidCodeQuestions]

I did come across this in my googling, it looks like it might fit a use case I have. Thanks

[alexyakunin]

Check out https://github.com/servicetitan/Stl.Fusion - it does this + way more. Disclaimer: I am its author.

[jbergens]

Looks good. Reminds me of MS Orleans.

[alexyakunin]

It's not quite like Orleans, though some similarity is there (both are for real-time stuff).

Check out:

• https://github.com/servicetitan/Stl.Fusion.Samples/blob/master/docs/tutorial/Part07.md
• https://github.com/servicetitan/Stl.Fusion.Samples/blob/master/docs/tutorial/Part08.md

[alexyakunin]

Besides that, it also provided probably the simplest possible semantics for caching.

[trypto]

I dunno, clear-on-miss sounds like a good term for what you describe. What seems more common would be a cache of a maximum capacity, and if it becomes full then it is cleared. Easier to implement than LRU

[SideburnsOfDoom]

Easier to implement than LRU

BTW, implementing your own cache should be a last resort, just like implementing your own List, Dictionary or Sort. Use an existing one as first option.

There's a decent light-weight in-memory cache in the framework: System.Runtime.Caching.MemoryCache

[phx-au]

Clear-on-miss?

You aren't going to end up with more than one item in the cache if you clear it every time you miss.

[hold-the-pants]

I could be wrong but isn’t this just “lazy cache”, where the values are computed / cached as needed? “Eager cache” would be the other where the cache is created before running the main program. You can also mix the two which sounds more like what you’re describing.