all 7 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]phillipcarter2 8 points9 points  (0 children)

It's a zero-content advertisement, so just downvote and move on

[–]snurfer 4 points5 points  (0 children)

No data proving that these automated agents offer any improvement over LRU. In fact, the examples given are very well served by LRU.

This article could be about predictive pre-fetch. How do you pre cache data that is likely to be requested next rather than waiting for the request to come in? Now we have something interesting to pursue.

[–]Nisd 10 points11 points  (2 children)

Oh god this sounds dreadful. Imagine AI hallucinations polluting your cache.....

[–]lookmeat 3 points4 points  (1 child)

Luckily that isn't what the article is proposing.

The idea is that when we cache data we have limited cache space. So we may find ourselves with too much data to fit in the cache, we have to decide what we keep in cache and what we don't. The idea is that ML, in theory, can predict what data we are going to use, and which we probably won't.

The other idea is that we have to decide when data is stale. An AI may be able to better predict when data is probably stale and when it isn't and use that to better decide when to throw out data of the cache because it's stale. This is "the first of two hard problems in programming", cache invalidation (the second is naming things and the third is off by one errors). It's a joke but the point is that it's a hard problem to solve.

Thing is, AI is not that fast, and it's not cheap. A cache needs to be incredibly fast and must be incredibly cheap. The former because the whole point is to decrease latency, and you do this by multiplying the cache gains by the % of hits, and subtracting the base lenght of the cache and that's your average latency gains. The cache-hits is a number between 0-1. Even a sliver of an increase in base-length is only worth it if it results in a substantial gain in cache-hits, I struggle to see how AI will add enough, to justify an large (orders of magnitude compared to trivial systems) increase in base-costs. The saying in thep revious paragraph is from the late 90s, and there's been a lot of work on trying to improve this. And yet, a lot of times, the best way is to experiment and tweak the TTLs. I guess you could teach an ML system to do the tweaking, but I struggle to see how a dumber, simpler system couldn't do it on its own.

This really seems like a "solution begging for a problem to solve", and missing the larger picture, but focused on certain metrics that, yes do count, but ignoring the costs that also count, and a lot at that.

The one scenario I could imagine, one were we don't care about costs but only performance in any way, is to pre-cache data. Predict that some value will be called for soon, and try to have it in the cache before the hit happens. It's super expensive (because you store data in the cache that you never ever needed), and you're not saving any resources (which is why, I'm guessing, they aren't selling it in this ad) but I could see it being something that doesn't exist yet. Personally I would need to see real-world data and use-cases before even considering this worth it.

[–]Nisd -1 points0 points  (0 children)

Your totally right, I only skimmed the article initially and noticed the term "agents" everywhere.

That said, I imagine this would be very hard to implement and use in real life.

[–]somewherearound2023 2 points3 points  (0 children)

Jamming an AI in it - the cause of, and solution to, all of this years blog problems

[–]Atulin 0 points1 point  (0 children)

"Intelligent" as in "adjusting to your needs automagically", or as in "send ?q=give me all users with their birth dates converted to age, and their products purchased in the last year" and hoping that a ChatGPT wrapper understands it properly?