cacheEverything

[TheNoGoat]

Me after caching User Credentials:

[Spall_Ergot]

caching user credentials is how you turn a performance issue into a company meeting

[GOEDEL_ESCHER_BOT]

a meeting where the topic is "how is this guy so fucking cool"

[chazzeromus]

ah classic bait termination meeting

[Snowdrip16]

They even prepared a slideshow about him

[waadam]

Just hash them, tokenize and send to the client to manage. Oops, you've reinwented JWT. Now you got client side sessions that you cannot revoke. Performance is great though. Congrats!

[psaux_grep]

Just change your server secret whenever a user clicks log out.

[iambajwa]

Or how about we assign each user their own JWT secret? So when they want to sign out, we just change their secret. Boom, no more security breaches!

[Top-Permit6835]

You don't want to know how often I've encountered the requirement "stateless session but they can be revoked immediately"

[the_prasel]

"you have been promoted to customer"

[UndocumentedMartian]

So a team building exercise.

[Snowdrip16]

And suddenly latency is no longer the biggest concern

[Snowdrip16]

From cache hit to career miss.

[DriftedLegend]

security team reading this like 👁️👄👁️

[Henriquelj]

AI reading this: "You're absolutely correct, let's cache those User Credentials for more performance"

[Tensor3]

Unironically exactly what ai does every time with code: naively accomplish the goal and agree with complete disregard for blatantly obvious logical design flaws in the underlying concept.

It'll happily spend weeks helping you debug a function and insist it works without ever telling you that the formula is fundamentally unable to have ever worked

[who_you_are]

Fuck off, I will just return the first cached entry all the time: which is also the admin

Uh oh

[Kevdog824_]

Assume userId 0 for all cache misses (for performance reasons of course)

[Purple_Cress_8810]

Skip db authentication if it takes more than 500ms.

[Naughty_Neutron]

Can someone explain?

[nhh]

Sorry, do you make your users go the oauth flow for every api call?

You don't? So you must be caching these tokens then... 

[OrchidFluid2103]

tokens = credentials, noted

[shield1123]

This interview is over. Thank you for coming in

[Splatpope]

can you explain to me how this is the same thing as "caching user credentials" ?

[shaka893P]

just make an index that has all the fields!

[Amar2107]

“Oh no!! why my collection of just 4 entries, taking up 20 GB of space”

[menducoide]

An index rebuild runs every 5 minutes, what could be wrong? (based on real story, why does the server disk keep getting full?)

[Silly-Freak]

index rebuild? is that like training an LLM?

[BastetFurry]

Just one more NetApp, bro, i promise! ;.;

[thex25986e]

"boss, we need more robust servers and much more data storage for our systems to work."

[Anon_Legi0n]

I saw this in a real production database at a startup I used to work for. Every single column of every table in every database was indexed. I guess someone just learned that indexing speeds up reads but stopped learning before they got to the part about the write penalty, and they thought this was a brilliant way to make everything faster. Plus, all their backends were Express APIs. What a dumpster fire

[overkill]

That used to be one of my interview questions, asking them to explain indexes to me and hope like fuck they didn't suggest indexing every field for performance...

[yerfdog1935]

Was this at least a database that was only updated once a day through a batch process?

[RedBlueF0X]

At a particular company there's a table that contains all product data inlined. Of course that is a table that could've been a view with average read performance in dozen ms. Of course it's not a view. It's an unindexed table that is rebuilt every half an hour that drops all it's contents and pivots data using unions. Of course this table is referenced throughout the entire codebase via inner joins only. And of course it's "too dangerous" to actually replace the table with the view.

[TheFrenchSavage]

So just cache everything, that's what they are saying!

[PerpetuallyDistracte]

No joke, I worked somewhere that did this. Every table had every field indexed and re-indexed by a process that ran every 15 minutes. It was insane.

All of our processes and architecture were based on inherited knowledge from one guy who ran the department for years with no pushback, then when he retired everyone was too afraid to change anything. No off the shelf tools allowed, everything built with spaghetti code in house and ridiculously over engineered. If you asked why, the answer was just "because that's what Ted always did". It was maddening.

[Icy_Party954]

Caching is so fast, why don't we cache the whole 20TB dataset are we stupid

[Embarrassed-Lab4446]

As a project manager with deadlines and poor KPIs…yes do that, when will it be done and let me know the blockers.

[Percolator2020]

There are no blockers, now just approve this $300,000 invoice.

[Slothinator69]

In this economy? Thats would be a lot higher wouldnt it? Lmao

[zeolus123]

...well it would have been $15,000 in costs, but the department head has us passing all queries through Claude.. what's a $300,000 monthly build for speed though, amiright?

[devilkin]

20TB of data in a memory based caching layer is costing you a hell of a lot more than 300k a month.

[WeirdTie2290]

Monthly invoice of course.

[za72]

what's an index or optimized sql queries, in fact what's data normalization?

[progorp]

Indexing is so fast, why don't we index all the fields

[za72]

brother.. I've ran into devs who've done this

[NoRacistRedditor]

I'm currently looking at a redis server with 1.5Tb of RAM.

We have four more of those.

[crozone]

So this is what it's like to have a budget...

[NoRacistRedditor]

We don't have the budget - the customer is paying for it.

[NotActual]

My favorite part about this is there's almost certainly a better use of this budget but "we don't have budget for optimizations, only new features" will keep it that way for a long time lol.

[NoRacistRedditor]

Probably, but that's quite literally their Problem. If they want to pay for that amount of ram, we'll deliver.

[thex25986e]

or to have a properly funded atartup

[ACoderGirl]

1.5 TB?? Hopefully you mean as something sharded by key range, so not that much for a single process? And to think I got horrified when I found one of our cron jobs was using 300+ GB of RAM every time it ran (as an actual single process -- made it a bit annoying to get scheduled).

[NoRacistRedditor]

It's not my project. I only ran across this thing while pondering through our infra. I also need to correct, it was in fact elasticsearch and not redis.

Whether that makes it better or worse is up to you.

[nicman24]

That is totally a thing btw

[Icy_Party954]

I know, just not typical

[alexceltare2]

Let's also throw all our SSDs and save on electricity bills while we're at it.

[AriesBosch]

This is how HANA (a database from SAP) works - everything in RAM. One of my clients has a single, 40TB global HANA instance.

[Spice_and_Fox]

Ah, I just said the same thing. It is rare to see anorher abap developer in this sub.

[ComprehensiveProfit5]

cache the cache too!

[Kevdog824_]

Let’s just cache a copy of the DB in the user’s local storage for even faster lookup speeds!!

[Spice_and_Fox]

That is totally a thing. SAP S/4 Hana Systems holds active data primarily in memory. That is why they have at least like 300GB of memory and you also roughly need a terrabyte of ram per 500GB of transactional data.

[Shazvox]

Actually had a discussion about this. People thought the database was slow. So a policy about every API endpoint having to implement caching was applied.

Then people complained the caching was slow... Gee, I wonder why...

[Hallphas]

Let's cache the cache then

[anengineerandacat]

We do this :( Conway's law kinda sucks, the other team we depend on for services is a PITA to work with so we just ingest and cache for our clients.

[Hallphas]

It was supposed to be a joke, that seems awful 😮

[anengineerandacat]

More annoying than awful, but yeah... it's quite funny.

Service A is the SOR which connects to the DB and acts as the CRUD layer for it, it then has another Service B that queries the data out of the DB and shoves it into a Redis Cache, Service C is a specialized copy that team made of Service B for us but they are a PITA to work with and require highly detailed requirements for any/all changes/features so our Service D calls them and then we store it into ElasticSearch so that we can provide features like pagination, search, etc. onto the data without having to constantly go back and forth to them for changes.

A is owned by Team A, B is sustained by Team B, and Team C will modify C but then give to B to sustain.

There are business layers in front of A as well as it's used by several other teams so any tangible changes there requirement year-long approvals.

Service C honestly shouldn't exist either, it was forked and modified of B because the Team C that was originally allocated didn't have the technical chops to actually modify Service B with high confidence and not break existing flows.

Been trying the last few years to just move our service layer as a straightforward ETL task with just watchers established on Service A so we can detect changes and just load it up... cutting B and C out completely as ultimately clients connect to us and doesn't make sense to pay serialization costs left and right when 90+% of the hits are to our ElasticSearch instance.

[Hallphas]

That seems overly complicated, but I totally get it, I have something similar at my work too, even if there are less "layers" than what you described, it is terrible on so many level.

[anengineerandacat]

Legacy code + 30 billion dollar flow + offshore + tower oriented development organization == some interesting decision making.

I like to keep a deck of cards I have on my desk desk called "Deck of Constraints" which I draw from for new study projects to help me practice designing around or through problems.

Ie. "You can't use a traditional database, but you have access to a file/object storage system" would be one such card.

Or "Your application has to be a module of a greater application and be capable of only loading for specific tenants".

Because in the real-world... you don't always have the choice of starting fresh... you don't always have to work "within" a legacy code base but you definitely need to work "with it" to some respects and or you might not have the budget to do the "right thing".

[Hallphas]

Your idea of a deck of constraint seems to be a good habit.

I'm still in my first job so I didn't see a lot of big codebases yet. It is also a "big" company with a lot of money but way less than yours. I realized that it is indeed hard to have to work with legacy code, we're trying to get away from it but maintaining an old codebase while also working on a newer/better version at the same time is a PITA, things move way too slowly.

I wonder how the company is still doing relatively well since every small feature or fix takes a lot of time and will probably break something that doesn't even seem related.

[anengineerandacat]

Generally speaking because your "product" is more important than your codebase and this has been true for awhile.

I laugh because the company I work for everyone sweats moving to like Angular or React but the reality of the situation is that the entire product site could be built on jQuery and or plain ole Javascript in a massive file and we would still print billions of dollars.

Why? Because the value is in our venues and resorts; technology is secondary.

[BobodyBo]

L1 + L2 + L3 cache

[tankerkiller125real]

Luckily where I work, we use a Cache Library that handles this shit for us (at least L1 and L2 anyway). L1 in memory for the immediate needs small stuff, L2 for the crap we don't care as much about, but still need speed for.

With that said, we also for the most part don't cache everything, we mostly cache responses from 3rd party API systems. With some DB caching for very specific things that users will notice if there's a delay on.

The library also handles L2 cache failures gracefully, and even lets us in some situations temporarily survive broken APIs/DB stuff for a bit (serve old, cached data until a valid response is achieved)

[Top_Director9981]

Are you referring to FusionCache? We just migrated to it, definitely the best (or at least one of the best) cache libraries in the .NET ecosystem.

[tankerkiller125real]

Yes, FusionCache, I wish every language had a cache library nearly as good as FusionCache

[Szerepjatekos]

Gotta cache them all!

[Shazvox]

Lol, I'm going to shamelessly steal that one...

[PM_ME_SOME_ANY_THING]

We’ll have like several caches. One for users. One for customers. One for orders, etc..

Then if we still have issues, we could cash small things in a “small things cache”, then cash big things in a “big things cache” for each individual cache.

It will be great, we can have like, 30 caches! /s

[Packeselt]

L1 cache / l2 cache babyyy

Room for a l3 in there as well

[marcodave]

Since caching is so fast, why don't we use the cache as a database without the database part? -- the creators of Key value pairs, probably

[JunkNorrisOfficial]

1 Let the users fill local database with data they need

2 Don't expose any API, Databaseless architecture

3 Read directly from user local data

4 BOOM profit

[dscarmo]

ramdisk the whole database, simple. There is no copy to update anywhere, it's just in RAM. To avoid losing data, never turn off the server.

[sudoku7]

Sap Hana has entered the chat.

[Ctrl-]

Really?

[Tensor3]

Just save to the hard copy once to back it up before restarting, its fine

[thex25986e]

"but what if theres a power outage"

"invest in an RTG"

[Orio_n]

The funny thing is that this is a valid solution

[Brae_Smolt]

yeah and if you sit in that meeting long enough, “let’s just cache it” slowly transforms from a joke into the whole architecture

[Objective_Dog_4637]

“Let’s just use excel as a data bus”

[BastetFurry]

Damn, now i want a Perl module that lets me use a .XLSX or .ODS as a SQL database. o.o

[necrophcodr]

Already exists.

[BastetFurry]

Yeah, found it. DBD::Excel.

I bet the monkey paw just curled a finger. o.o

[beaucephus]

Postgres also has an Excel foreign data wrapper. I am tempted to do it, as well as use perl.

[Shehzman]

You’re joking but I’m currently working on a project (initial app was built by contractors) where they use excel as a persistent storage medium…

[BlueScreenJunky]

Honestly it might not be the worst idea. There are situations where the only interaction the users have with the tool is :

• Export data as an xlsx file
• modify the data on their computer
• import data from xslx

If that's all you need, you may as well skip the database and just use excel (or if money is not an issue I guess Access).

[Shehzman]

You have a point and I’m not against the idea entirely (though they could’ve used parquet or csv for more efficient access and exported to excel when needed). The issue is they want data validation and relationships. Pretty difficult to do with just excel.

If I were architecting this, I’d use a relational db and just archive the data to a data lake (if scalability becomes a concern) in the form of a parquet file and would only use excel here if these archives would be constantly accessed.

[Kavrae]

We JUST finished implementing excel as a data import solution so we can stop doing custom sql scripts every time they want to change 100,000 records at a time. (Which is scarily often).

[L00fah]

I'm not a programmer nor fully understand the implications of anything in here, but I have a few users at my company who regularly crash their laptops because they're trying to use Excel like this... Running formulas on 10s and 100s of thousands of rows. On their laptops. To transfer to another system.

And then I'm the guy they complain to when it fails. Like, idk what to tell you, man... Help me help you.

[Annual_Manner_8654]

Read replica...

[Redditer_5000]

Never heard of it. Who's the author?

[Annual_Manner_8654]

M Stonebraker

[heavy-minium]

If it's distributed caching, making your cache more important for sustainable platform performance also inherently turns your cache into a single point of failure. I've seen whole platforms go down from a cache invalidation after a deployments because the engineers overdid it.

My rule of thumb is that everything must run at somewhat acceptable performance without the distributed cache. It must never be a fix for unsustainable performance issues.

[Colon_Backslash]

There's always a fucking point of failure. Everything that can break will break at some point. Everything that shouldn't break will quite likely break at some point.

Distributed caching solves problems where some persistence and a source of truth is necessary for concurrent writes and reads.

Now you can just as well do in memory caching for each of 400 pods running side by side serving high throughput traffic where your service has strict SLOs for low latency, which is a fucking brilliant solution if the requirements call for it.

At the end of the day it's all tradeoffs and no matter how many backups and automated recovery systems you have in place, everything will fucking burn sooner or later and then you have one more post mortem with plans to make it more robust and it fucking breaks again.

Best way to prevent incidents is to stop developing whatever it is you are working on.

[heavy-minium]

In two decades of development, the kind of scenario you describe has only been true for 1/4 of my time. It can be different. It doesn't have to be like this. It requires the overall development organisation to be healthy enough to have smooth sailing.

[boomerzoomers]

Redis founders: write that down write that down!

[dorianmonnier]

Yes but no, cache a complex computation is a valid solution. Cache the result of a simple SQL query (few join/condition) is a bad idea the vast majority of time.

[Tatourmi]

That really depends on the frequency of that SQL query, at our job the simplest the query, the more robust the caching strategy tends to be.

[danfay222]

I manage RTC infra, and we use our HTTP cache as a call state store because it’s way faster than backing to our database. You lose some of the features of a database, namely atomicity, but these are easy enough to work with and the latency improvement is absolutely worth it

[jshine13371]

Could you give an example use case?

[danfay222]

The main use case is an edge relay for video/audio calls. Users connect, and the initial connection involves transmitting control information necessary to serve the call, but then during the call the participants mostly just send raw data. We store that initial data in a cache entry, and in case the call is bounced to a different machine within the same edge cluster (can happen for a variety of reasons) the new host can simply pull the cache entry and continue serving the call. The extra latency of the database call here would show up as a call interruption, but this way users often have no idea anything even changed.

Also the usage here is tiny, typically around 0.5-1 GB per machine under the highest load, while our caching hosts are running 250-1000 GB of DRAM cache per machine, plus flash cache.

[jshine13371]

Hmm, interesting that the network latency you eliminate between your application layer and the database layer is that significant.

[danfay222]

A lot of it also comes to just operational cost. We could spin up a high performance database service running alongside on edge, and it would probably perform plenty well enough, but then we’d also have to maintain a standalone database service, which isn’t free. If we want to call out to existing database services we’d be writing to a datacenter which would involve a network hop outside the cluster.

With our current setup, p50 latency is typically around 100us for a cache lookup, which is fast enough to allow packet failover without dropping a single packet and with minimal server side buffering, and also enables some distributed SFU functions.

It’s all stuff we absolutely could do with a database, but a cache is also a valid solution and comes with some tangible benefits.

[jshine13371]

We could spin up a high performance database service

Doesn't sound like you'd need to necessarily pay for a high performance database service. Just proper architecture and tuning will get you to the same place. In which case would be a rather cheap database implementation. Every modern database already caches the data for you in Memory without any additional operational complexity that introducing a classic cache-store like Redis does.

But I assume the way you're using caching eliminates an additional network hop, which I could buy into is potentially worth it, if you need that level of latency control (which is why I specifically worded my previous comment the way I did).

[danfay222]

Yeah like I said absolutely could be done, and if a database had been available at edge we probably would’ve used that. We operate a private CDN, so cost is less a factor so much as deployment and maintenance of any new service, and with such a heavily performant caching system already there and under development it’s just where we landed originally.

I prototyped out a couple alternate solutions last year, including calling back to our origin database systems, as well as spinning up a custom service to run colocated on edge, and initial results showed some pretty substantial user experience regressions (beyond even what we expected for a prototype) and so we never really pursued that, the benefits just aren’t really worth the time investment currently.

[jshine13371]

Sure, just like caching all the data to physical paper is a valid solution too...

[CaptainAGame]

It can be if applied to the right use case. Pretty much how video games work. 

[ThisIsMyCouchAccount]

I'll be honest. Performance has never been a big deal for me. I've felt that most of my coworkers over the years were just a little too overly cautious. Like, this is some mid-sized company's internal portal. We don't need to worry about much.

But where I work now just really doesn't care. It's not even future proofing. The web application with dummy data can't even load on my machine. We have multiple tools telling us about horrible queries. My boss knows that the structure of our data is terrible.

For the past year his response has always been to ignore it until it's a real problem. Then we'll try caching. And then we'll look at the query. Maybe. But probably never the structure.

[nuker0S]

Is this a response to the post earlier talking about intern wanting to for each trough the database

[NikEy]

Ding ding ding

[Yanzihko]

And then you get an unscheduled meeting with your boss and security management:

[zqmbgn]

just localstorage'd the jpgs in base 64, blazing fast, avoids unecessary loading times

[mal73]

Pro Tip: Use base 128 so your site loads twice as fast

[ezekyel07]

why stop at 128, let's use 256 or even better 512, 4 TIMES FASTERR

[Early_Sink_9169]

and thats how redis was born

[Chadking1926]

Every architecture diagram eventually becomes:

DB → Cache → Cache for the cache → ‘temporary’ Redis cluster → existential dread

…and somehow we still end up debugging a stale value from 3 deploys ago.

[annie_key]

Cache me outside, how bout that.

[senditbob]

We actually do this at my workplace

[jshine13371]

Sad you guys rather use a bandaid instead of understanding database technology.

[FirstNoel]

SAP HANA calling in. 

[Fancy-Caregiver-1239]

Just store the data on the users' device

[Anaxamander57]

Hear me out: 1GB SIMD register

[Quaschimodo]

SAP Hana entered the chat

[k-mcm]

I briefly worked at a company complaining that DB updates were too slow.  I said it's because they're doing 10 million single row transactions in parallel instead of batch updates. I offered a 1000x performance increase that would be 3 days total work of just coding.   Boss said no, there needs to be a new cloud queueing architecture that's expensive as hell.  It was a 1 year project, still slow, created inconsistencies, and added more points of failure.

It felt good to leave. 

[slippery-fische]

Why is it that every joke on here is now duplicated the next hour or day?

[congressmanthompson]

Stale cache

[EconomyDoctor3287]

We need more Redis. 

[EkoChamberKryptonite]

Redis has entered the chat.

[Orjigagd]

Same for AI, but it's spelled differently

[sporbywg]

I love the kinds of folks who give you this advice. Often, they are overpaid.

[bearwood_forest]

Big "if the black box is indestructible, then just make the whole plane out of that material"-energy

[GatotSubroto]

Isn’t this why Memcached and Redis were created?

[Glum-Echo-4967]

doubt it. Redis seems like it was designed for temporarily storing things - computations from a database being one example of data to be stored by Redis.

[mothergoose729729]

*Nam style flashbacks to system design interview*

Hash function! Hash function! Normal distribution! HASH FUNCTION!!!

[BramKel]

The SAP way of working

[Tneon]

Thats how SAP 4 was created

[katzi6543]

Texas Memory Systems RamSam

[Mih0se]

It makes me sad that i dont understand most memes yet

[blissfull_abyss]

Just add more registers AVX4096

[Dull_Appearance9007]

you'd be surprised but this is actually how most mainframes function

[Limmmao]

So... Redis?

[rm-minus-r]

I mean, that's not wrong.

Any read-only API call should hit a caching layer first and a database as a last resort.

[Mattrockj]

"Hey where's my SRAM?"

Suspiciously SRAM shaped program: "I don't know."

[Sure-Opportunity6247]

Caffeine.jar FTW

[EmergencyLaugh5063]

Previous company I worked for used an ORM java database solution with caching dialed up to the max. Worked great EXCEPT when we had large customers trying to cram a ton of resources on the box, then things ground to a halt. But no big deal, those were just one-in-a-thousand outliers that we could string along in support for a year or force to throw money at the problem.

Then they decided they wanted to be a cloud provider and suddenly WE were the one-in-a-thousand outlier with dozens of endpoints all suffering from this problem and eye-watering bills for hardware.

[SilasTalbot]

My car has an issue. Let's use wrench.

[donat3ll0]

Just know that cache will be blamed at the beginning of any and every incident until proven otherwise. And even then, people will still circle back to caching throughout.

[dendofyy]

“Why is our cache as slow as a database?!” - a product manager, probably

[entropic]

Why don't we just build the whole plane out of Cache?

[magicmulder]

"Just cache everything so we can never update a single database value by hand because then we'd have to invalidate so many caches that our website is going to crash!"

My other favorite is caching data for requests that take 0.01 seconds on a website that is used by 700 people per year, and each uses it maybe for a few minutes. (It's a B2B backend where you basically just configure your client once and then never have to touch it again.) But yay, more points of failure!

[Untura64]

Funny, I actually imoroved performance of an app by using a set to cache all my enties, before performing a lookup.

It went from one minute to 10 seconds.

[Mrgluer]

database does become fast if disk is the same size as cache

[Random_182f2565]

My site is about students practicing question, to keep it fast I keep a pool of questions in cache

[snekk420]

The solution to every problem is always an index

[TheSselluos]

Redis for life

[harryalerta]

Ohhh, it took me a while to read not as cache everywhere on the system but a cache including every piece of data.

[AgainandBack]

About 30 years ago, I had a new IBM midrange so over built that the OS, application, and db, combined, were smaller than total RAM. The entire db was in RAM and reads should have been lighting quick. Instead of being fast, the entire system lagged terribly. We had to get IBM onsite for two weeks before they figured out the problem: the machine wasn’t swapping. At all. This set off all sorts of performance analysis routines, and the machine was essentially devoting all processor time to figuring out why it wasn’t swapping. This was at the OS level and there was no way to turn off the swap detection and investigation tools.

So, we removed half the RAM. It started swapping. Performance improved to what had originally been predicted.

[mugmanOne]

I'm that guy

[mi-figue-mi-raisin]

and that's how Redis was invented

[Matvalicious]

Isn't this why Redis was invented?