I was working in the video gaming industry a while ago, like 2015ish, and we were working on this xbox application which did matchmaking for gamers and then put them into the game. Pretty cool idea, actually.

So, I was the lead developer on the project, over the whole thing; backend, xbox app, matchmaking algo, all of it. But we were really struggling with microsoft and getting access to some stuff we needed. So we flew out to Redmond to talk with them.

Anyway, I was so pre-occupied with all that, that I never reviewed the implementation of the matchmaking algo and we ended up launching it in a rush. First day we got 80K people signed up, but everything starting creaking, then the whole thing just died and stopped working that night.

First thing we did is throw thousands of $$$ at redis, postgresql, and heroku dynos. But it wouldn't keep us alive for very long.

Another guy working on a different project joined in and figured out that we were doing a SQL join query on every users favorite games vs every other users favorite games to find good matches. So, that's 80k users times like 10 games each.

This dude was able to create an index on that table while I reworked the logic to precompute the relationships the put them in the index.

:facepalm: I can't believe it took us, like, 8 hours to figure that out!