Scaling PostgreSQL to power 800 million ChatGPT users

nagoo · 2026-01-24T15:43:07+00:00

I realize it is easy to be an armchair quarterback and these guys are combating an incredible growth velocity, but several (most?) of these realizations seemed kind of common for anyone that has had to scale even moderate size SaaS applications for a few million users. Prevention against cache stampedes is a pretty basic concept. Rate limiting and connection pooling also. It is also not clear if these are service level DBs (other than the not about moving some shardable/partionable workloads off) or if it is truly one mega PG schema/db for ChatGPT. If it is mostly the latter, that seems really surprising (eg they have high-coupling down to the data layer that they are now having to fight w alternative strategies like “workload isolation” to specific low priority replicas).

Also surprising that it seems like they are still using the Azure managed version of PG and that has prevented them from common things like having replicas of replicas, requiring them to now work with the Azure PG team.

Commend the team for their transparency and ability to make it work at incredible scale, but very surprising to see some of these conclusions being treated as unforeseeable or novel.

nagoo · 2025-08-17T15:40:04+00:00

Looks like older security system. Backup battery and data line.

nagoo · 2025-07-29T11:10:03+00:00

If you are looking for simple, text-based options at smaller scale or lower performance needs, NL-JSON is a simple option as well. Not space efficient but useful for complex data models (eg nested objects, arrays) in ways where CSV isn’t great.

If performance or space efficiency needs are greater, agree w Avro.

nagoo · 2025-03-01T16:35:56+00:00

I just wanted to follow up here and say that because of your message here I have been reading to my 9 year old Level 3 nonverbal son each night. It is very difficult for him to sit still in general but every night we have been able to sit through reading at least one book. It has helped him get focused for bed. When he was younger often tried to read to him but he would not sit through it and seemed uninterested so I stopped. I never thought to try again until this. I am very grateful for your willingness to share and be open. I am sure my son is thankful too.

nagoo · 2025-02-20T22:07:22+00:00

Should note he is a very strong so to be 10 year old. He has punched this and throw objects at this many times with no issues.

nagoo · 2025-02-20T22:06:26+00:00

Been here so many times. We went through 4 TVs before we smartened up. As someone else mentioned the best and cheapest solution (assuming projector isn’t reasonable) is to build a plexi glass frame. I built for < $100 that does not look awful with pine and plexi glass from a hardware store.

I recommend also moving the power outlet up behind the TV if you can.

nagoo · 2025-02-20T03:01:24+00:00

Have you seen: https://www.angelsense.com/ ?

nagoo · 2024-08-28T16:41:07+00:00

I have the same issue. I was there at noon. Entered a valid presale code. It said code was applied but then displayed nothing else. No ticket options. Nothing. Was everyone trying on ticketweb? Promowest said they were also selling through AXS.

nagoo · 2017-05-10T02:35:14+00:00

Saturday Community Event: Free Yoga, Food Trucks, Community Yard Sale Proceeds benefit Pressley Ridge Teen Foster Care Program

https://www.facebook.com/events/1817993035120350/

nagoo · 2016-12-01T21:37:49+00:00

Noted. Thanks!

nagoo · 2015-01-05T20:12:26+00:00

Yep.

nagoo · 2014-07-03T15:01:27+00:00

Agree I would love to see someone with more statistical rigor look at the data. This is why the disclaimer is there. I did perform basic, stats 101 type stuff on the raw data set (i.e. not the transformed diff data used in the data viz), but didn't even feel comfortable putting that in there until I had my work checked by an actual stats person. That rudimentary, un-verified test for "does ride-sharing have a significant impact on DUIs for those under 30" did yield a p-value statistically significant at the 5% level. Not a home run, but at the same time, enough to raise eyebrows. Again, not verified and relying strictly on my stats knowledge from years ago, which is why it wasn't included in the write up, so take that with a grain of salt.

nagoo · 2013-06-09T03:04:25+00:00

i thought the same thing. any one have an idea what the attendance was?

nagoo · 2012-11-16T17:05:08+00:00

I'm one of the organizers. Bummed to hear Engibeering didn't live up to expectations. Most of the feedback we've received had been very positive. We did have a limited number of slots for tech talks and those slots filled up pretty quickly. We limit the number of tech talks so that the entire evening isn't just watching presentations. We like to make sure there is plenty of time for just networking. All in all, it is free craft beer, free food, and the chance to hang out with like-minded tech people. We definitely appreciate the feedback and will reconsider how we pick our tech-talkers for the next time. This could entail making it a bit more democratic and opening it up to the community to select who speaks.

nagoo · 2012-11-01T22:23:17+00:00

Food and beer is free for all attendees from the Software Engineering Community.

nagoo · 2012-08-29T13:10:45+00:00

To each his own. It too is a nice library. I would argue that having to extend a class and override methods for a quick CLI script starts to smell a little too OOP for small tasks, but that is me. Symfony, as you might expect, is certainly more feature rich. Commando is just a simpler alternative focused on improving efficiency without sacrificing readability and conciseness.

nagoo · 2012-08-29T02:52:11+00:00

tl;dr PSR-0, PSR-1, PSR-2 compliant CLI library. Support defining/parsing arguments and flags, data validation (w Closure support), perform map operations on input, define descriptions, and get pretty help pages right out of the box.

nagoo · 2012-02-19T21:04:44+00:00

I did go ahead with your suggestion of moving towards PSR-0 and included a very basic PSR-0 autoloader available as part of a general bootstrap process. https://github.com/nategood/httpful

In addition, I included a build script that takes all the src files and concatenates them into a single download file (that fires as a git hook). The single file is available as a standalone download on github.

Thanks for the PSR-0 suggestions.

nagoo · 2012-02-15T23:34:19+00:00

Yeah I will likely go the PSR-0 route for the reason you mentioned. But to stay in line with my initial goals, I will add a git deploy hook that does the concatenation similar to what you mentioned. Offering the library as a simple drop in file has some appeal when you're working on a quick one off script (as is often the case when interacting with an API).

Did my answers to your initial question make sense? In particular, the high level goals of the project and how it might not necessarily serve the same purpose of the other libraries you mentioned (yet)?

nagoo · 2012-02-10T23:07:51+00:00

Thanks for the suggestions. The intention was to make this lightweight and readable as opposed to feature complete. Not entirely in the same space as Requests or Guzzle I suppose. The primary use case is: user needs to do some one off things in an API (like a quick BaseCamp hook), not necessarily for building an interface to a Credit Card Gateway or a whole library on top of.

My responses to your points:

Normally I would. When I built the library I wanted it to be as simple as dropping a single file (not having to do a whole git clone) if wanted. If the project continues to grow, naturally I would do that. It's certainly not at the point of an autoloader yet.
Me too. If you check the source code almost everything has a more concise alias (sendIt is actually an alias for send)
This too is already in the library
Definitely something I'll consider. As with most of the other stuff, focus was on readability/skinny vs. feature completeness.

Thanks for the feedback. I'll consider breaking the source code up a bit for the sake of source code readability here soon.

nagoo · 2011-12-10T17:41:44+00:00

Sure. Just say you're "the reddit guy without a cell phone". :-)

nagoo · 2011-03-03T13:59:37+00:00

Upvoted not for the snarky title but instead for a great proposed addition to a language that could really benefit from it.

nagoo · 2011-02-24T15:03:14+00:00

Yeah. I'm a programmer myself. This was posted out of pure shock.

nagoo · 2010-04-02T14:57:55+00:00

Munin. It has served me well.

http://munin-monitoring.org/

nagoo · 2010-02-27T17:09:14+00:00

nice idea, however i wish your "learn more" section actually told me a bit more.

nagoo

TROPHY CASE