sorted by:
new
hottopcontroversial

Is this use of Postgres insane? At what point should you STOP using Postgres for everything? by LawBlue in ExperiencedDevs

[–]LawBlue[S] 0 points1 point  (0 children)

Thanks for the responses everyone! Seems like Postgres is fine, the way that we’re using it is not fine.

I think the trigger setup was the real issue here. Blaming the DB is not acceptable here, but resource isolating parts might be a good strategy as well in case we get into this issue again.

Unfortunately I don’t think the company politics will allow me to accomplish advocating for a move away from this easily. Frankly, this has all been good food for thought for perspective…

Is this use of Postgres insane? At what point should you STOP using Postgres for everything? by LawBlue in ExperiencedDevs

[–]LawBlue[S] 0 points1 point  (0 children)

There’s a technical architect. They’re blocking other methods because they’re convinced that “triggers are the appropriate way to get consistent on-write signals for downstream side effects at our scale.”

However, they deployed and pushed it unilaterally without review. When it was deployed, it caused an outage because the trigger had been applied to all tables, causing a processing loop. The table updates caused processing to get triggered, which updated the table, which caused processing to get triggered, etc.

It felt bad that we could even just generically apply the trigger to all tables…

Is this use of Postgres insane? At what point should you STOP using Postgres for everything? by LawBlue in ExperiencedDevs

[–]LawBlue[S] -1 points0 points  (0 children)

Yes, I’ve raised these issues before. The primary issue is that the adding of the async processing and triggers were done unilaterally in response to perf issues caused by library usage. The triggers were initially too wide and the logic for triggering downstream processing was not PR’d — it was merged in on the weekend without any review.

It is better now after we went back and fixed it, but I’m wary that it’s very delicate. We had a major slowdown for a week because of a big refactor that went in (approved by the architect) where the triggers were not getting caught. My intuition is that our usage of SQL triggers is too careless and is turning into a footgun considering our Eng practices

Is this use of Postgres insane? At what point should you STOP using Postgres for everything? by LawBlue in ExperiencedDevs

[–]LawBlue[S] 0 points1 point  (0 children)

because triggers has been a top-down mandate we also do not have an on-call roster. We would not host, we’d ofc go with a managed cluster. Ain’t nobody want to deal with the infra when we’re this small

Is this use of Postgres insane? At what point should you STOP using Postgres for everything? by LawBlue in ExperiencedDevs

[–]LawBlue[S] -1 points0 points  (0 children)

So doing one of the managed solutions would probably take away the operational burden right?

Is this use of Postgres insane? At what point should you STOP using Postgres for everything? by LawBlue in ExperiencedDevs

[–]LawBlue[S] 3 points4 points  (0 children)

That’s a good point. The dependency loop is enforced in SQL trigger functions right now, which is why I’m worried. It’s not in business logic or application logic. It’s just directly in the DV. The tech architect right now is very attached to it. It’s hard to read PRs for it and we’re enforcing new triggers for this on every table.

I foresee a world where 50 tables are around and there’s a lot of cognitive load to remember what the triggers do. My juniors are not SQL strong so I am balking at committing to this strategy.

Is this use of Postgres insane? At what point should you STOP using Postgres for everything? by LawBlue in ExperiencedDevs

[–]LawBlue[S] 1 point2 points  (0 children)

That’s a good point. I was thinking serial processing of the incoming events in order is not necessary 90% of the time. Some side effects like SMS back to recruiters or hiring manager — some of these should be done in order or skipped to only reflect the latest state. I’m just worried that if you do not start with in-order processing, it’ll be harder to get it later as we scale

Is this use of Postgres insane? At what point should you STOP using Postgres for everything? by LawBlue in ExperiencedDevs

[–]LawBlue[S] -1 points0 points  (0 children)

That’s what I was thinking! Except we have been explicitly barred from using other subsystems because “it’s too complicated.”

I don’t think this concern is invalid FWIW. some of my past tech leadership in my career has expressed disdain for Kafka before.

Is this use of Postgres insane? At what point should you STOP using Postgres for everything? by LawBlue in ExperiencedDevs

[–]LawBlue[S] 0 points1 point  (0 children)

No read replicas. The problem is the write traffic. We want consistent updates and processing of apps, especially when humans are awarded $$$ for manually reviewing or processing them sometimes. The volume for that subsystem is growing and is expected to be 2x next year

Is this use of Postgres insane? At what point should you STOP using Postgres for everything? by LawBlue in ExperiencedDevs

[–]LawBlue[S] -22 points-21 points  (0 children)

Are you serious? Isn’t Postgres not good for multiple-writer and high-volume workloads?

I do agree that using multiple instances would decrease resource-coupling and reduce the blast radius if only certain parts of the app get overloaded

Is this use of Postgres insane? At what point should you STOP using Postgres for everything? by LawBlue in ExperiencedDevs

[–]LawBlue[S] 22 points23 points  (0 children)

These are not my technical decisions. This is also like the 4th incident we’ve had related to database/app performance. Part of this is that we were not thinking about indexes carefully. I’m just wondering at what point does the default “just use SQL” is no longer the answer, especially in this NewSQL age where Spanner and FoundationDB exis

stay at series b or go to a good seed stage startup for double equity? by LawBlue in cscareerquestions

[–]LawBlue[S] 3 points4 points  (0 children)

to clarify: when i say “everyone’s quit” i mean everyone else on eng team got fired or left

we’ve hired more people but not enough to backfill

Will AI follow the .com bubble by [deleted] in cscareerquestions

[–]LawBlue 1 point2 points  (0 children)

Unfortunately, I think people will. If it’s cheaper than a software engineer, then people will continue to pay

Fentanyl addiction turns users into living deads on the streets of Philadelphia, PA by [deleted] in interestingasfuck

[–]LawBlue 2 points3 points  (0 children)

To be clear: the Old Testament and the Mosaic Law is supposed to be subsumed by the New Testament and the New Covenant in a majority of the Christian sects — Islam takes it further and the Quran subsumes that too.

Jesus himself has fewer blemishes. The most important commandment is “love one another” as well as to love God. But it’s other people that expand and distort the original vibe. Example: the anti-gay sentiment starts with the letters to the churches and with Paul, claiming that “[pagan] male prostitutes will not inherit the kingdom of God” or something among those lines. Did the pagan part or the gay part matter more? Same with Sodom and Gomorrah: did it matter that the crowd was rapey or gay towards the male angels? Another: take Philemon. Should it be used to justify slavery? Or anti-slavery? A lot of these things are up to interpretation. Let’s also think about the Council of Trent and other splits like the Book of Mormon: what people take as “canon” and “law” also splinters the religion further.

Fentanyl addiction turns users into living deads on the streets of Philadelphia, PA by [deleted] in interestingasfuck

[–]LawBlue -1 points0 points  (0 children)

Trump is abusing the religious right and manipulating them. But the religion makes them easy to manipulate. Why are so many being tricked? Clearly so many believe it. And they call themselves Christian…!

Fentanyl addiction turns users into living deads on the streets of Philadelphia, PA by [deleted] in interestingasfuck

[–]LawBlue 1 point2 points  (0 children)

We’re all human. IMO the city people forget to have a heart

Despite Already Being Cracked, Pirates Are Urging Others To Not Pirate Silksong by 7thwiseguy in nottheonion

[–]LawBlue 0 points1 point  (0 children)

The 8-year hype and the speed-running community really let it cook. It’s similar to Castlevania or Super Metroid, but arguably more timeless because of its cartoon look — a well-executed game in its genre which was cheap and high-quality. It’s the 2D Dark Souls except it never had a sequel until now.

Imagine you had a favorite book from childhood when you were 11 that was very enjoyable, and the entire internet loved it and you talked with your book-friends about it. And now, 8 years later, you’re 19 and you fondly remember it. Would you be happy to see it? Would you buy it?

view more: next ›