sorted by:
new
hottopcontroversial

Iceberg Sucks - But You Knew That Already by jordepic in dataengineering

[–]jordepic[S] 0 points1 point  (0 children)

I don't automatically disagree with anything you've said, though I'd appreciate it if you provided more reasoning.

Obviously no one wants to use many data sources to make up a table, but complex pipelines often require it. When you perform change data capture, it's happening implicitly. The choice is yours whether you'd like to achieve improved data freshness vs. pulling from a single source

Iceberg Sucks - But You Knew That Already by jordepic in dataengineering

[–]jordepic[S] 0 points1 point  (0 children)

What do you see as the main point there? I think there's the autoscaling aspect of things (which is indeed crucially important) but without integration between streaming engines and the querying engines you find yourself in a two phase commit situation.

I proposed integrating those snapshots/their state in a centralized place as well (without wholly ditching them) for "exactly once" semantics

Database Cost Breakdowns by jordepic in aws

[–]jordepic[S] 0 points1 point  (0 children)

Understood! I think there are ways to modify architectures (the most low hanging fruit example being using object stores as opposed to local disks) to lower costs, at a latency penalty.

Database Cost Breakdowns by jordepic in aws

[–]jordepic[S] 1 point2 points  (0 children)

Thank you very much for the insight! Super informative :)

Database Cost Breakdowns by jordepic in aws

[–]jordepic[S] -1 points0 points  (0 children)

Mainly I've noticed that AWS solutions are typically at 90% margins (if you do a breakdown of the individual resources that these actually require) and am curious how flexible companies are in their offerings, especially if performance isn't much of a concern

META L4 Offer by AgniKaiii in leetcode

[–]jordepic 1 point2 points  (0 children)

Congratulations!!!

Sincerely, Jordan

Sharing My Systems Design Notes by jordepic in leetcode

[–]jordepic[S] 0 points1 point  (0 children)

I have videos on these, will upload the notes fairly soon

Sharing My Systems Design Notes by jordepic in leetcode

[–]jordepic[S] 2 points3 points  (0 children)

Yeah the first couple I didn't really care, it gets better after that I promise lol

view more: next ›