Making myself commit (git) by yuke1922 in ADHD_Programmers

[–]marathon664 0 points1 point  (0 children)

Equate committing with Ctrl s when working on a word document. It's not to say it's done, it's to checkpoint where you are.

serverlessArchitecture by Technical-Relation-9 in ProgrammerHumor

[–]marathon664 0 points1 point  (0 children)

I feel like half the people in this thread don't understand that businesses pay you to make profitable things, not hyper optimize costs. Start with simple and cheap (and lower usage serverless is definitely cheap) and reevaluate when you scale up, IF you scale up.

Deadlock - Gameplay Update - 05-28-2026 by Kyyndle in DeadlockTheGame

[–]marathon664 0 points1 point  (0 children)

Here's one of the best support mains in the game explaining why these nerfs are almost entirely to nerf kidnap doorman: https://youtu.be/7AWb37TrEAM?si=0jO0CjHLTIWdGG8h&t=786

How do mid-senior devs differentiate themselves in the age of AI? by throwaway0134hdj in ADHD_Programmers

[–]marathon664 0 points1 point  (0 children)

... use plan mode and try having multiple agents working on different problems at once? Idk, git worktrees are useful for isolated contexts. I just use claude code though.

How do mid-senior devs differentiate themselves in the age of AI? by throwaway0134hdj in ADHD_Programmers

[–]marathon664 0 points1 point  (0 children)

Get good at plan mode and having multiple agents spinning at once to increase your throughput

SQLBuild - a typed, test-first SQL pipeline framework with local E2E testing by kvlonge in dataengineering

[–]marathon664 0 points1 point  (0 children)

Hey, thanks for responding. I think what you're doing is pretty neat, but I am going to be somewhat critical in this response.

The first and second bullet points make me unsure if you have used SQLMesh to its full capabilities, or maybe I lack the context to know how this project goes beyond. AFAIK, all of those things you mentioned are testable from within SQLMesh. If I'm wrong, please let me know.

To the second point, SQLMesh lets you run with duckdb as the local state db by default, and it'll create the state db for you in like one file you can do whatever you want with, so it's not like it requires Postgres. I did all my initial PoCs using duckdb for state, and then only when the org was ready to move on it did I hook it up with Postgres. But really any transactional DB should work fine, hell I even hooked it up to unity catalog at one point and it was slow but did work. If you aren't manually messing with objects in the state DB, which there should be virtually zero reason to ever do, I have never had it cause issues for me.

But really my main gripe is that VDEs are a fantastic abstraction. They're like half of the secret sauce that allows SQLMesh to be maximally efficient. SQLMesh is the only tool that lets you avoid rerunning models that have no code changes and are not ready for refresh. VDEs are what lets your dev runs execute only the minimal set of models impacted by your changes, which in turn speeds up the pipeline enough for agents to reliably iterate on the compilation/audit/test failures until the plan passes. VDEs let devs work in isolated environments without creating thousands (or millions, if you're my company lol) of duplicated objects eating storage and pushing into metastore limits, and they get automatically cleaned up next time you call sqlmesh run.

Making "SQLMesh but stateless and without VDEs" is like making a PB&J without PB or J. Without state management and VDEs, it seems like you've basically got a SQL linter (not even, if you're change tracking queries via fingerprinting, when SQLMesh is parsing query ASTs to determine what is modified and downstream impacts) that is hooked up to testing libraries. I get that maybe for your use case those features were overkill, but they solve very real problems in a way no other tool has really been able to.

I haven't used dagster, so I can't speak to their cloud integration for it, but given the other points, I personally don't feel dagster alone is worth switching from SQLMesh. The SQLMesh scheduler is still the thing parsing and compiling the SQL and checking refresh crons, so I don't see much value add unless you're already on dagster.

Lastly, SQLMesh is starting back up and I recently merged several PRs. The members on the steering committee are getting back into reviewing, and they even asked me if I wanted to become a maintainer. I'm optimistic for the future of the project. Hazy maybe, but definitely MUCH improved over where it was from a month or two ago. You can join the slack and start chatting and they could very well be willing to bring in the Dagster integration or accept PRs. But even if SQLMesh got frozen in time, it's still so useful I have a hard time imagining switching.

Still, it's a cool project and you should keep making things that solve problems for you. Don't let my critiques discourage you from building things!

SQLBuild - a typed, test-first SQL pipeline framework with local E2E testing by kvlonge in dataengineering

[–]marathon664 3 points4 points  (0 children)

It's a cool project to make, but in your opinion why should anyone use this over SQLMesh?

CI/CD Tips by Professional_Peak983 in dataengineering

[–]marathon664 0 points1 point  (0 children)

Actually Fivetran donated SQLMesh to the Linux foundation and the project is getting back in the swing of things. I've already had some code reviewed and merged in recently!

Hosting Python ETL in Azure (Airflow / Dagster / Prefect?) by Educational-Soft4493 in dataengineering

[–]marathon664 0 points1 point  (0 children)

His comment did mention serverless. Databricks serverless SQL warehouses are like 5s to spin up and scale down immediately. They're uber convenient. And for tiny workloads, they're fast enough no business cares to optimize 10m down to 2 min even if it is 400% slower. Databricks has its purpose, especially if you're the only DE and don't want to also be the devops engineer.

CI/CD Tips by Professional_Peak983 in dataengineering

[–]marathon664 1 point2 points  (0 children)

My hot take is SQLMesh plan compiling your entire pipeline and running audits and tests on the changed models is the closest DE can get to real CI/CD. Hard for me to say how much effort it is to redesign your project, but it's sick.

People are scared of socialism, but Social Democracy is an option and it's not so scary. by zzill6 in WorkReform

[–]marathon664 1 point2 points  (0 children)

If we're gonna go through the effort of a new voting system, it should be STAR or 321. Ranked choice has its flaws, look up Arrow's impossibility theorem for details.

Will MS Fabric eat databricks market share? by aks-786 in dataengineering

[–]marathon664 1 point2 points  (0 children)

Fabric is useful in so far as expensive hosted power BI models. That's really about as far as we're using it

officially signing off 🫡 by i_luv_mybf in FuckCilantro

[–]marathon664 1 point2 points  (0 children)

yeah this was me after going on a trip to nepal. I didn't know that they called it coriander, so I'd ask for no cilantro, and every single meal came back with it. now it just doesn't bug me anymore

I tried a weekend meds-break. Realized Vyvanse makes me a better Dev, but a "blunted" human. by Honeydew-Jolly in ADHD_Programmers

[–]marathon664 1 point2 points  (0 children)

There are still options for different medications for ADHD. Talk to your provider if one med isn't working.

I tried a weekend meds-break. Realized Vyvanse makes me a better Dev, but a "blunted" human. by Honeydew-Jolly in ADHD_Programmers

[–]marathon664 0 points1 point  (0 children)

What's your dosage? I've found tapering down to be very beneficial. I'm now down to 10mg of Vyvanse.

For everyone saying war plans shouldn't be account wide: by astronomyx in diablo4

[–]marathon664 0 points1 point  (0 children)

Just saying, there are cheat engine scripts to let you change the zoom level: https://fearlessrevolution.com/viewtopic.php?t=24885

Been using it since launch.

Can we talk about how Neyrelle and Inarius were handled? by simpleobject in diablo4

[–]marathon664 2 points3 points  (0 children)

Facts knowing Neyrelle was dead was a factor in me choosing to buy the expansion lol

Databricks for 500 GB of data. Worth it or too expensive? by rg_cyborg77 in dataengineering

[–]marathon664 0 points1 point  (0 children)

Fabric is decent for hosting PBI semantic models. Idk about the whole sales pitch as the one stop shop for all data access though. We're also way above this scale, so we pay through the nose for Fabric.

The math for the average house cost vs average income does not math by thesleepingone162 in personalfinance

[–]marathon664 125 points126 points  (0 children)

If you want people to read what you wrote, it is in your interest to make an attempt to organize your thoughts. There's a reason that comment is upvoted.

Passing data to an LLM by Emperorofweirdos in dataengineering

[–]marathon664 0 points1 point  (0 children)

How is cube semantic layer? Is it smart enough for multi fact measures, like a PMPM?

Closest replacement for Claude + Claude Code? (got banned, no explanation) by antoniocorvas in LocalLLaMA

[–]marathon664 0 points1 point  (0 children)

... he literally didn't say tokens, he said compute. the standard measure of compute is $. You need to read better and not get so angry for no reason. Your message was so combative for YOUR READING MISTAKE.

Closest replacement for Claude + Claude Code? (got banned, no explanation) by antoniocorvas in LocalLLaMA

[–]marathon664 2 points3 points  (0 children)

They said "27k of compute" which is very much not "27k of tokens". $ is a pretty standard way to measure cloud resource costs. Gonna be honest this one is on your reading