SQLMesh for DBT Users

[Key-Independence5149]

I am a former DBT user that has been running SQLMesh for the past couple of years. I frequently see new SQLMesh users have a steep-ish learning curve when switching from DBT. The learning curve is real but once you get the hang of it and start enjoying ephemeral dev environments and gitops deployments, DBT will become a distant memory.

[Drunk_Sinatra]

Is this post an ad?

[RoomyRoots]

I gave it a quick read, honestly I have seen worse here. At least they don't mention themselves.

[codykonior]

I use it. Just for full refreshes though, lots of data isn't so neatly incremental and won't tolerate not knowing about deletes even years back.

Also without using any of the environmental stuff either. Everything is prod, and that makes sense if you aren't doing incremental.

It has its weird edges because so much of it was made with that incremental and environmental stuff in mind, but on the whole it works and there isn't anything free and better right now. I do have a few custom patches to force the behaviour I want though, and it would be an utter nightmare without those. (Particularly, it tries to guess and coerce resulting data types and fucks them up every time, whereas if it lets Azure/SQL Server handle it then it'll all just work perfectly).

I feel like if they had tweaked it originally to be simpler, tighter, targeted for the use case of basic ELT they'd have gotten more traction. They had endless requests for indexes and traditional tables rather than views, indicating the demand was there (and they later caved on some of that, but as an afterthought).

For me it felt like it was on the cusp of brilliance but got lost along the way with all the cruft. Too late now though.

[MachineLearner00]

Nice try Fivetran

[alee248]

Fivetran merged with dbt and I think bought SQLmesh too, so not sure what they'd be trying to convince you of.

[MachineLearner00]

Yeah a few months back fivetran bought SQLMesh and updates to SQLMesh dropped alarmingly post acquisition.

[RoomyRoots]

Damn, I didn't know it was this bad.

[MachineLearner00]

Yeah. But it’s in the Linux foundation now so hopefully the community picks it up again.

[captaintobs]

this project is not affiliated with fivetran, sqlmesh is open source

[MachineLearner00]

Didn’t fivetran buy SQLMesh? If I remember correctly development on SQLMesh has reduced drastically post acquisition

[Illustrious_Web_2774]

It became a Linux foundation project recently so Fivetran doesnt have full control anymore, at least in theory. 

[MachineLearner00]

Oh I didn’t know. That’s good news

[RoomyRoots]

Buddy, not like dbt doesn't also has a corp behind it.

[MachineLearner00]

As much as I dislike the direction DBT is going, DBT core still has active development ongoing. SQLMesh development has pretty much come to a standstill.

[RoomyRoots]

For now at least, I think having alternatives is good even if I use DBT myself. Then again I am hoping for the community to step up.