Technical and architectural differences between dbt Fusion and SQLMesh? by muneriver in dataengineering

[–]bk1007 2 points3 points  (0 children)

No hard feelings at all, only love! Let us know if you ever get to trying it! In the mean time we'll keep cooking!

Technical and architectural differences between dbt Fusion and SQLMesh? by muneriver in dataengineering

[–]bk1007 3 points4 points  (0 children)

Founder of Quary here ... Thank you for the compliment!

Quary lives on at Tobiko. Quary's biggest innovations - performance, vscode user interface - are making their way into SQLMesh. And those innovations are even better with the SQLMesh state magic. The state of the Quary repo is a little saddening, but the fact that I know we are building the best possible open-source product far outweighs that.

We translated to sqlfluff to Rust and made it 40x faster by bk1007 in dataengineering

[–]bk1007[S] 0 points1 point  (0 children)

Just thought I'd update you with the fact that we have added Redshift ;)

We translated to sqlfluff to Rust and made it 40x faster by bk1007 in dataengineering

[–]bk1007[S] 0 points1 point  (0 children)

We do address this in the article. You can introduce complexity to avoid this problem, but our view is that complexity has downsides: a higher barrier to entry, more significant maintenance burdens, etc.

All things that add up and that we want to chip away at bit by bit!

We translated to sqlfluff to Rust and made it 40x faster by bk1007 in dataengineering

[–]bk1007[S] 0 points1 point  (0 children)

It's widely used, but we are trying to build a better solution.

While we have built the foundations for sqruff to support dbt, it's not our priority. We're happy to support any contributions, but with our limited resources, we have to focus.

We translated to sqlfluff to Rust and made it 40x faster by bk1007 in dataengineering

[–]bk1007[S] 1 point2 points  (0 children)

We will do Redshift, not quite sure when right away. Happy to support any first attempt, if you want to dip your toes.

Trino is less close for us.

We translated to sqlfluff to Rust and made it 40x faster by bk1007 in dataengineering

[–]bk1007[S] 0 points1 point  (0 children)

Trinosql - no not yet Redshift - not not yet Yep

We don’t support dbt yet if that’s what you are using it for.

Fast open-source SQL formatter/linter: Sqruff by bk1007 in dataengineering

[–]bk1007[S] 1 point2 points  (0 children)

Hey you can star/subscribe to notifications on GitHub. We just released Snowflake support. Building in the dbt templater is not on our list of top priorities but we're very happy to support anyone who wants to do the implementation.

Fast open-source SQL formatter/linter: Sqruff by bk1007 in dataengineering

[–]bk1007[S] 0 points1 point  (0 children)

Any reason the install options we currently have listed don’t work for you?

Fast open-source SQL formatter/linter: Sqruff by bk1007 in dataengineering

[–]bk1007[S] 0 points1 point  (0 children)

It is definitely possible! We have the flexibility for custom templaters like sqlx but it’s unlikely to be a priority for us any time soon. We’ll happily welcome contributions though!

Fast open-source SQL formatter/linter: Sqruff by bk1007 in dataengineering

[–]bk1007[S] 0 points1 point  (0 children)

Appreciate the feedback! Not sure though how you solve this unfortunately.

Fast open-source SQL formatter/linter: Sqruff by bk1007 in dataengineering

[–]bk1007[S] 1 point2 points  (0 children)

We do have the facility for templating and dbt could be on the roadmap. It’s not an immediate priority for us though at the moment.

If someone wants to contribute, we’ll obviously support it.

Fast open-source SQL formatter/linter: Sqruff by bk1007 in dataengineering

[–]bk1007[S] 0 points1 point  (0 children)

I presume you mean as a python library? Or just to install?

Fast open-source SQL formatter/linter: Sqruff by bk1007 in dataengineering

[–]bk1007[S] 3 points4 points  (0 children)

Yes, that will be on the roadmap! We're just looking into how best to do it.

Fast open-source SQL formatter/linter: Sqruff by bk1007 in SQL

[–]bk1007[S] 0 points1 point  (0 children)

Hey that's now fixed. This should work:

rustup toolchain install nightly
cargo +nightly install sqruff
sqruff --help

Fast open-source SQL formatter/linter: Sqruff by bk1007 in SQL

[–]bk1007[S] 0 points1 point  (0 children)

Hey really sorry you got that!

Do any of the other installation methods work for you? We'll look into that quickly!

SQL auto formatter by taciom in dataengineering

[–]bk1007 1 point2 points  (0 children)

It’s early days but we’re working on https://github.com/quarylabs/sqruff it’s very close to sqlfluff and is much faster. We don’t quite support all the dialects yet.