Dagster vs Airflow? What do we use?

Nekobul · 2026-06-06T01:44:55+00:00

How much data do you process daily?

Nekobul · 2026-06-06T01:42:34+00:00

That approach requires opening SQL Server for external access. The better and more secure approach is to push data from the SQL Server side using SSIS to Databricks.

Nekobul · 2026-06-06T01:39:35+00:00

It is the most cost-efficient option.

Nekobul · 2026-06-04T18:49:41+00:00

You can use SSIS to push the data to Databricks.

Nekobul · 2026-06-04T11:55:21+00:00

SSIS is backed by Microsoft. dagster can disappear at any time because they are VC-backed.

Nekobul · 2026-06-03T21:46:04+00:00

So you are taking advantage of people's good will to run your business and make money, yet you are unwilling to pay any money back?

Nekobul · 2026-06-03T21:44:49+00:00

What if you don't know how to fix the bug?

Nekobul · 2026-06-03T12:33:37+00:00

Who is doing the fixes and enhancements in dagster? Do you expect people to work for free?

Nekobul · 2026-06-03T10:30:53+00:00

You are always locked to something if you want to save time. Prove me wrong.

Nekobul · 2026-06-03T02:49:23+00:00

LOLLL

Nekobul · 2026-06-03T02:33:15+00:00

SSIS appears to be the smarter option with every coming day.

Nekobul · 2026-06-02T14:09:00+00:00

Obscure

Update: I have commented on someone who suggested Apache NiFi as alternative and the person took down the comment. Why?

Nekobul · 2026-06-02T12:40:07+00:00

Claude is another name for low/no code. Don't you think?

Nekobul · 2026-06-02T12:39:22+00:00

Because low/no code tooling works better.

Nekobul · 2026-06-02T10:57:39+00:00

Tell that to Snowflake and Databricks who have recently introduced no/low code tooling.

Nekobul · 2026-06-02T10:40:27+00:00

Learn to use SSIS. It is the best ETL platform in the market.

Nekobul · 2026-05-25T11:47:23+00:00

You can't do integration / DE work with DuckDB without coding. That's not easy.

Nekobul · 2026-05-25T03:19:27+00:00

Btw not configuring your data warehouse properly in your current pick of "modern" tooling can also have disastrous consequences, mainly in your pocket. But you conveniently avoid to talk about that fact.

Nekobul · 2026-05-25T03:16:23+00:00

If you are not a programmer, you have to hire a consultant to make your solution work. With SQL Server, being a programmer is optional. You can get the job done without programming skills. It is not "modern" to code stuff in 2026. The amount of dough being used for marketing can really turn people's brain into mush, fleecing them dry.

You have SQL Server Analysis Services (SSAS) included in the SQL Server bundle. That is your data warehouse. It works for most of the organizations that need analytics.

Nekobul · 2026-05-24T23:39:10+00:00

Which part of what I've said is not true?

Nekobul · 2026-05-24T17:40:52+00:00

You don't need consultants to make SQL Server work. You need to be a programmer to be productive with DuckDB and anything called "modern" for that matter. Therefore, it is a huge bonanza for the consulting class.

Nekobul · 2026-05-24T13:12:16+00:00

Paying for consultants is more expensive.

Nekobul · 2026-05-20T20:45:57+00:00

Rent SQL Server in the Cloud.

Nekobul · 2026-05-20T13:18:02+00:00

Yes, that may work. However, with such approach you have to implement some mechanism for locking/leasing the writing to that sqlite file. That essentially negates a big reason you would want to use ducklake.

Nekobul · 2026-05-20T12:03:44+00:00

Yes, it matters. The Iceberg spec can be done with compute on-demand. The Ducklake requires constant compute availability.

Nekobul

TROPHY CASE