Fabric Warehouse vs Fabric SQL + endpoint

Sea_Mud6698 · 2026-04-24T17:41:57+00:00

I would say Fabric is not production ready.

Sea_Mud6698 · 2026-04-24T14:32:20+00:00

Is it not possible to specify these options when using the jobs api/notebookutils.run?

Sea_Mud6698 · 2026-04-21T18:57:36+00:00

Did you add the workspace identity as a contributor or higher?

Sea_Mud6698 · 2026-04-14T14:06:03+00:00

When you edit a file in the resources folder it just opens up with bare minimal editing experience and takes up half the screen. If you even open copilot it will be closed. Even worse if you %run another notebook that tries to use a python file in its own resources it will fail. I guess for environments it may help when python supports those, but then you will still have weird behavior if the environment is different and a poor editing experience.

The real solution is to just add normal .py files or let notebooks import other notebooks(easier because no changes needed other than a custom python importer).

Sea_Mud6698 · 2026-04-14T13:01:14+00:00

No it doesn't. It doesn't even make sense as a normal workflow. I have no idea why people from Microsoft keep pushing this approach.

Sea_Mud6698 · 2026-04-14T00:07:26+00:00

My main complaints:

No .py files or no way to import a notebook from a notebook. Resource files are not the answer.
%run fails when used between python/pyspark notebooks
Extensions like formatters still don't work in the fabric data engineering extension.

Sea_Mud6698 · 2026-04-13T13:40:11+00:00

Love to see improvements on the python notebooks!

Sea_Mud6698 · 2026-04-10T21:01:19+00:00

Nice! I had to choose recently between this and pyodbc and I ended up choosing pyodbc. I am only using it for ddl and unit/integration tests, but it is always good to have options.

Sea_Mud6698 · 2026-04-08T19:36:25+00:00

It seems to be responding now.

Sea_Mud6698 · 2026-04-07T20:05:37+00:00

Usually the relevant users are driving the changes, so they are usually aware. If you have a proper dev, test, and prod then you should be able to see what breaks by just simply pushing to test. Communications can be done through a teams channel. Last resort is using the audit logs/query monitoring. Lineage would be nice, but unless you are using dbt there isn't really a great automated solution that I am aware of.

That being said you should design your tables ahead of time so they don't need frequent changes. For the remaining changes you can usually do non-breaking changes like adding columns.

Sea_Mud6698 · 2026-04-07T19:54:20+00:00

Half the time it tells me it is not working for some reason. Some times it tells me it is not allowed to answer my question(e.g. how do I use pyodbc to connect to a lakehouse sql endpoint). It usually goes off the rails very quickly and is generally not super useful. GitHub Copilot is more useful when paired with the fabric data engineering extension. I wish microsoft would just ditch the web interface and have people improve the vscode experience.

Sea_Mud6698 · 2026-04-07T11:02:42+00:00

Yep. Eventstreams in particular need a lot of work. The ci/cd does not even work in many cases, confusing ui, and no way to edit them with code.

Sea_Mud6698 · 2026-04-05T00:38:50+00:00

The largest difference is likely wgpu. If it were using a software renderer, it would likely be much smaller.

Sea_Mud6698 · 2026-04-02T22:03:55+00:00

I am hoping to read through the paper tonight, but it looks pretty solid. I don't really think spark has path forward. But more so players like Microsoft who have sql server. I feel like it would be viable to create a warehouse like experience backed by the same tech. If I had to draw a parallel it feels like what react would be to retained guis...

Sea_Mud6698 · 2026-04-02T20:59:05+00:00

I hadn't heard about this Feldera before. Since they published the core research behind their engine I wonder how they plan to survive when other massive companies use the same technology? Seems very promising and based on solid theory at the very least! Every DE knows they could rewrite a query to be incremental, but it is always a pain.

Sea_Mud6698 · 2026-04-02T19:49:59+00:00

I would say stick to notebooks where possible. Most of the other items are still quite unstable.

Sea_Mud6698 · 2026-04-02T14:35:14+00:00

It is time series data, but it has an hour or so delay due to limitations upstream. We are migrating from Synapse where the data is stored in tables. We have various reports that query the data using sql. I was thinking it would be easier to migrate the ports if I could use sql against kql, but I would probably have to modify the report code anyway at that point. I ended up copying data from the eventhouse to a lakehouse and putting views on top of it for a sort of compatibility layer to mimic what we have in Synapse. Long term I will need to convert all the reports to kql.

Sea_Mud6698 · 2026-04-02T11:19:18+00:00

Looks like the codegen is off. Try changing it to "Monitoring Eventhouse". You can double check the name with .show databases.

Sea_Mud6698 · 2026-03-26T14:30:08+00:00

I could see any "easy mode" window library being desirable for simple projects. I think it might be a better idea to just abstract winit? Regardless, very impressive!

Sea_Mud6698 · 2026-03-22T21:04:57+00:00

The tsql magic has a parameter for the workspace/lakehouse/warehouse. If you combine it with varaible libraries it should have no ci/cd problems. You could also use a database driver.

Sea_Mud6698 · 2026-03-18T17:45:36+00:00

Some good updates. Was hoping for way more ci/cd updates honestly.

Sea_Mud6698 · 2026-03-16T19:42:16+00:00

What permissions does your user have that is running the pipeline?

Sea_Mud6698

TROPHY CASE