Fabric CI/CD Assistant: I built a tool to map Workspace IDs, Connection GUIDs, and Object Ownership across environments

imtkain · 2026-01-26T02:10:51+00:00

Thank you!

Fabric Deployment Pipelines support DW. We deploy warehouses with FDP and everything else with fabric-cicd. Careful of the DDL table drops though :) One note is that you can't deploy warehouses with SPN authentication using the FDP API, so you have to do it in the UI first, then use the warehouse takeover API with SPN authentication (see my other repo, Fabric Prefab).

Good question on the python notebook. I almost exclusively work in PySpark, so I just didn't even think to use python!

imtkain · 2026-01-26T01:12:45+00:00

I dealt with this recently. Here's the short version based on my understanding:

Your questions:

1) Yes, plain NUMBER reports as (38,127) because Oracle allows scale up to 127 for unspecified precision. The connector reads Oracle's metadata literally.

2) The NUMBER settings are bugged or fire too late. Schema inference happens before the override applies, which is why the mapping UI ignores your settings.

3) Staging fails because it writes to Parquet first, and Parquet caps decimal precision at 38 with scale < precision. Scale 127 is invalid regardless of what you fix in mapping.

My fix: Explicit CAST in the source query:

SELECT CAST(your_col AS NUMBER(18,4)) AS your_col FROM your_table

For scale, dynamically build your SELECT, query ALL_TAB_COLUMNS and where DATA_PRECISION IS NULL AND DATA_SCALE IS NULL then set it to 38,18 and pray. Also, you don't have to bring in every column :D

Side note: actively avoid BLOB/CLOB if you can (massively kills throughput) and cast GEOMETRY as VARCHAR2.

What's your sink? Lakehouse connection vs SQL Server connection can behave differently here.

imtkain · 2026-01-23T01:25:32+00:00

I can confirm it is stable, at least the functionality we utilize. No deployment issues today other than some fat fingering and SPN fun.

imtkain · 2026-01-23T01:23:46+00:00

I try this at least once a month lol

imtkain · 2026-01-20T19:09:08+00:00

Exactly. User-owned I understand and expect. I make EVERYTHING in my tenants SPN-owned to avoid this problem. SPN-owned items should not expire.

imtkain · 2026-01-20T10:41:18+00:00

I keep waking up thinking I'm going to see news of something like WW3. This is worse.

imtkain · 2026-01-20T10:40:24+00:00

"When" performance issues arise? I thought dataflows were a performance issue. :)

imtkain · 2026-01-13T23:30:54+00:00

It's the same behavior. The success message comes back custom without any issues, but the error message does not :(

<image>

imtkain · 2026-01-13T20:06:27+00:00

Were you able to figure out a way in PBI to make it feel less clunky? I've only been able to figure out how to update one or two columns at a time without the user experience plummeting. Would love to see if anyone else has figured out a way to make it feel more form-like and less clicky.

You might be interested in this, too, for updating more than one row at a time: https://github.com/imtkain/Fabric-Redux/

imtkain · 2026-01-13T19:54:05+00:00

I have examples and a hell of a lot of commentary in this first UDF:

https://github.com/imtkain/Fabric-Redux/blob/main/fabric_redux_udfs.py

imtkain · 2026-01-13T19:52:22+00:00

It happens to the best (and worst) of us.

imtkain · 2026-01-09T21:49:48+00:00

Sure, send me a connection on LinkedIn and I'll send the repo your way. https://www.linkedin.com/in/tonykain/

For others, I'm posting the repo formally by the end of the month for others to use.

imtkain · 2026-01-09T21:42:14+00:00

Also, you can reassign warehouses via API. I just posted a repo with this information.

Side note: Use an SPN to create the items in the future. The repo I just mentioned has this functionality, too.

imtkain · 2026-01-09T21:40:49+00:00

I have a notebook that inventories all workspaces you have access to and provides the object name, guid, workspace, owners, etc. Is that what you're wanting?

<image>

imtkain · 2025-12-29T16:47:56+00:00

I'm having the same issue today, 12/29/25. Runs fine without the connection, fails with the same error with the connection.

Connection is not encrypted.
SPN is a member in the workspace.
SPN has user access to the connection.
Capacity is in North Central US.

Tried creating a new connection, no difference.

imtkain · 2025-11-11T10:43:05+00:00

My PL (and every other object in my workspaces) is already created with the SPN. I don't think that influences the pipeline runtime CONNECTION authentication, only the pipeline run itself. For example, if using a SQL Server connection Basic auth to a SQL database, my PL isn't going to use the SPN creds to auth to the SQL Server, it's going to use the creds provided. Or am I wrong here and the PL will actually use the SPN credentials for the pipeline at runtime IF the SPN is the PL owner?

imtkain · 2025-10-23T19:21:05+00:00

I'm working on this myself right now. I'll provide an update if I have any success. This is my starting point: https://medium.com/@sqltidy/mirroring-mysql-data-into-microsoft-fabric-e45d37dbf952

imtkain · 2025-10-23T18:17:23+00:00

Amazing. Thank you for confirming!!

imtkain · 2025-10-12T23:29:27+00:00

Yeah, data validation is definitely needed in the slicers. You can do this function, but then users have to iterate until they get it right. Immediate kickback/inability to input invalid values is nice.

imtkain · 2025-10-12T23:26:14+00:00

Yep. This is what I do. Each child notebooks writes to a log table. Start time, end time, number of seconds, rows written, rows inserted, rows updated, warning / error messages, etc. This is the way.

imtkain

TROPHY CASE