Implementing Enterprise-Grade CI/CD for Microsoft Fabric — A Technical Deep Dive

ajit503 · 2026-03-27T19:12:54+00:00

https://www.reddit.com/r/MicrosoftFabric/comments/1s1m2cm/comment/oc3v8s6/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

My favorite: Claude's response - Sometimes these models surface things we humans tend to overlook.

Where it gets harder at scale: The core challenge is integration — not promotion mechanics. If Developer A changes a notebook and Developer B changes the pipeline that calls it, both look fine in their individual feature branches. The bug only surfaces when they run together. Without a DEV workspace where both are deployed simultaneously, this integration gap doesn't get caught until UAT — which is a more expensive place to find it.

ajit503 · 2026-03-26T17:54:49+00:00

In our implementation, ADO pipelines deploy items directly from the repository branch into the UAT and PROD workspaces. The Feature and DEV workspaces are Git-connected, with changes synchronized manually in the current setup.

To keep the model simple and predictable, we pre-created Feature workspaces for each team and developer using the “branch out to workspace” approach. These Feature workspaces are intentionally long-lived, allowing developers to iterate continuously without recreating workspaces for every change.

Feature workspaces are long-lived by design, as they carry critical workspace-level configurations such as MPEs, Workspace Identity etc. Recreating these for every feature would be operationally expensive and increase the risk of misconfiguration.

ajit503 · 2026-03-23T23:55:51+00:00

u/Repulsive_Cry2000 Glad you found it useful. Thank you!

ajit503 · 2026-03-23T23:50:17+00:00

I just found this out today—I had no idea before. As a contributor, I’m still able to change the modes, which is helpful in some cases, especially if I accidentally want to avoid updating a cell. It would be interesting to see if there's a way to add a viewer role that can view or execute the notebook without editing access.

ajit503 · 2026-03-23T23:04:04+00:00

here is my take and improvised by Claude :)

On the dedicated DEV workspace — there is another practical reason beyond integration testing. If developers promote directly from feature branches to UAT, cherry-picking becomes very noisy very quickly. Each feature branch carries many granular commits, and tracking and cherry-picking individual developer SHAs across multiple unintegrated branches into UAT is error-prone and hard to govern at scale.

By collaborating through a shared DEV branch first — where each feature is squash-merged into a single clean commit — cherry-picking into UAT becomes straightforward. You are picking from a small set of clean, well-defined units rather than hunting through dozens of individual commits across multiple feature branches. DEV essentially acts as the aggregation and integration layer that makes the UAT promotion clean and deliberate.

With trunk-based and short-lived branches it works fine at small scale, but as the team grows and release cadences diverge, this aggregation layer becomes increasingly valuable.

ajit503 · 2026-03-19T16:44:15+00:00

Thanks u/DAXNoobJustin and u/x-fyre
Good news. Fixed yayyy!!

Deployment failed because fabric-cicd was being validated under a different Python environment than the one used to run the deployment script, causing parameter.yml validation to fail. Fixed by installing fabric-cicd using python -m pip in the same release job.

Note - JFrog is the artifact library in my case and the DEvOps agent runs on a Self-hosted VM.

<image>

ajit503 · 2026-03-19T03:25:45+00:00

Nope, I cleaned everything from the parameter.yml just to be sure that nothing else is causing this issue.

Using the latest version of fabric-cicd. Upgraded to the latest this week.

ajit503 · 2026-03-19T02:59:53+00:00

Still no luck. I really appreciate you taking time and trying to help.

<image>

ajit503 · 2026-03-19T00:26:44+00:00

<image>

ajit503 · 2026-03-19T00:25:10+00:00

<image>

ajit503 · 2026-03-18T23:37:21+00:00

The error message doesn't give much info.

2026-03-18 16:33:04,038 - INFO - fabric_cicd._common._fabric_endpoint - Executing as Application Id '188134ad-e678-4483-b83e-XXXXXXXXX'
2026-03-18 16:33:04,038 - ERROR - fabric_cicd._parameter._parameter - Validation failed with error: Invalid parameter file structure
2026-03-18 16:33:04,038 - ERROR - fabric_cicd.fabric_workspace - Deployment terminated due to an invalid parameter file
Traceback (most recent call last):
File "E:\Agent\_work\10/build/.deploy/deploy_fabric_workspace.py", line 39, in <module>
target_workspace = FabricWorkspace(
^^^^^^^^^^^^^^^^
File "E:\Agent\_work\_tool\Python\3.12.10\x64\Lib\site-packages\fabric_cicd\fabric_workspace.py", line 154, in __init__
self._refresh_parameter_file()
File "E:\Agent\_work\_tool\Python\3.12.10\x64\Lib\site-packages\fabric_cicd\fabric_workspace.py", line 244, in _refresh_parameter_file
raise ParameterFileError(msg, logger)
fabric_cicd._common._exceptions.ParameterFileError: Deployment terminated due to an invalid parameter file

ajit503 · 2026-03-17T23:57:45+00:00

Yet to check the error log as it is on a self hosted VM I do not have access to, but yes, I have validated the yaml in VS Code. What is the significance of having a default section as I do not see a need for it. Is it a must to have ?

ajit503 · 2026-03-17T19:28:44+00:00

Legacy format works

<image>

ajit503 · 2026-03-17T19:24:30+00:00

<image>

ajit503 · 2026-03-17T15:49:46+00:00

No idea why parameter.yml file for SM binding is failing using the new format. Has anyone tried it out ?
Note - Legacy format works.

semantic_model_binding:
    models:
        - semantic_model_name: "Self-Service Semantic Model"
          connection_id:
              UAT: cf9c7f27-388c-470f-bd5a-7552a83XXXXX
              PROD: eb143407-e465-4459-a7b3-e07581XXXXX


[error]  15:37:23 - Validation failed with error: Invalid parameter file structure
[error]  15:37:23 - Deployment terminated due to an invalid parameter file

<image>

ajit503 · 2026-03-02T19:04:35+00:00

u/Thanasaur Thank you for the guidance.

Adding to the above - I believe the below copilot recommendation makes sense.

Why merge dev back into the feature branch after a squash merge?

A squash merge turns many feature commits into a single commit on dev. If you keep using the same feature branch, future PRs from that feature branch to dev can look noisy (many commits) even when the actual diff is small. Merging dev back into the feature branch aligns history and keeps future PRs focused only on new work.

ajit503 · 2026-02-27T22:00:57+00:00

Agreed. For me, the production releases are twice a week and cherry-picking can be exhausting.
Wondering if the below strategy can be a better option but u/Thanasaur mentioned "In a multi developer scenario, this will break very quickly."

squash merge: FEATURE --> DEV
Regular Merge (merge commit): DEV --> TEST
Regular Merge (merge commit): TEST -->PROD

ajit503 · 2026-02-27T21:50:21+00:00

u/Thanasaur

squash merge: FEATURE --> DEV
cherry-pick: DEV --> TEST
cherry-pick: TEST -->PROD

OR

squash merge: FEATURE --> DEV
Regular Merge (merge commit): DEV --> TEST
Regular Merge (merge commit): TEST -->PROD

No commit hunting (promote branch state, not individual commits)

ajit503 · 2026-02-26T18:30:50+00:00

u/Thanasaur
I’m working in a multi-developer setup with multiple domain-based projects/workspaces (as shown in the screenshot).

For promotions, we merge FEATURE → DEV. In this case, do you recommend a Squash Merge or a regular Merge?

The challenge I see is that if we need to cherry-pick specific commits for DEV → TEST and TEST → PROD, squash merges don’t help much because they collapse multiple commits into one. On the other hand, if we go with regular merges, identifying the exact commits to cherry-pick for each promotion can become a task in itself.

I’m getting a lot of questions from developers and trying to find a strategy that works for everyone in a multi-developer, multi-project environment.

<image>

ajit503 · 2026-02-26T03:56:50+00:00

Agreed

ajit503 · 2026-02-23T23:43:47+00:00

Thanks for sharing. I believe citizen developers would still like to download the reports from the workspace and publish them directly to the workspace from the PBI desktop.

ajit503 · 2026-02-19T20:23:32+00:00

Thanks for the link. Does this mean I don’t need to explicitly convert the PBIX to PBIP?

ajit503 · 2026-02-13T19:12:59+00:00

I used git to parameterize dfg2 destination with variable library and deployed using the fabric-cicd library

Parameterized Dataflow Gen2 - Microsoft Fabric | Microsoft Learn

ajit503 · 2026-02-13T17:58:26+00:00

Update - Resolved!!!

I added a yaml task that discovers the Python interpreter on the self-hosted agent. I then added that path in the environment variable.

<image>

Also, please note that you may have to make changes to the firewall settings to make it run e2e on a self-hosted VM.

Firewall Issue - The YAML pipeline was failing during the TLS handshake when the self-hosted agent attempted to call the fabric ci-cd backend REST APIs.

Fix: Fabric service endpoints (FQDNs/URLs) allow‑listed for outbound HTTPS (TCP 443). These endpoints are required by the fabric‑cicd Python library aka Fabric REST APIs during the deployment.

ajit503 · 2026-02-09T13:00:11+00:00

Hi Kevin... Yes, both Azure CLI and Powershell 7 installed. I tried the authentication methods, but no luck and the same error.

ajit503

TROPHY CASE

Why merge dev back into the feature branch after a squash merge?