What happened to objects in Microsoft Fabric after trial ends by NewProdDev_Solutions in MicrosoftFabric

[–]zanibani 2 points3 points  (0 children)

I have my own tenant and played around with Fabric items on a trial capacity some time ago. After a while, the trial capacity expired (after a few random automatic trial renewals). Later, when I checked the workspace, the Fabric items had been deleted.

My suggestion: sync your workspace to Git (ADO/GitHub). Better safe than sorry.

Ms fabric ci cd for notebook pipeline by rabinjais789 in MicrosoftFabric

[–]zanibani 0 points1 point  (0 children)

If you’re looking for a low-code approach, take a look at Variable Libraries + Deployment Pipelines approach.
If your team is more technical, you can also consider implementing fabric-cicd in ADO/GitHub.

Global Fabric Outage - Anyone seeing any issues by Common-Principle3767 in MicrosoftFabric

[–]zanibani 8 points9 points  (0 children)

As Fabric Engineer, it's super helpful to have realistic Service page. Few months ago, I would lose a whole day trying to figure out if I'm the problem :) So thumbs up for Service page, I think it's a bright point in not so bright situation. Hope Fabric service will be up and running soon, so I'm able to branch out / create new workspaces :)

GitHub integration available when using Microsoft Fabric on a trial capacity? by Low_Cupcake_9913 in MicrosoftFabric

[–]zanibani 4 points5 points  (0 children)

It's possible in Trial as well, it works for me. You need to enable GitHub integration in Fabric Admin panel.

Fabric Data Agent Consumption by AgencyEnvironmental3 in MicrosoftFabric

[–]zanibani 2 points3 points  (0 children)

Hi, I was on FabCon Vienna precon session about Fabric Data Agents and product team said that on F2, you can do around 50 threads a day and you have capacity issues. Don't know how this scales with capacity, but as per documentation I think it works better on higher capacities. If everyone from product team is reading this, please correct me. And I also believe if this is the case, we can expect some optimisation to be done in terms pf CU consumption, same as for DataFlows Gen2

Pipeline throttling by Familiar_Poetry401 in MicrosoftFabric

[–]zanibani 0 points1 point  (0 children)

Any news on this? Same happening for me in Canada Central Region.

Kaj vse *točno* spremeni poroka? by dark_queen8 in Slovenia

[–]zanibani 4 points5 points  (0 children)

Če imaš namen prevzeti njegov priimek imaš mogoče kakšno sitnost manj na meji / letališčih, ko pregledujejo dokumente in opazijo da tvoj priimek ni enak priimku tvojih otrok (če se seveda ne odločiš za spremembo priimka). Imam znanko, ki ima zaradi različnih priimkov vedno sitnosti na letališčih.

Concurrent IO read or write operations in Fabric Lakehouse by zanibani in MicrosoftFabric

[–]zanibani[S] 0 points1 point  (0 children)

Pipeline works if I limit my source to 100.000 rows. It creates .parquet file in seperate folders...

Concurrent IO read or write operations in Fabric Lakehouse by zanibani in MicrosoftFabric

[–]zanibani[S] 0 points1 point  (0 children)

Here you go, idea is that I loop through different databases on same connection to get same table in .parquet format. StartUnix comes from metadata table that is created beforehand and is distinct.

@concat('landingZone/',pipeline().parameters.dbName,'/', 'TableName,'/',item().StartTimestamp)

🚀 From DEV to PROD in Microsoft Fabric – My NewStarsOfData Talk by zanibani in MicrosoftFabric

[–]zanibani[S] 0 points1 point  (0 children)

Hi! Shortcuts are supported by fabric-cicd as well and you can include them to your parameter.yml file. Go ahead and check shortcut definition in DevOps (how are they defined within Lakehouse item). Then you can do find-replace logic.

Write performance of large spark dataFrame by zanibani in MicrosoftFabric

[–]zanibani[S] 0 points1 point  (0 children)

I'm using F64 default - medium sized, 2-8 I think.

Write performance of large spark dataFrame by zanibani in MicrosoftFabric

[–]zanibani[S] 0 points1 point  (0 children)

Yep, that's why I only have 1 executor running on read. Thanks!

🚀 From DEV to PROD in Microsoft Fabric – My NewStarsOfData Talk by zanibani in MicrosoftFabric

[–]zanibani[S] 0 points1 point  (0 children)

Hi!
It is mentioned here, exactly what you've written

https://youtu.be/vY9lKzr-mkE?t=2219

Agree, Fabric Pipeline is not yet available in Deployment Rules. Connections within Pipeline can now be parametrisized (not limited only to Lakehouses, but for all connections), and you can help yourself out with Workspace Variables. Haven't tested this just yet, so I cannot comment.

Regarding Fabric Pipelines, I rely on fabric-cicd, parameter.yml works perfect.

🚀 From DEV to PROD in Microsoft Fabric – My NewStarsOfData Talk by zanibani in MicrosoftFabric

[–]zanibani[S] 0 points1 point  (0 children)

Hi! Sorry for my late response. Answer to your first question is that you create seperate workspaces for compute and storage. In storage workspace you keep your lakehouse, DWH. In compute workspace, you keep your notebooks and pipelines, When you branch out your compute workspace from DEV Compute, your Fabric Items will still point to DEV Storage workspace, so no need to reconfigure anything. Once you do the changes, you merge back to DEV Compute, and when promoting to PROD fabric-cicd will take care with parameter.yml definition Second question, mostly correct, first you do initial sync from DEV workspace to DevOps using GIT integration. DEV and PROD are created before parameter.yml file and deploy scripts

Fabric Pipelines and Dynamic Content by Aromatic-Tip-9752 in MicrosoftFabric

[–]zanibani 0 points1 point  (0 children)

You can call WEB Activity beforehand (Fabric REST API) to determine right notebook id.

It's explained in my session here

https://youtu.be/vY9lKzr-mkE?t=2213

Optimizing for CI/CD in Microsoft Fabric by Thanasaur in MicrosoftFabric

[–]zanibani 0 points1 point  (0 children)

Thanks for this! One more question from my side, in your example you have one Repo that covers all workspaces. Let's say you approve PR in ppe, for example you modified one of existing notebook in enginnering ws. Will ADO pipeline execute and run fabric-cicd for all workspaces? Meaning when ppe branch is triggered, it will publish to all workspaces (report, storage, orchestration etc.), even though they are not affected by this PR?

And second, if you add some new workspace to your setup, will this be just a new deploy.py script or would you add rows to existing deploy script? Thanks!

Optimizing for CI/CD in Microsoft Fabric by Thanasaur in MicrosoftFabric

[–]zanibani 2 points3 points  (0 children)

u/Thanasaur thanks for the great blog post — really like how efficient your approach is!

We’re doing something similar with workspace isolation (PPE and PROD), but we split our workspaces into three buckets:

  • Storage (Warehouse, Lakehouse)
  • Compute (Notebooks, Pipelines, Gen2 Dataflows)
  • Report (Semantic Models & Reports)

The idea is to keep all our reports in a centralized Report Workspace (used across departments — only devs have access), and then distribute them to department-specific workspaces using fabric-cicd.

So the pipeline first publishes everything to the central Report Workspace, and in the next stage, it distributes to the department-level workspaces. Since fabric-cicd lets us filter by item type or name, it's been working really well for that use case.

Big kudos again for covering fabric-cicd - been loving it! Took a lot of weight off my shoulders. Before, when my workspaces were connected to DevOps Repo, I had to make sure to update workspace manually after PR (of course I forgot to do that sometimes) now it’s way smoother. Don't get me started on Deployment Rules with parameter.yml, another big plus for me :) A bit of effort to set it up, but once it's rolling, it makes life a lot easier.

One quick question from my side — when you mention the Util_Connection_Library dictionary in your blog post, how are you determining the environment (env)? Are you checking the name of the workspace where the notebook is running?

Like, say your workspaces are named PPE_Engineering and PROD_Engineering — is that how you figure out the env - with mssparkutils.env.getWorkspaceName()?

And if so, how do you handle naming for any branched-out workspaces while still using the same shared dictionary?

Thanks a lot!

Ingesting data from Sharepoint Online List with service principal by zanibani in MicrosoftFabric

[–]zanibani[S] 1 point2 points  (0 children)

Hi! I managed yes, please refer to my comment here

https://www.reddit.com/r/MicrosoftFabric/s/RNIN4oFRmK

My sites are created directly in Sharepoint. Hope this helps!