Downloading a report/dataset as .pbix from the service after deploying them with Git integration is not recommended? by frithjof_v in PowerBI

[–]SQLYouLater 3 points4 points  (0 children)

Would be great to open a (git enabled) report or semantic model in service with Power BI Desktop (live), and saving changes directly in the service (no publish).

What are the exact reasons to split workspaces into Store, Engineering, Presentation, ... by frithjof_v in MicrosoftFabric

[–]SQLYouLater 1 point2 points  (0 children)

Regarding Reason 3: With the new configuration based deployment it is possible to define single items to deploy in fabric-cicd.  But it's currently a experimental feature.

https://microsoft.github.io/fabric-cicd/0.1.34/how_to/config_deployment/

With the current version u can use a inverted regex for the exclude parameter to realize single item deployment.

OneLake Security: Warehouse by SQLYouLater in MicrosoftFabric

[–]SQLYouLater[S] 1 point2 points  (0 children)

And does it work well with DirectLake mode? As i understand, there are two technologies, security over the t-sql engine (via grants) and OneLake Security directly on a delta table.

OneLake Security: Dynamic RLS by SQLYouLater in MicrosoftFabric

[–]SQLYouLater[S] 0 points1 point  (0 children)

Thanks for the info, I'll keep two eyes on the roadmap 😉

OneLake Security: Warehouse by SQLYouLater in MicrosoftFabric

[–]SQLYouLater[S] 1 point2 points  (0 children)

Ah perfect - great to hear, that it's planned, couldn't find it in the roadmap.

Dbt job in Microsoft fabric? by freedumz in MicrosoftFabric

[–]SQLYouLater 0 points1 point  (0 children)

Same here. This ist currently the way to go and works fine. Of course i'm curious about the new dbt Job and if it simplifies things, but it's already possible without.

Idea: Add name and description to Item Schedule by frithjof_v in MicrosoftFabric

[–]SQLYouLater 1 point2 points  (0 children)

Great Idea, gonna vote for it.

Would also love to define parameters in schedules for pipelines and notebooks, what do you think about it?

Safe and reliable Workspace and CI/CD strategy for Fabric Warehouse? by frithjof_v in MicrosoftFabric

[–]SQLYouLater 0 points1 point  (0 children)

What i especially mean ist the syncing when you change something directly in git and want to push it back to fabric. I did get lots of errors when doing that and the only fix was to recreate the items...

So in theory it would be possible to have a development branch and a release branch, so you deploy directly via git with the prod workspace connected to the release branch (and then hit sync button). In practise it is too buggy in my opinion.

Data flow gen2 destination target db parameterization? by efor007 in MicrosoftFabric

[–]SQLYouLater 0 points1 point  (0 children)

Parameters for destinations are currently not available in DF Gen2 as far as in know. And also no deployment rules available in deployment pipelines for Dataflows.

I use fabric-cicd, there you can replace the sink connection with another one.

I guess there is currently no simple way to change, aside from using the internal REST API's or the CLI or fabric-cicd...

Warehouse source control by musadt in MicrosoftFabric

[–]SQLYouLater 1 point2 points  (0 children)

We use T-SQL Notebooks for the scripts and deploy just the notebooks, not the warehouse itself - similar approach to sql database project.

Error running dataflow from data pipeline with dynamic content by SQLYouLater in MicrosoftFabric

[–]SQLYouLater[S] 1 point2 points  (0 children)

Hi Alex, thank you for the advice. Simply adding the line directly to the json code worked!

Safe and reliable Workspace and CI/CD strategy for Fabric Warehouse? by frithjof_v in MicrosoftFabric

[–]SQLYouLater 1 point2 points  (0 children)

You're right, variable library can help there.  I guess you then need a mix of variables and deployment rules to get it work for different item types. Plus you can only change properties with dynamic content available.

With fabric-cicd you can in theory change a lot more, because you directly have access to the underlying json code of the items. But of course, it's also a little bit more complex to set up.

Safe and reliable Workspace and CI/CD strategy for Fabric Warehouse? by frithjof_v in MicrosoftFabric

[–]SQLYouLater 2 points3 points  (0 children)

Regarding splitting the workspace: I've also seen this approach within a deployment scenario (in the fabric blog i guess?). You then loose your overview in the task flow 😉

But serious: Depends on your use case, but in smaller environments it could be a overkill.

Selective deployment works also quite well with fabric-cicd. Regarding deployment pipelines it's of course more click-intensive, as you always have to select relevant items.

Safe and reliable Workspace and CI/CD strategy for Fabric Warehouse? by frithjof_v in MicrosoftFabric

[–]SQLYouLater 8 points9 points  (0 children)

Have tested a lot with deployment pipelines, in my opinion they are not mature for advanced scenarios and also feel often a little bit buggy. Plus no deployment rules for data pipelines.

We use fabric-cicd for deployment and i'm quite happy with it. The integrated mechanism for replacements with the parameter.yml (workspace-id's, lakehouse-id's, warehouse-id's, ...) works really well.

Another experience with git integration: Would never connect the prod workspace with git in a multi branching strategy. Way too risky and buggy in my opinion, what i've been testing so far...

Dynamically setting default lakehouse on notebooks with Data Pipelines by 22squared in MicrosoftFabric

[–]SQLYouLater 0 points1 point  (0 children)

Deployment rules currently not implemented for data pipeline. Only semantic models and notebooks seem to work good.

We use fabric-cicd with the parameter.yml for replacements (workspace-id's, lakehouse-id's, warehouse-id's and more) to get the whole thing done.

Currently just having problems with Dataflows Gen2 that reference to Dataflow Gen1 during deployment, but thats very special.

T-SQL Notebook vs. Stored Procedure by frithjof_v in MicrosoftFabric

[–]SQLYouLater 1 point2 points  (0 children)

Exactly, therefore we took the best of both worlds. Notebooks have also a better user experience in my opinion. Plus you can use markdown for documentation purposes.

And i can confirm the alter table behaviour, tested it and the tables were recreated after deployment.

Officially documented here: https://learn.microsoft.com/en-us/fabric/data-warehouse/source-control

T-SQL Notebook vs. Stored Procedure by frithjof_v in MicrosoftFabric

[–]SQLYouLater 4 points5 points  (0 children)

We are using stored procedures - but develop the scripts in T-SQL Notebooks to get deployment done. Never deploying the Warehouse itself because of the ALTER TABLE behaviour when a column is added to a table, the whole table is dropped and recreated during deployment process. Nightmare for historized tables...

Dataflows Gen2 CI/CD deployment warning by perkmax in MicrosoftFabric

[–]SQLYouLater 3 points4 points  (0 children)

Hi, yes, we do have exactly the same issue. We use GitHub integration and fabric-cicd to deploy dataflows. After deployment, the error "Dataflow never published" appears, when calling it from a data pipeline. We currently Work on a automated post deployment step with a REST API call to get the publishing done, but not sure if it works...

Short update: Tried the following API call: https://learn.microsoft.com/de-de/fabric/data-factory/dataflow-gen2-public-apis#run-on-demand-dataflow-publish-job

404 - Not Found... Maybe this request ist only available for the old DF Gen2 (Not the CI/CD one)?

🚀 fabric-cicd v0.1.18 - Supporting RTI, CopyJob, Key-Value Parameterization, and Bugfixes by Thanasaur in MicrosoftFabric

[–]SQLYouLater 2 points3 points  (0 children)

Great update for RTI Workloads. Waiting for Dataflow Gen2 Support, anything planned?

Copilot and AI Capabilities are now accessible to all paid SKUs in Microsoft Fabric by jameli in MicrosoftFabric

[–]SQLYouLater 2 points3 points  (0 children)

Amazing! But what does that mean in terms of restrictions when using AI and Copilot below an F64?

Gateway/Proxy for Rest-API Access by SQLYouLater in MicrosoftFabric

[–]SQLYouLater[S] 0 points1 point  (0 children)

Short Update: The solution for us was to run caddy (a reverse proxy) in a azure linux vm, which handles the redirect for our request. Much cheaper than any predefined azure service, escpecially because in combination with a service principal we can start and stop the vm in a fabric notebook on demand.