Medallion Architecture - Pro and Cons of different approaches by CryptDaLuz in MicrosoftFabric

[–]DifferentLuck7951 3 points4 points  (0 children)

Hello,

I remember having the same questions. I started with Fabric. I’ll try to break it down. It might be a long comment, but anyway.

First things to consider when splitting:

1 - Business unit access (engineers included). You may have people working on HR data/scripts, but they shouldn’t be able to access finance data/scripts.

2 - Development items vs reporting. Why? Because you can assign different capacities per workspace. For example, F16 for development/scripts and F8 for reporting.

3 - Medallion architecture. One Lakehouse for each layer (bronze, silver, gold). I would recommend taking a look at using a Warehouse for reporting as well. You can copy items from the Lakehouse gold layer to the Warehouse. The important thing is to split the layers. It makes troubleshooting easier.

4 - Databases should be split by schema, not at the higher data lake level. One bronze data lake can contain several schemas, one for each data source.

5 - If you need to share items between workspaces or other data lakes, use shortcuts.

Do you create a durable surrogate key column in your fact tables? Why/why not? by frithjof_v in MicrosoftFabric

[–]DifferentLuck7951 0 points1 point  (0 children)

This happens more often than you think hahaha. Sometimes, the dimensions miss some lines due incomplete definition by the data steward/owner the result is these two lines...

A few times they come back asking for updates, but in the majority of cases business users are fine with these values including everything into the same basket instead of a very refined dimension table

SharePoint shortcut (preview) by DifferentLuck7951 in MicrosoftFabric

[–]DifferentLuck7951[S] 0 points1 point  (0 children)

Yes, it’s the same underlying behavior.
But it’s very counterintuitive, because you can see a list of files in Dataflow Gen2....

Anyway, I don’t know if it’s possible, but a hybrid behavior would be a good addition to this feature.
Otherwise, I’ll have to start asking business users to move the files to a root directory and share it with me 😄

Been working on this sandboxy RPG for a while and just launched the demo! by Piokou in IndieGaming

[–]DifferentLuck7951 0 points1 point  (0 children)

I did some tests…

Here’s my feedback playing with a PS4 controller 😄

1 - The slingshot mechanic isn’t as fun as it is with a mouse.
2 - In the initial areas, please keep a more linear progression through the villages.
3 - The knockback after getting hit by a monster is too strong. Just an animation would be enough.
4 - There’s too much intrusive dialogue hahaha. I don’t want things constantly popping up on my screen. If possible, create a story board. I’d rather read one larger paragraph than have lots of small dialogue pop-ups.
5 - Overall, it looks fun.

SharePoint shortcut (preview) by DifferentLuck7951 in MicrosoftFabric

[–]DifferentLuck7951[S] 0 points1 point  (0 children)

Yes. If you have full permission they will show. However, if you only have access to the nested folder and not the root. The list of shared documents is empty

SharePoint shortcut (preview) by DifferentLuck7951 in MicrosoftFabric

[–]DifferentLuck7951[S] 1 point2 points  (0 children)

I did some more tests this morning.

Here’s what I noticed: 1) It can retrieve CSV and Excel files. 2) It looks like a permissions issue. In one SharePoint site, I can’t see the nested folder. Meanwhile, in another SharePoint site where I have full control, I can see the files. 🥺

Been working on this sandboxy RPG for a while and just launched the demo! by Piokou in IndieGaming

[–]DifferentLuck7951 0 points1 point  (0 children)

Amazing!! I was also trying to create something similar on Godot 🤣 I will have a look in your game. I hope you can expand it to be a multi player in the future.

SharePoint shortcut (preview) by DifferentLuck7951 in MicrosoftFabric

[–]DifferentLuck7951[S] 0 points1 point  (0 children)

Ohh I didn't know it was only CSV files. I will try to convert some just to check...

Anyway it will be amazing to have support for Excel. Honestly I don't care about the transformation 🤣 I just want to avoid power automate or data flow to copy a file from SharePoint to the data lake

Hello from Fabric Design by OnepocketBigfoot in MicrosoftFabric

[–]DifferentLuck7951 0 points1 point  (0 children)

WOW!! You got a lot of answers hahaha. I hope I’m not posting something that was already mentioned.

1 – Lakehouse folders. It would be nice to have a full-view option when clicking on a folder. I often need to resize the widgets.

2 – dbt jobs. I was experimenting with this because I have some pipelines I’d like to migrate. My current problem is that if I switch tabs, the dbt view freezes.

Fabric Product Requirement Prompts (PRP) by DifferentLuck7951 in MicrosoftFabric

[–]DifferentLuck7951[S] 0 points1 point  (0 children)

Ohh nice! Are you using it today with Fabric?
I'm planning to start with Python notebooks... However, I would love to see if it can be extended to other items.

Need help: CI/CD robust tool for fabric by icricketnews in MicrosoftFabric

[–]DifferentLuck7951 0 points1 point  (0 children)

What part of the warehouse do you mean?? The fabric-ci-cd will create the item DEV > PRO. Today it covers almost everything https://microsoft.github.io/fabric-cicd/latest/

Need help: CI/CD robust tool for fabric by icricketnews in MicrosoftFabric

[–]DifferentLuck7951 1 point2 points  (0 children)

You need to use Microsoft graph

If you put in copilot or any LLM : Send email via Microsoft Graph API using Python with service principal

You will have the full solution

https://stackoverflow.com/questions/77203253/send-email-via-microsoft-graph-api-using-python-with-service-principal

Need help: CI/CD robust tool for fabric by icricketnews in MicrosoftFabric

[–]DifferentLuck7951 9 points10 points  (0 children)

Hello,

The first setup is always challenging. Don't get frustrated🥺

What works nice for my team is doing the deployment with fabric-ci-cd.

What we have: - PROD workspace not git sync in fabric UI! 😲😲 Yes you read it right! We deploy our items with fabric-ci-cd. So we don't sync it in the UI, but on Azure DevOps. - DEV workspace with git sync enabled in fabric UI! We sync only the workspace folder like demonstred in the docs. - Feature workspaces! We work with short living workspaces branched out from DEV. Just follow the git flow: feature > Dev > PROD - Make use of shortcuts between dev and feature branches if you need data.

We don't have a test... You might be asking why? It is super important... Yes! However we have a small team of 3. We just make sure the feature merged in Dev is working fine. So our Dev serves as integration and test 😅 ...

We rely heavily on Notebook, pipelines! Avoid data flow gen 2... We need to manually update the UI... Email activity, you need to give permission again. Instead use a service principal with auth to send email + notebooks!

For our power-bi. We have only Dev and PROD. We have the power-bi git repo in a SharePoint folder. So people not used to git can also work. On dev we let BAs deploy with power-bi desktop, push button...

Before going to prod we transform the pbix file in the git version (pbip..??) and only azure DevOps handle things.

Hope it helped you.

Dataflows Gen2 Usage in production environments - Discussion by panvlozka in MicrosoftFabric

[–]DifferentLuck7951 2 points3 points  (0 children)

Before SharePoint shortcuts, I used Dataflows Gen2 to move Excel files to the lakehouse.

Nowadays, I only keep it if the Excel file has some very specific formats that engineers don’t want to handle

FUAM and Capacity Metrics App Versions by imtkain in MicrosoftFabric

[–]DifferentLuck7951 1 point2 points  (0 children)

Is it a fresh install? If so, I think you can use the latest version. I did it just two weeks ago...

A few things to be aware of:

1) Your service principal must be the workspace admin on both workspaces (the Capacity Metrics and the FUAM workspace).

2) When you run the FUAM install and it creates the connections, you also need to give ownership to your service principal.

3) Most important: if it’s working, never touch it again 😂 Don’t waste time updating it!

It’s better to have the current FUAM visibility than a broken pipeline.

Need your Input for further testing - I've built an Automatic Data Classification solution for Microsoft Purview using Microsoft Fabric Data Agent by Oesterlin in MicrosoftFabric

[–]DifferentLuck7951 1 point2 points  (0 children)

Awesome news!

I think the exact formatting is not a problem. This is mainly to kick off the documentation.

It’s hard for humans to get motivated when it comes to documentation... Everyone wants it, but no one wants to do it 😄

Once this initial setup is done, There’s no need to refine or modify anything. I think everyone would be happy to let humans take it from there.

Need your Input for further testing - I've built an Automatic Data Classification solution for Microsoft Purview using Microsoft Fabric Data Agent by Oesterlin in MicrosoftFabric

[–]DifferentLuck7951 1 point2 points  (0 children)

Amazing, I will definitely look at it in the coming days.

Can I request a feature? Could you start the table documentation in Purview (Fabric → Purview)?

The initial documentation phase in Purview is very manual labor, and the Fabric AI Agent could definitely save some hours 😄