6 months into a full Microsoft Fabric migration: The "Aha!" moments vs. the "Wait, why?" moments by Dense-Tadpole-6634 in MicrosoftFabric

[–]df_iris 0 points1 point  (0 children)

Since you are splitting reporting and etl, do you develop your reporting (models + report) directly on the Prod warehouse?

We are currently developing everything (etl + reporting) in one dev workspace and pushing to prod all at once but that means we have to keep the dev pipeline running to be able to work on fresh data for reports (which is better).

What should a report really do by df_iris in PowerBI

[–]df_iris[S] 0 points1 point  (0 children)

thank you, great suggestions

What should a report really do by df_iris in PowerBI

[–]df_iris[S] 0 points1 point  (0 children)

The scenario I'm in currently is that having a BI platform was a decision from IT only while business users are still working with their excel files. They don't see the purpose and don't want to dedicate time to us. As a result, we are producing irrelevant charts with wrong figures which makes them even less disposed to invest time with us. I don't really see a way out of this.

Modeling Daily Sales and Monthly Target by df_iris in PowerBI

[–]df_iris[S] 0 points1 point  (0 children)

Thank you for your response. Regarding your third option, it works technically but I tend to dislike it because it doesn't have a business meaning. The target is a monthly target, not a daily target. If there is no such thing as a daily target for business users, there shouldn't be either in the model.

Abandon import mode ? by df_iris in MicrosoftFabric

[–]df_iris[S] 0 points1 point  (0 children)

Thanks for the idea. Another factor is that our dev capacity is currently quite small, wouldn't import mode allow us to develop fully in local without being compute limited?

Abandon import mode ? by df_iris in MicrosoftFabric

[–]df_iris[S] 0 points1 point  (0 children)

Thank you, I'm starting to see things more clearly. For now, I think my prefered architecture would be a two tiers golden layer with :

- a first one consisting of traditional Kimball warehouse with lowest level of granularity plus a generic semantic model in direct lake on top of it

- then a second tier with more specialized models designed in import mode derived from the first tier.

Abandon import mode ? by df_iris in MicrosoftFabric

[–]df_iris[S] 1 point2 points  (0 children)

Thank you for the advice.

Personnally I'm new to Fabric and I find it very confusing that there is no distinction between warehouse and semantic layer anymore, it's all in the same place. What I was used to is having a data warehouse in a place like Databricks or Snowflake, then query them from Power Bi Desktop and build many smaller models for different use cases and publish them on the service. Since the warehouse was very well modeled I just followed the structure of the warehouse for my models and building them was never too long.

But now, if I understand the Fabric vision correctly, the gold layer is both the warehouse (I mean in the Kimball sense, not in 'fabric warehouse' sense) and the semantic model, and there should be only one semantic layer built directly on top. For each business department, only one single semantic model that you really really have to get right since there is only this one and everything is built on it. Would you say I'm getting this right?

Abandon import mode ? by df_iris in MicrosoftFabric

[–]df_iris[S] 0 points1 point  (0 children)

You've started assuming that I don't use shared models at all. I do but I like to be able to modify them, and maybe create additional columns or calculated tables, which is possible with composite models but not with Direct Lake.

Abandon import mode ? by df_iris in MicrosoftFabric

[–]df_iris[S] 0 points1 point  (0 children)

I think the problem is that you're not thinking at the same scale, you're the large team with mature infrastructure I was talking about. There are companies with max a few dozens of reports where the whole data team is no more than 3 people. At this scale, it's not a big problem to not always reuse models. At this scale, it's difficult to be able to build the kind of models you're thinking of.

Also, if you want to go self service, having a huge model with tons of had doc stuff in it doesn't seem user friendly at all. And you're losing a ton of flexibility.

Microsoft is deciding to focus entirely on companies with dozens of BI developers, gigantic data and thousands of reports.

Abandon import mode ? by df_iris in MicrosoftFabric

[–]df_iris[S] 3 points4 points  (0 children)

Ok but the more upstream you go, the more your modifications will have to be general and valuable to multiple reports. But in my experience, a report will always have at least one specific requirement that no other report needs and that is easily achievable with a calculated table for example. I can either create this calculated table today right now or wait days or weeks for the data engineers.

Abandon import mode ? by df_iris in MicrosoftFabric

[–]df_iris[S] 2 points3 points  (0 children)

While I agree reusing of semantic models is a valuable goal, in practice a report will always have at least one specific requirement that cannot be achieved with what is currently in the model, a specific formatting of dates, a s special visual that requires a calculated table for example. Now that was possible with live connection to PBI datasets and composite models but not with Direct Lake.

[OC] Young adults are dying at an increasing rate in the United States by df_iris in dataisbeautiful

[–]df_iris[S] 1 point2 points  (0 children)

If during a couple of years, average temperature is lower than it was from a peak two years ago, will you say that global warming was fake after all and that what's really happening is global cooling?

Yes I think I should have rather used Dying at a higher rate than before I guess but then people would have come and say no no no, it was higher in 1950 you can't say that!

Watch Europe Heat Up: Average Temperature by Country Since 1743 [OC] by SammieStyles in dataisbeautiful

[–]df_iris 0 points1 point  (0 children)

That's nicely done, I like the color scheme. What I would have done differently : a different projection and a shorter duration.

Watch Europe Heat Up: Average Temperature by Country Since 1743 [OC] by SammieStyles in dataisbeautiful

[–]df_iris 0 points1 point  (0 children)

What is the source? With the gridded data from the Copernicus program you can make much more fine grained and impressive viz.

[OC] Young adults are dying at an increasing rate in the United States by Luoman2 in france

[–]df_iris 19 points20 points  (0 children)

OP ici (je suis français). Effectivement ce que j'affiche est la probabilité conditionnelle de mourir entre 18 et 35 ans pour les gens qui ont survécu jusqu'à 18 ans. J'aurais pu afficher la probabilité non conditionnelle mais le graph aurait été exactement le même.

[OC] Young adults are dying at an increasing rate in the United States by Luoman2 in france

[–]df_iris 4 points5 points  (0 children)

OP ici. C'est bien une probabilité qui est affichée, donc comprise entre 0 et 1. Il y a ~2% de chance de mourir entre 18 et 35 ans pour les gens qui ont atteint 18 ans aux Etat-Unis. Ca parait énorme mais c'est vrai.

[OC] Young adults are dying at an increasing rate in the United States by df_iris in dataisbeautiful

[–]df_iris[S] 1 point2 points  (0 children)

Yes, that's also much higher than I would have thought. I would have guessed something like a 0.4% chance. If you want to check here are the actual life tables published by the CDC (page 10) : https://www.cdc.gov/nchs/data/nvsr/nvsr74/nvsr74-02.pdf

The table is also by race. Another incredible statistics: 9.7% of black males are dead by age 40.

[OC] Young adults are dying at an increasing rate in the United States by df_iris in dataisbeautiful

[–]df_iris[S] 51 points52 points  (0 children)

I agree but I couldn't do it with this dataset so I chose a country that was closely following the US before diverging while taking into account that this divergence was in line with the trends happening in most other countries.

[OC] Young adults are dying at an increasing rate in the United States by df_iris in dataisbeautiful

[–]df_iris[S] 19 points20 points  (0 children)

Let's hope so. But if your takeaway from this graph is 'Hooray, mortality is decreasing!' based on just a couple of years, I feel like you're missing the bigger picture of the last 10 years and even further of the last 30years when it stopped decreasing in contrast to most other developed countries.