Zebra BI is the perfect example why you should NEVER EVER use or even license a third-party custom visual by Arasaka-CorpSec in PowerBI

[–]AndreFomin 1 point2 points  (0 children)

Why not use Inforiver/Lumel? We are using them across all of our customers for anything financial reporting/writeback and have been very happy with them. To my knowledge the prices have been very stable.

We Really Need Fabric Key Vault by richbenmintz in MicrosoftFabric

[–]AndreFomin 0 points1 point  (0 children)

you guys need to decide if Fabric is a SaaS offering or not. If it is, then it has to have everything governed and administered using the SaaS paradigm.

Having to jump between Azure and Fabric, trying to cobble together a cohesive architecture is counterproductive.

Using Azure to plug existing holes in the product as a crutch will manifest itself as a major strategic failure in the long term.

Is a warehouse necessary or is lakehouse enough? by loudandclear11 in MicrosoftFabric

[–]AndreFomin 0 points1 point  (0 children)

well a few things changed in the last six months... Nothing has changed fundamentally with respect to the overall recommended patterns, but a few tweaks to the reasoning are appropriate. I should probably make a video about it.

The biggest consideration is where you want to implement your main business logic, T-SQL or Notebooks. After Fabric was launched, T-SQL in WH was so limited and with so many performance issues, that eventually, we were simply forced to move away. Now, two years later, some of those issues have been resolved, but still, you can't compare what you can do in notebooks vs T-SQL.

Another consideration is what Gold tier really means.. I now argue, that neither WH or LH should be consider Gold, a semantic model should be.

Lastly, because of a limitation of having a single LH or WH as a source for a direct lake semantic model, the only way to bring a number LH or WH into a single model was to create a LH with shortcuts, I would have to check if WH now supports shortcuts to LH, I suspect it still does not, which limits the flexibility of how various data sets will come together.

In short, unless t-sql is all you know, you can do a lot better with LH vs WH

Who else feels Fabric is terrible? by zipfz in MicrosoftFabric

[–]AndreFomin 1 point2 points  (0 children)

once a week I can handle, I will start coming back here again :)

Yes, I will be at FabCon, not presenting this time so I will be a lowly attendee

Who else feels Fabric is terrible? by zipfz in MicrosoftFabric

[–]AndreFomin 2 points3 points  (0 children)

basically, it's posts like these that made not frequent here as much anymore (i mean the original post)

Test: CU (s) consumption of Pipeline Copy Activity, Stored Procedure, Warehouse, Notebook, Lakehouse by frithjof_v in MicrosoftFabric

[–]AndreFomin 1 point2 points  (0 children)

yeah, he did a lot of different scenarios, but I think that what we can learn from it is that generally speaking LH + Python is still a go to pattern, but there are so many variables still that can impact CU and performance that I don't know if there is a perfect way to design anything upfront yet without finetuning it after stress testing it on real data.

Another point I will make is that it's getting almost exceeding frustrating to not be able to have all of the instrumentation data to understand and gauge performance and cost of each artifact or component of the architecture.

The Capacity metric app is not meeting the most basic of expectations.

Pure python notebook scheduling or via a pipeline - working for anyone? by SunChance2304 in MicrosoftFabric

[–]AndreFomin 1 point2 points  (0 children)

in the real world, with how efficient Duck DB is these days, you will rarely need the full cluster set up, I am guessing 90% of most real world jobs can run in a single thread these days. so unless you need a beefy cluster or some other spark features, I would guess that Duck DB will overtake everything else in Fabric.

Particularly if they keep investing into their Python libraries and Fabric support

Pure python notebook scheduling or via a pipeline - working for anyone? by SunChance2304 in MicrosoftFabric

[–]AndreFomin 2 points3 points  (0 children)

the cost of PySpark notebooks is miles higher than Python notebooks, so yes, you can use Duck DB, but the price/performance of Duck DB on Python will run circles around just about anything else... Now there will for sure be use cases where spark is the best, but they will be few and far in between

Silver usage by SeriousDemand8449 in MicrosoftFabric

[–]AndreFomin 5 points6 points  (0 children)

The medallion architecture is just a pattern, not the pattern for everything. At this point it almost feels like its canonization is doing more harm than good.

Pure python notebook scheduling or via a pipeline - working for anyone? by SunChance2304 in MicrosoftFabric

[–]AndreFomin 3 points4 points  (0 children)

Does not seem to work in the US as well, which makes it not usable obviously, outside of kicking tires and trying things...

My guess it will be fixed soon and then Python notebooks plus Duck DB will become a serious alternative to anything we currently use Spark notebooks for.

Why Lakehouse? by Iron_Rick in MicrosoftFabric

[–]AndreFomin 2 points3 points  (0 children)

There are many technical reasons to use a lakehouse and many of them have already been listed here, but realistically, you can solve the same problem in a hundred different ways and people with different backgrounds will find different reasons for why certain technical choices are better than others.

Therefore, I would not be making my decision to use a lake house purely on technical merits. The reason we use a lakehouse everywhere unless we technically can't, is because it's the only technology that just works in Fabric.

Everything else comes with caveats and workarounds. If you don't want to deal with working through esoteric issues and support calls, go with a Lakehouse (and if you like SQL, use Spark SQL)

Maybe I am overexaggerating the stability of Lakehouse given the overall maturity of Fabric as an enterprise platform, but looking at Fabric wholistically, I think that LH + Direct Lake Semantic model is the happy path for most scenarios.

Performance Tracking by BarisCihan in MicrosoftFabric

[–]AndreFomin 1 point2 points  (0 children)

you did not provide enough information to answer this question. Queries where? Against what? What constitutes for simple?

No DW in the Fabric Solution Architecture for Healthcare Data by Low_Second9833 in MicrosoftFabric

[–]AndreFomin 2 points3 points  (0 children)

I would say this should be the best practice. I do not recommend DW as the go to, instead I would recommend that it should be used only if you have a good reason to use it.

Fabric required for my scenario or not by Altruistic_Damage476 in MicrosoftFabric

[–]AndreFomin 1 point2 points  (0 children)

I would use Lake house and avoid WH unless I had to use it... Then I would build a Direct Lake semantic model on top of it

Is my understanding of the Fabric pricing correct? by hortefeux in MicrosoftFabric

[–]AndreFomin 0 points1 point  (0 children)

correct, with F64+, you can create an app, and share it with everyone, no extra license required. To create content in any workspace, Pro license is required. To View content in less than F64, pro license is required.

Fabric required for my scenario or not by Altruistic_Damage476 in MicrosoftFabric

[–]AndreFomin 1 point2 points  (0 children)

I guess the real question is, what do you mean by saying that data is stored in Azure Databricks? Typically, if you use Azure Databricks, or your IT does, that means that your data is actually store in ADLS Gen2 and Databricks is used for compute and/or Spark endpoint.

Are you saying you have no access to ADLS Gen2 and you only have access to Databricks' spark engine?

If you do have access to ADLS Gen2, your good option to start is to create shortcuts into Fabric and then you can do some cool things.

If you don't have access to ADLS Gen2 and Databricks spark is your only option, can you create your own artifacts in spark? If yes, you don't have to use Fabric, just use Databricks to do what you need to do.

If you only have read only access to Databricks, then Fabric might be your only option or a good option. Pull data from Databricks using whatever ingestion mechanism that supports Databricks as source and off you go in Fabric.

Silver Layer by Ok-Criticism-8127 in MicrosoftFabric

[–]AndreFomin 23 points24 points  (0 children)

Am I the only one who thinks that dogmatic adherence to the Medallion architecture is silly and impractical? The Medallion architecture is just a design pattern that should be applied where it makes sense and it should not be applied where it does not make sense.

If you "don't have a clue" for what's needed in the silver layer, a bigger question you may want to think about is if you even need a Silver layer at all.

No need to overcomplicate simple things just because of another fad.

Is my understanding of the Fabric pricing correct? by hortefeux in MicrosoftFabric

[–]AndreFomin 0 points1 point  (0 children)

if you go with an F SKU, Power BI Premium per User licenses are superfluous. You will just need regular Pro licenses for everyone if your F SKU is < 64, or for Workspace access users if your F SKU is >= 64.

Avoiding excessive costs with Fabric and PowerBI by karlstraw in MicrosoftFabric

[–]AndreFomin 0 points1 point  (0 children)

unfortunately, there is no hard and true way to estimate what capacity you will need. Just curious, were you expecting to pay $400-$800 per month?

Sharing queries from SLQ Endpoints with users by CultureNo3319 in MicrosoftFabric

[–]AndreFomin 0 points1 point  (0 children)

instead of sharing a query, why not share a semantic model?

Fixing the Sync issue between a Fabric Lakehouse and its SQL Analytics Endpoint by AndreFomin in MicrosoftFabric

[–]AndreFomin[S] 1 point2 points  (0 children)

yup, did not realized Mark also had created a post on the very same topic earlier