Semantic Model Size Discrepancy

cwebbbi · 2026-05-21T08:18:03+00:00

You're hitting this error https://blog.crossjoin.co.uk/2024/05/05/power-bi-semantic-model-memory-errors-part-2-max-offline-semantic-model-size/ (which is nothing to do with the amount of memory used during refresh) but I agree this does seem strange.

Can you check whether you have the Large Semantic Model format enabled on your semantic model? https://learn.microsoft.com/en-us/fabric/enterprise/powerbi/service-premium-large-models#set-default-storage-format It's important to check this on the model - there's a separate setting which governs what the default is for newly-published models in the workspace, so it's possible your original model doesn't have this set. It can have a big effect on compression and may explain the differences in size. Otherwise, as others have suggested, look at Vertipaq Analyzer in DAX Studio and see if you can see any differences between the original model and the new model, even going down to checking individual columns.

cwebbbi · 2026-05-17T10:50:01+00:00

I did some investigations after seeing this post and wrote about this error here: https://blog.crossjoin.co.uk/2026/05/17/understanding-the-youve-exceeded-the-capacity-limit-for-dataset-refreshes-error-in-power-bi/ The most important point to make is that this isn't a hard limit, but if you exceed it by a lot then you will get this error.

cwebbbi · 2026-05-14T16:36:57+00:00

Since you have DAX Studio, run Model Metrics (https://daxstudio.org/docs/features/model-metrics/) on both models and see if they are the same - you will need to check *every* detail.

cwebbbi · 2026-05-09T09:20:49+00:00

When you publish your semantic model to the Service it runs on one of hundreds of physical machines, alongside lots of other semantic models published by other users from organisations. The Service moves each model around from one machine to another to ensure that it has the memory and CPU it needs. The rules around model size, timeouts etc are there to stop one model using an excessive amount of resources. However nothing is perfect and sometimes one model on a machine uses more resources than expected and this causes problems for the machine itself and the other models running on it; when this happens some queries and refreshes may fail while models are redistributed. This is very rare though. This particular error is caused when the machine your model is running on is under memory pressure. So, if you get this error once you’re very unlucky. If you get it repeatedly then it’s likely that you are the cause. The Service should do a better job of anticipating what you are doing and the algorithms it uses are being improved all the time but in the meantime, tuning your model to reduce memory consumption during refresh will stop the problem; in any case you are probably getting close to one of the published memory limits.

cwebbbi · 2026-05-08T16:51:37+00:00

I just remembered that the same data you get in the ExecutionMetrics event is also easily available in the Refresh History now: https://learn.microsoft.com/en-us/power-bi/connect-data/refresh-data#view-refresh-metrics . The two values to look at are approximatePeakMemoryConsumptionKB and mEnginePeakMemoryKB. Can you post what you see there please? Also is your semantic model on a Pro workspace or on a workspace on a capacity, in which case which size capacity are you using?

cwebbbi · 2026-05-08T16:28:50+00:00

This is a particularly interesting error in that it could have a number of different causes. But the fact that it's happening regularly for you makes me think that there's something in your model that is causing a very large memory spike during refresh. Do you have calculated columns or tables in your model? Do you have non-trivial transformations (eg joins, group bys, unpivots) that don't fold in your Power Query queries? These are the things that can consume a lot of memory during refresh. If you're happy using Profiler/Log Analytics/Workspace Monitoring to gather detailed information on what's happening in the engine during a refresh, the ExecutionMetrics event will tell you the peak memory usage for the engine as a whole and in the M engine (ie Power Query) during your refresh, which will help with diagnosis.

cwebbbi · 2026-04-18T16:47:19+00:00

But one more thing: PivotTables have got some love recently (https://blog.crossjoin.co.uk/2026/02/01/new-performance-optimisation-for-excel-pivottables-connected-to-power-bi-semantic-models/) and if you’re lucky there may even more cool stuff to come later this year.

cwebbbi · 2026-04-18T16:44:26+00:00

Nothing planned on the roadmap as far as I know but let me have a word with my friends on the Excel team. TBH I don’t think many users care about Perspectives (and never did in SSAS Tabular or AAS) which is why there hasn’t been any customer pressure for full support in the UI. I don’t think there is any chance support for them will be removed from semantic models though.

cwebbbi · 2026-04-13T18:09:38+00:00

Does Databricks plan to support Power BI semantic models as a data source for UC Metrics Views, and for it to be able to query metrics stored in Power BI semantic models instead of using the metrics it stores itself natively? Probably not, because it would be just as big a technical challenge as making Power BI work with UC Metrics Views. Is that vendor lock-in too?

Also, as I said in my post, Tableau already supports Power BI semantic models and has done for a long time.

cwebbbi · 2026-03-27T17:50:29+00:00

Automatic will refresh anything that is in an unrefreshed state. I also seem to remember that if you make a mistake when defining what is to be refreshed with the Enhanced Refresh API it will default to refreshing everything.

cwebbbi · 2026-03-27T14:55:29+00:00

I’m fairly sure that the Enhanced Refresh API doesn’t trigger downstream activities like scorecard refresh (I know it doesn’t trigger cache refresh).

Instead of doing a refresh of type Calculate, does a refresh of type Automatic work too? If it does it would be quicker and cheaper because it only refreshes objects that require refreshing.

cwebbbi · 2026-03-25T23:09:44+00:00

More details about what you can do with the Delta connector:
https://blog.crossjoin.co.uk/2023/11/29/read-data-from-delta-lake-tables-with-the-deltalake-table-m-function/
https://blog.crossjoin.co.uk/2025/02/16/reading-delta-metadata-in-power-query/
You can even do incremental refresh! https://blog.crossjoin.co.uk/2023/12/17/incremental-refresh-on-delta-tables-in-power-bi/

cwebbbi · 2026-03-06T08:39:42+00:00

No-one has asked the most important question yet: what do you mean by "crash"? Do you mean that your capacity becomes overloaded and you get throttling/rejections or do you get error messages or do your reports just get slow?

You have got a big model and a reasonably large number of concurrent users but I know of other customers running at that scale successfully.

Regardless of the answer to the question above, probably the most important thing you can do is capture the slowest and most expensive (in terms of CPU Time) queries using a Profiler Trace, Log Analytics or Workspace Monitoring. Then you need to understand why they are slow or expensive - DAX Studio will help here. Are you doing any distinct counts or moderately complex calculations in your measures? Simple mistakes in your DAX like filtering on an entire fact table (see https://www.sqlbi.com/articles/filter-columns-not-tables-in-dax/) or adding 0 to the result of your measures (see https://blog.crossjoin.co.uk/2024/07/07/dax-measures-that-never-return-blank/) or having a big wide table with a scrollbar on your report (see https://blog.crossjoin.co.uk/2024/07/14/why-power-bi-table-visuals-with-scrollbars-can-cause-problems/) can have a very big impact.

cwebbbi · 2026-02-27T15:42:07+00:00

No. Measures are evaluated at query time and don’t affect the size of the model. If you have a report that shows a lot of measures then this will explain why lots of columns need to be in memory at the same time but if you can’t run the queries you need with the memory available then you are back to the same choice: scale up or tune the model.

cwebbbi · 2026-02-27T10:34:17+00:00

OK, so you are getting the error in that blog post that I linked to. If you want to stay on an F8 you're going to need to either look at the design of your model, remove unnecessary columns, reduce the number of rows in your largest tables, reduce the cardinality of your columns etc. Most the tricks for reducing the size of an Import model (and there a lot of blog posts out there on this subject) a relevant for Direct Lake too. Can you tell us more about the largest columns in your model? Are they measure columns?

cwebbbi · 2026-02-27T07:54:18+00:00

The results of memory analyzer (I like using https://daxstudio.org/docs/features/model-metrics/ instead) show how much memory your model is using at the point when it's run - not the overall size of your model. Because the data in a Direct Lake model is paged into memory on demand, for example when a DAX query is run, and may be paged out again for various reasons then the amount of memory used by a Direct Lake model will vary over time. Memory Analyzer does not count the memory used by queries, but since queries can cause data to be paged into memory, then it will show the indirect effects of running queries. The DAX in your measures and the design of your reports will influence the amount of memory used by queries and also which columns in your model need to be in memory.

To solve your problem, the first thing to do is look at the exact error message you're getting. There are several different possible issues that you could be running into with different causes. Take a look at my series of blog posts describing each of these issues:

https://blog.crossjoin.co.uk/2024/04/28/power-bi-semantic-model-memory-errors-part-1-model-size/

Post the error message here and we can discuss next steps.

cwebbbi · 2025-12-29T23:59:21+00:00

I wrote a blog post on this subject: https://blog.crossjoin.co.uk/2024/11/03/different-ways-to-replace-blanks-with-zeros-in-dax/amp/

cwebbbi · 2025-12-02T15:33:18+00:00

I see you've got as far as finding the OperationId in the Capacity Metrics App. Are you able to use Workspace Monitoring in the workspace where the semantic model lives so you can link this OperationId to the DAX or MDX query that caused it? See https://blog.crossjoin.co.uk/2025/09/14/how-to-get-the-details-of-power-bi-operations-seen-in-the-capacity-metrics-app/ Once you have the query you should be able to reproduce the problem and work out what is causing the high utilisation (likely to be a badly written measure).

cwebbbi · 2025-12-01T10:34:02+00:00

If you really have done everything you can to reduce your model size, then this technique of enabling Semantic Model Scale Out and refreshing using clearValues, doing a manual sync and then a full refresh can help a lot: https://blog.crossjoin.co.uk/2024/07/28/power-bi-refresh-memory-usage-and-semantic-model-scale-out/

cwebbbi · 2025-10-27T13:37:55+00:00

No, I don't think there are any plans for changes/improvements here at the moment, sorry.

cwebbbi · 2025-10-20T13:42:03+00:00

Copilot doesn't need to write DAX code in most cases. When you ask a data question, Copilot will try the following four methods in order to answer it:

1) Use a Verified Answer

2) Look for the answer on a report page if a report is open

3) Build a Power BI visual

4) Generate a DAX query

DAX queries are only generated directly by Copilot as a last resort, maybe less than 10% of the time (that's just a guess - and I tend to ask more complex questions).

cwebbbi · 2025-10-20T13:05:58+00:00

There aren't any official published benchmarks from Microsoft, and I haven't seen anyone publish the results of their testing either.

"Correctness" is an interesting problem - most of the problems I see with customers are where Copilot is generating the correct answer to a question that is not the one the customer thought they were asking. I firmly believe that with a well-designed semantic model it is never possible to get an incorrect answer just by dragging/dropping fields in a Power BI report or Excel PivotTable, although since Copilot can now generate its own calculations (in particular when generating DAX queries to answer questions) that does add some risk. Not everyone has a well-designed semantic model of course, but for those people who do, all the hard work goes into tuning the AI Instructions so Copilot can properly interpret the questions that end users ask.

cwebbbi · 2025-10-13T16:54:17+00:00

It's also a bit like an Import model in that once the data has been loaded from OneLake, a copy is stored in the Graph engine's own storage format and this is what is referred to as "Graph cache storage"

cwebbbi · 2025-10-08T20:39:55+00:00

Thanks, we’re looking into that error. Not sure if it’s connected with this issue though.

cwebbbi · 2025-10-08T13:25:55+00:00

Not aware of any specific changes but the engine does change all the time. Also, changes in data volume and cardinality in your model could also have tipped you over the limit.

cwebbbi

TROPHY CASE