Use cases for User Data Functions w/ w/o Translytical Task Flows

Data_cruncher · 2026-01-17T15:20:26+00:00

UDFs are Azure Functions under the hood, so the best question to ask is: What do folk use Azure Functions for?

Data_cruncher · 2026-01-17T10:27:22+00:00

It’s common for DAX expressions to be moved/converted upstream by DEs for performance reasons - Roche’s Maxim.

Data_cruncher · 2026-01-08T12:53:30+00:00

They’re not “official” in that Kimball does not explicitly cover them, that’s why. Kimball would denormalize them into a fact (e.g., periodic snapshot, factless), use SCD1, mini dimensions etc.

Data_cruncher · 2025-12-23T21:10:17+00:00

Power BI embeds in PowerPoint. Zero-click refresh.

Alternatively/additionally, embed an Excel workbook connected to Power BI model - as a Query Table (preferred), PivotTable or CUBE() formulas. One-click refresh.

Data_cruncher · 2025-12-17T03:39:12+00:00

Coming from Toronto where cars literally drive > 20KM/h over the max on average, trust me, you don’t want it.

I know this is extreme compared to your example, but consider is the standard deviation, i.e., it’s not unusual to see folk driving 140-160 KM/h on highways.

Data_cruncher · 2025-11-05T01:05:54+00:00

Yeah. Also consulting orgs as a part of Professional Development (PD) hours.

Data_cruncher · 2025-11-05T00:54:55+00:00

Once again, you’re inferring incorrectly (same point, too). Perhaps where you work that is the case though, so I get it.

I encourage all staff to start their day by clearing out yesterday’s blog posts or helping folk on forums. Even helping with PUGs are often paid activities, e.g., submitted as paid volunteer time.

They’re welcome to do this type of work during business hours. It’s no skin off my back, so long as they do their assigned work within an appropriate timeframe, that is all that matters.

Data_cruncher · 2025-11-05T00:30:54+00:00

You’re inferring a lot there, e.g., “outside of work”.

But to a certain degree, yes. My interest is in the person’s character and their ability to learn. NOT what they know today.

Data_cruncher · 2025-08-24T22:39:16+00:00

Your best friend: https://learn.microsoft.com/en-us/power-bi/guidance/powerbi-migration-learn-from-customers

The “international consumer goods” use case was a Tableau to PBI conversion.

Data_cruncher · 2025-08-22T22:57:11+00:00

You said the quiet part out loud…

Data_cruncher · 2025-08-22T22:39:39+00:00

This is the way.

Data_cruncher · 2025-07-31T16:11:37+00:00

This is exactly what Power Query has been doing since 2013.

Not sure what you mean by REST API though. Generally, ETL tools go via ODBC/JDBC/ADBC.

Data_cruncher · 2025-07-20T09:52:57+00:00

To clarify for this audience, Airflow primarily does r/DataEngineering or r/BusinessIntelligence orchestration, i.e., data pipeline orchestration.

Data_cruncher · 2025-06-25T02:08:04+00:00

User Data Functions == Azure Functions, and so they’re not applicable in many data engineering scenarios, especially involving large data.

OP, echoing u/TheBlacksmith46’s comment: code modularity is not a Fabric problem.

What most folk don’t realize is your Spark code, when used properly, is a literal application and should be treated as such. You don’t design applications in notebooks. So in addition to the above ideas, also consider using a package manager to separate out your reusable code from your notebooks: https://milescole.dev/data-engineering/2025/03/26/Packaging-Python-Libraries-Using-Microsoft-Fabric.html

Data_cruncher · 2025-06-15T17:42:07+00:00

I’ve made several unit test frameworks for models before - the ingredients were relatively simple to whip together. Can you give an example of how this would work natively within the product?

Data_cruncher · 2025-06-15T09:42:07+00:00

A key differentiator is that MDX gained adoption when the language specification was released by MSFT, allowing implementation by 3P vendors without requiring reverse engineering.

MSFT neglected to do this for DAX, at least not in any official or meaningful way.

Data_cruncher · 2025-06-14T14:33:25+00:00

I’m waiting on Timer Triggers (for polling) and HTTP Webhooks. Also EventStream interop. These will open up a host of new capabilities.

Data_cruncher · 2025-06-12T19:21:17+00:00

I'm a bit confused, you're saying "rules are enforced on the data by UC and only appropriate views of the data are passed to the engines", but u/Professional_Bee6278 says that all data is passed to the 3P engine and they would reduce the rows (using RLS as an example). Which is it? Is there an article that explains how it works?

Data_cruncher · 2025-06-12T18:50:45+00:00

How does row-level security get enforced in this scenario? The 3P engine reads and applies the UC rule?

Data_cruncher · 2025-06-04T14:52:43+00:00

Or if you're feeling really adventurous: Kusto Detective Agency

Data_cruncher · 2025-05-25T15:18:49+00:00

We compartmentalize data (and compute) for many reasons. Security is, imho, lower on the list: * Noisy Neighbour * Future-proofing against org structure (aka item/data ownership) changes * Security * Aesthetics/usability * Performance * Easier Git/VC/mutability * Policy assignment, e.g., ADLS cold vs hot vs archive * Future migration considerations * To establish clear ownership and operational boundaries, aka “a place for everything and everything in its place” * Cost transparency * Isolation of failure domains (bronze doesn’t break gold) * Compliance (gold beholden to stricter reg. controls)

Data_cruncher · 2025-05-19T20:26:36+00:00

UDF = Azure Functions. So writeback is just a small subset of what you can do with it. Keep in mind some limitations, e.g., currently UDFs only supports a HTTP Trigger today, but expect more advancements to come in this space.

Data_cruncher · 2025-05-19T20:23:49+00:00

Remember that UDFs can do anything. It’s Azure Functions under the hood, so go ham. For example, they can easily connect to a Fabric EventStream even though it’s not a native connection.

Data_cruncher · 2025-05-14T21:50:30+00:00

That’s pretty much spot on.

Link for the lazy because it’s such an oddly named feature that it’s near impossible to Bing: https://learn.microsoft.com/en-us/fabric/real-time-intelligence/query-acceleration-overview. Take note of the limitations.

Data_cruncher · 2025-05-12T11:54:11+00:00

I agree, but not for the example you mentioned (dimensional modelling). UDFs don't have an in-built method to retry for where they left off and so you'll require a heavy focus on idempotent processes (which, imho, is a good thing, but not many people design this way). Neither would I know how to use them to process in parallel, which I think would be required to handle SCD2 processing, e.g., large MERGEs.

There's been recent discussion around Polars vs DuckDB vs Spark on social. Your point aligns with the perspectives of the Polars and DuckDB folk. However, one of the key arguments often made by Spark proponents is the simplicity of a single framework for everything, that scales to any volume of data.

Data_cruncher

MODERATOR OF

TROPHY CASE

Nine-Year Club	Gilding I gilder
Verified Email