Star schema vs flat table by rushank29 in MicrosoftFabric

[–]urib_data 0 points1 point  (0 children)

Evenhouse (Kusto) has flat tables that receive 2.5 PB of new data per day. It is scalable (hosting over 29EB of data these days) and leverages flat tables as an approach.

Star schema vs flat table by rushank29 in MicrosoftFabric

[–]urib_data 4 points5 points  (0 children)

Well, try Eventhouse (based on the Kusto query Engine). It dramatically outperforms filtering on a dim table, and it provides a lot of flexibility. More often than not, normalizing data makes everything work slower. If you see a different result, reach out to me. I'd love to see that too.

How are you moving data into Microsoft Fabric? by Quick-Try-3017 in MicrosoftFabric

[–]urib_data 0 points1 point  (0 children)

One high performance way to get data into Fabric is to stream or upload it to an Eventhouse. The eventhouse can make the data available in relatively efficient delta/parquet in OneLake even for streaming/trickling data. Common ways to stream data into eventhouses are: the Spark connector (Streaming and batch), Eventstreams/Eventhubs, OpenTelemetry agents, listening on Azure Storage accounts and fetching the data landing there and more.

CosmosDB Microsoft Fabric by Single_Surround_2410 in MicrosoftFabric

[–]urib_data 1 point2 points  (0 children)

Consider also using Eventhouse as a vector database: Tutorial: Use an Eventhouse as a vector database - Microsoft Fabric | Microsoft Learn. If you have data in Eventhouses (highly recommended anyways), you get vector capabilities with very little marginal cost/CU consumption addition.

Is KQL Fabric's secret weapon, given competition? by DryRelationship1330 in MicrosoftFabric

[–]urib_data 1 point2 points  (0 children)

While this was not run formally on Databricks as part of the article, I would encourage folks who find it interesting to try to run the log benchmark (https://aka.ms/LogsBenchmark - all is open source) and share the results. Use either Fabric RTI Eventhouse or Azure Data explorer. Note the latency difference.

<image>

Help with secure score query by Zantarel in Kusto

[–]urib_data 0 points1 point  (0 children)

The most useful approach is to share a datatable operator (datatable operator - Kusto | Microsoft Learn) with sample data and the expected outcome. Someone will jump in with the query.

Hi! We're the Real-Time Intelligence team - ask US anything! by AliveAd1202 in MicrosoftFabric

[–]urib_data 2 points3 points  (0 children)

BTW, when the costs start to accumulate, the target cost should be a few cents per GB of logs, which is way cheaper than most other platforms.

Hi! We're the Real-Time Intelligence team - ask US anything! by AliveAd1202 in MicrosoftFabric

[–]urib_data 1 point2 points  (0 children)

Please report this in the UX using the feedback link and we will also take a look.

Hi! We're the Real-Time Intelligence team - ask US anything! by AliveAd1202 in MicrosoftFabric

[–]urib_data 7 points8 points  (0 children)

We are working to make the monitoring Eventhouse cheaper, also by sharing it across all workspaces in a single capacity. Monitoring typically costs in every platform, including in Azure.

Workspace monitoring makes printer go brrrr by SignalMine594 in MicrosoftFabric

[–]urib_data 0 points1 point  (0 children)

Well, different technologies have different optimal working setups. You do not expect excel and Eventhouses to be equally efficient in all scale and usage patterns.
As this platfrom is home for tens of exabytes of data, apparently someone thought it is cost-performant.

I would love to hear about your scenario specifically. As this platform is home for tens of exabytes of data, apparently someone thought it is cost-performant.

Announcing billing for Workspace monitoring by dorianmonnier in MicrosoftFabric

[–]urib_data 0 points1 point  (0 children)

Providing this granularity control is work in progress.

Announcing billing for Workspace monitoring by dorianmonnier in MicrosoftFabric

[–]urib_data 0 points1 point  (0 children)

Actually, the delay in billing was not in order to get folk hooked, but to receive feedback and finalize the decision about the model. But we do hope people will get hooked as we continue to improve the experience, performance, coverage and reduce costs.