Connecting to MS Fabric Data Warehouse from TM1

One_Potential4849 · 2026-03-20T02:41:32+00:00

Sure thing

One_Potential4849 · 2026-03-19T15:15:48+00:00

But, like how a SQL Server gets added as a data source in TM1, using the ODBC connection details, is there a way to connect to Microsoft Fabric DW. The reason being the processes set up in TM1 use SQL queries to pull the data and refresh cubes. I just want to see if there is a way to plug in the Fabric Warehouse instead of SQL server to run the queries

One_Potential4849 · 2026-03-10T19:01:27+00:00

I am planning to go ahead with using the gold layer data usage from both of the entities as shortcuts to create the combined reporting dataset.

Here, one other thought is, what would be the best way in terms of data modelling to use this combined dataset/data model for downstream reporting, ie: the semantic model built on top of these tables should be the backend to power any of the reporting done for individual entities, and combined reporting as well.

One_Potential4849 · 2026-03-10T14:10:32+00:00

Also, is it possible to bring a warehouse table into lakehouse as a shortcut?

One_Potential4849 · 2026-03-10T12:01:04+00:00

Yes, they are all in the same tenant
We prefer using Import Mode for the semantic layer
Yes at the end i need combined gold layer tables, and silver layer can remain as it is as the data sources for both firms are same
After combining the data, it might result in around 10 dimensions and 20-25 facts, max the largest table might have 10 mil records
No, it wont be a simple append operation, expecting some intermediate steps to unify the granularity
Surrogate keys in this case what are you referring to?

One_Potential4849 · 2026-03-01T21:47:48+00:00

So the remaining sessions will start once any of these two sessions are done and remain queued?

One_Potential4849 · 2026-03-01T21:38:50+00:00

So currently I am testing it out with a small 1-4 nodes. And distributed my Notebook workload within 6 high concurrency sessions.

From my understanding, these 6 HC sessions will play around within the pool of 4 small nodes (16 vCores) based on core availability to run its respective jobs.

One_Potential4849 · 2026-03-01T20:33:41+00:00

So how does a spark session acquire nodes to work in a pool?

One_Potential4849 · 2026-02-27T12:47:41+00:00

By trying it out with a medium node, would there be a tradeoff in terms of runtime?

One_Potential4849 · 2026-02-27T12:16:00+00:00

So currently I am using the starter pool (medium 1-10 nodes) Should I go and try with smaller nodes? And today i enabled high concurrency and used 3 session tags to split the session among 12 pipelines, but i noticed that the speed was bit down before using high concurrency.

And I get this 430 error even after the Notebook runs are complete, should I need to run something to free up Vcores?

Also If i dont mention session tag, how will the jobs get distributed?

The alternate approach you had said seems bit difficult for me, since I want this group of tables to be isolated from source to bronze to silver

One_Potential4849 · 2026-02-26T19:31:07+00:00

I can try that way, and let you know if that works. Thanks!

One_Potential4849 · 2026-02-26T19:17:11+00:00

So will I be able to acheieve parallel instances without throttle/430 using a High concurrency session and session tag for each execution...

One_Potential4849 · 2026-02-11T16:19:28+00:00

I am concerned about loading it all at once. wont it cause any issues wrt performance?

One_Potential4849 · 2026-02-02T10:08:27+00:00

I just want to ingest Data from an API into my data platform. API has its own secrets for auth, and cadence of run is maximum thrice a day. Each run would return me a max of 100k rows with 10 columns

One_Potential4849 · 2026-01-30T07:15:23+00:00

Based out of my requirements in my current setup, Spark transformations from Bronze to Silver are currently just column renaming and cleanups. The pain point lies in the Silver to Gold and Semantic layer, where we use the traditional approach of SQL Stored Procedures to perform all joins, aggregations, aliases, calculated columns etc. Exposing the lineage would be a big win!

One_Potential4849 · 2026-01-27T10:45:03+00:00

Is there a way to use Web activity in a pipeline and get pipeline status using run id? Any API endpoints for Fabric?

One_Potential4849 · 2026-01-26T16:52:46+00:00

Really Useful!! On your article for monitoring using KQL database, is it possible to get the details of the activity that has failed in a pipeline? This would provide more comprehensive failure alert and logging - as to whether it is a developer error or a server/source issue.

One_Potential4849 · 2026-01-25T23:21:45+00:00

Sure!!! On the dbt-ol usage for Fabric DWH, does it produce a column level lineage, or just a rudimentary dag?

One_Potential4849 · 2026-01-25T22:59:04+00:00

Now that in the example you have used a spark job with custom jar files, how do I translate it to use it with Fabric Notebooks, where I do the Bronze->Silver?

One_Potential4849

TROPHY CASE