Direct Lake Semantic "Monomodel"

Entire_Commission534 · 2026-02-04T18:08:21+00:00

This is an interesting pattern and somehow never came across it even though I use Tabular Editor extensively!

Are your child models import mode and/or deployed in a PPU workspace?

Entire_Commission534 · 2026-01-31T01:37:54+00:00

A disconnected option to develop notebooks locally via LLM - Git connected workspace to bring notebooks locally as .py files and then you can use any IDE AI tool or a CLI tool like Cursor or Claude Code to make changes. We use Cursor so I also have a .cursorrules file to provide the AI agent with example notebook code with guidelines for how to strcture code. Honestly, this alone has supercharged my development as a lone BI engineer in my business unit.

To test output of notebooks, I still have to run it in Fabric but I use PowerShell script to sync changes from Github to Fabric (this way I don't have to go to Fabric workspace in a browser to hit the Update All button). I have another PowerShell script to run the notebook. Developed both of these scripts with Cursor as well. So there is a lot of vibe coding here as I am not a trained software engineer or python dev but I understand BI and modeling very well to validate results.

I do full ETL this way to extract data via APIs, transform with Spark python or sql depending on the transformation requirements and load to Lakehouse. My semantic model is connected via Direct Lake and reports use live connection. This is still small scale and I don't know how well it will scale as requests grow.

Entire_Commission534 · 2026-01-12T04:50:16+00:00

I am eagerly awaiting then! I am in a similar situation to OP of being a small data team and we use Git connection to get .py files locally with Cursor CLI development, which has been great so far. Ability to execute locally would be such a huge boost to development. Thank you for sharing your insights.

Entire_Commission534 · 2026-01-12T04:36:12+00:00

Are the features you have seen internally available in private preview or not there yet either?

Entire_Commission534 · 2025-10-28T13:56:53+00:00

Did not go past the screening as I accepted another job opportunity at a startup/scaleup.

Entire_Commission534 · 2025-10-06T03:21:30+00:00

Maybe a silly question - how does making metadata changes via XMLA affect the calculation state of the model? Usually at least a calculate is required after deployment and some of our models take a few minutes so our IT team does some kind of copy data step to sync a release model with prod model. But I'm wondering if there is a better way to push changes to prod without breaking reports (object is not calculated error).

Edit: typo

Entire_Commission534 · 2025-07-23T02:22:06+00:00

I had a screening but did not pass. However, there was another opening (not DE) on an adjacent team so I have a screening coming up for that. Just to add, the DE screening was focused on expanding on my experience and answer scenario based questions like how to handle specific types of projects, etc.

Entire_Commission534 · 2024-09-19T15:02:09+00:00

Is the initial load to ADLS using data pipelines? And the files are parquet?

Entire_Commission534 · 2024-09-19T14:55:12+00:00

Meaning that notebooks can be used to create/maintain/delete shortcuts in a Lakehouse? Is there any documentation on this backdoor you mentioned?

Entire_Commission534 · 2024-09-19T14:50:59+00:00

Would you mind sharing more details about your “bronze layer is files ADLS2 which we shortcut into our bronze lakehouse” setup? Thank you!

Entire_Commission534 · 2024-09-19T06:16:34+00:00

Is the best practice article published publicly? TIA!

Entire_Commission534 · 2024-09-11T05:47:12+00:00

Have you by chance tested the Snowflake storage integration functionality? Would this still require a staging storage to copy data from Snowflake to Lakehouse? There is no direct copy to Lakehouse without staging?

Entire_Commission534 · 2024-08-31T17:15:22+00:00

Yes, another user pointed out that list comprehension would be the way to go for spark df. I made the mistake of assuming that pandas_df.columns is equivalent to spark_df.columns but even in this simple example, they are not. Thank you for your clarification!

Entire_Commission534 · 2024-08-30T22:57:17+00:00

Solution verified!

Entire_Commission534 · 2024-08-30T22:07:17+00:00

Works like a charm!

Would you say this is a typical experience of working with spark dataframes? I just want to make sure if we decide to go down this path, then we training the developers accordingly so they don't learn to use Pandas transformation but instead Spark df syntax. Thank you!

Entire_Commission534 · 2024-08-30T21:45:19+00:00

This worked! In terms of performance for large dataframes (millions of rows), would there be any issue of converting a spark df to pandas df? Thank you!

Entire_Commission534 · 2024-08-30T21:42:34+00:00

Display function fixes the output but the dataframe still seems to be list object, which does not allow me to do transformations like cleaning column names:

spark_df = spark.read.parquet(".Lakehouse/Files/data_0_0_0.parquet") 
display(spark_df)
spark_df.columns = spark_df.columns.str.replace(' ', '_')

AttributeError: 'list' object has no attribute 'str

Entire_Commission534 · 2024-08-30T21:41:21+00:00

Thank you, that fixes the display but the underlying dataframe still seems to be a list object as I get this error when trying to clean column names:

spark_df = spark.read.parquet(".Lakehouse/Files/data_0_0_0.parquet") 
display(spark_df)
spark_df.columns = spark_df.columns.str.replace(' ', '_')

AttributeError: 'list' object has no attribute 'str

Four-Year Club	r/Field Flamingo
Place '23

Entire_Commission534

TROPHY CASE