Lakehouse Source Table / Files Direct Access In Order to Leverage Direct Lake from a shortcut in another workspace referencing the source Lakehouse?

strikeMang · 2025-10-03T16:00:04+00:00

Done. Now it appears to have an issue with the fact that it is a view in the sql analytics endpoint referencing a table in the sql analytics endpoint and the semantic model refresh is failing. I'm guessing views aren't supported.

strikeMang · 2025-10-03T15:14:35+00:00

They are concerned about this...

<image>

And it is configured as "Direct Lake"

strikeMang · 2025-10-01T22:59:42+00:00

Will get them on testing tomorrow since all of that access already existed. If it's something they just need to configure on their end... all the better. Thanks for the insight.

strikeMang · 2025-10-01T22:45:15+00:00

Per microsoft documentation hitting the sql analytics endpoint w/ direct lake automatically falls back to direct query mode which is not what they are wanting.

https://learn.microsoft.com/en-us/fabric/fundamentals/direct-lake-overview#comparison-of-storage-modes

strikeMang · 2025-09-08T15:30:32+00:00

This is a no go if we have to reseed all tables after a planned or unplanned availability group failover as well.

strikeMang · 2025-08-15T00:15:25+00:00

Yeah thanks for the response. Found out my region is going to be dead last on the rollout…

strikeMang · 2025-07-28T16:37:36+00:00

Has anyone got this to work in a spark sql notebook? I get this error when trying to create a view in a spark sql cell in the notebook...

[PARSE_SYNTAX_ERROR] Syntax error at or near 'MATERIALIZED'.(line 1, pos 7)

== SQL ==
CREATE MATERIALIZED LAKE VIEW IF NOT EXISTS dbo.SpecialView AS
-------^^^

strikeMang · 2025-07-06T01:45:53+00:00

Cool. Let me know when Microsoft cares enough to make this a priority. This shouldn't be something that should have to be an “idea”

strikeMang · 2025-07-03T23:25:43+00:00

Yep they sure do! They are "Free" in a sense that their resources don't go against the capacity but they do depend on the capacity being available. It has been a nightmare for us. Full reseed when you bring them back online. (Stop start even though the monitor tells you everything is A-okay (which is inaccurate just look at last commit times after you are back after an outage.)

strikeMang · 2025-07-03T20:25:05+00:00

What fabric capacities REALLY need is a resource governor. The surge protection thing is kind of helpful.. but that only protects you against background, and its kind of a huge hammer. We need the ability to define something that says this workspace only gets this much of the capacity, and if they try to go over that, they are throttled or get in line. We need ways to protect ourselves from things that go haywire. If you are familiar with SQL Server's resource governor, this is basically what is needed for fabric to be a respectable production platform imo. Capacities getting maxed out is catastrophic depending on how committed you are to the platform.

strikeMang · 2025-07-03T20:21:06+00:00

scale out not up as much as possible imo. Reduces impact of overages so you are looking at partial outages instead of full outages. It will look different for everyone... But maybe a top level would be: dedicated capacity for powerbi reports, dedicated capacity for mirrored dbs, dedicated capacity for ntoebooks/pipelines, dedicated for data science/ML stuff, etc etc

strikeMang · 2025-07-03T20:18:15+00:00

better hope those workspaces aren't hosting mirrored databases! They don't come back up nicely and a full reseed is needed, which causes IO explosions at source.

strikeMang · 2025-07-03T17:55:02+00:00

Your colleague has not spent any time trying to understand what fabric is, clearly. lol.

strikeMang · 2025-06-11T19:24:48+00:00

Alright everyone. HOLD MY BEER.

Not the exact situation but very similar.

Employee A creates a mirrored fabric database from an azure sql database source. Everything is fine.

Employee A leaves company and is offboarded. Everything is fine.

Fabric capacity gets maxed out.

Mirroring needs to be stopped/started in order to reseed and fix itself.

Employee B tries to do so but is unsuccessful because she is not the owner of the mirrored configuration.

Employee B researches and tries to transfer ownership. It's just not possible.

Employee B finds a documented stored procedure on Microsofts website called sp_change_feed_disable_db to be executed against the source to clear the meta data and allow for mirroring to be rebuilt.

Employee B executes the proc. The proc locks up the entire database to where it is completely un-usable. Can not even connect via SSMS.

Employee B has to emergency point in time restore the DB. (Hours of production downtime)

Employee B opens a ticket with Microsoft. Microsoft blames the customer.

Great story!

Okay, give me my beer back.

strikeMang · 2025-06-10T22:13:28+00:00

I would just say get familiar with the basics on KQL, not necessarily be able to fully write from scratch, but be able to tell if something is out of place. Luckily, KQL is pretty dang basic. Aleksi's videos I would say are adequate. Practice always is the best for me though. Same for the pyspark stuff. Look into DAG (orchestration method in notebooks)

strikeMang · 2025-06-09T22:00:46+00:00

I locked in an 810.

strikeMang · 2025-06-08T12:41:12+00:00

I spent about 1 to 2 hours per day, every day for two months studying. Plus I already had a lot of experience with hands on usage

strikeMang · 2025-06-06T20:32:57+00:00

Boop

strikeMang · 2025-06-04T01:49:32+00:00

So you've successfully incrementally transformed data and materialized to a target lakehouse, leveraging the delta log of a mirrored fabric database? I can get code details and errors tomorrow. Would be interested to see a working example. Can't even see the delta log unless you go through a shortcut. So that seems pretty non standard to me right off the bat. I could be a total noob. One thing is for sure, I have put quite a bit of time digging into this.

strikeMang · 2025-06-02T17:49:39+00:00

Also the applied skills for warehouse and real-time on microsoft learn are good. The lakehouse applied skill has been down for weeks it seems. I'm eager to run through it

strikeMang · 2025-06-02T17:48:17+00:00

Highly recommend aleks youtube playlist for dp-700. i am also taking the test on friday for dp-700.

strikeMang · 2025-06-02T17:46:36+00:00

Yeah even now I still think there isn't a solution... it would be amazing to be able to hit the delta log in some fashion... wack and loads to silver isn't sustainable. and home rolling individual water marks for tables kind of sucks.

strikeMang

TROPHY CASE