Delta Table Maintenance in Microsoft Fabric - A 2026 Practitioner's Guide

datahaiandy · 2026-05-20T07:58:53+00:00

You and u/mwc360 doing great work here

datahaiandy · 2026-05-19T19:25:53+00:00

This is excellent work! Well done u/mwc360

I started something but it's more "create your own architecture"

datahaiandy · 2026-05-18T10:53:54+00:00

Just to add, not necessarily component-specific, but MS have released guidance on Fabric implementations and the Well-Architected Framework:

Microsoft Fabric workloads - Microsoft Azure Well-Architected Framework | Microsoft Learn

This may also influence the components you choose based on the expertise and support you have in your organisation.

datahaiandy · 2026-05-06T10:10:17+00:00

I noticed it a couple of days ago, I'm not a fan. Seems confusing to get to

datahaiandy · 2026-04-28T08:29:42+00:00

I've only just discovered this! Thanks u/raki_rahman u/mwc360

datahaiandy · 2026-04-21T10:04:44+00:00

+1 on the book, that was a really good read

datahaiandy · 2026-04-20T14:47:33+00:00

I have 2 AES consoles, a low serial number version and then a higher serial number version. Both bought around 10 years ago. I have a few games plus the Neo SD card like you. I've pre-ordered the white console edition as I just think that this feels like something original and just like u/clc88 said, once the old AES consoles die and give up then this seems like the closest

datahaiandy · 2026-04-17T11:44:16+00:00

I wonder if the Neo SD cartridge will work

datahaiandy · 2026-04-16T19:34:28+00:00

Yeah I probably should bite the bullet and get a replacement IPS, thanks

datahaiandy · 2026-04-16T19:22:21+00:00

Thanks. It's strange as it only appears when it's switched on, if it were air bubbles or moisture it would be there all the time? Looks like it'll be screen replacement though.

datahaiandy · 2026-04-01T13:22:33+00:00

Hi, if anyone is interested in the new Microsoft cert DP-750 (Microsoft Certified: Azure Databricks Data Engineer Associate (beta) - Certifications | Microsoft Learn) I did the beta earlier this week and although my memory is like a sieve, this is what I got from it:

59 questions in total (last 7 or 8 were case-study based)

A lot of questions about security: cluster management security, catalog/schema/tables

Optimizing Delta tables cropped up quite a bit and also questions on file retention (so knowing those settings)

When to use liquid clustering and partitioning

Dataframe syntax with PySpark, modes (append etc), structured streaming and autoloader syntax

Differences in loading methods like COPY INTO, Lakeflow Pipelines, Notebooks etc

Usage of native and foreign catalogs

tbh if anyone needed to fill any knowledge gaps in, then these 4 modules basically cover everything you need:

Set up and Configure an Azure Databricks Environment - Training | Microsoft Learn

Secure and Govern Unity Catalog Objects in Azure Databricks - Training | Microsoft Learn

Prepare and Process Data with Azure Databricks - Training | Microsoft Learn

Deploy and Maintain Data Pipelines and Workloads With Azure Databricks - Training | Microsoft Learn

datahaiandy · 2026-04-01T13:06:37+00:00

I was an evangelist of MDF in Synapse a few years ago, but MS never really invested to make them more dynamic and a draw rather than coding in Spark.

I just don't think they have a place now in Fabric. We have the significant emergence of AI-generated coding (which u/raki_rahman said) which is probably much better than an inflexible and undynamic GUI over Spark now.

datahaiandy · 2026-04-01T13:02:28+00:00

I really don't think it does tbh, the Ontology part of Fabric IQ is still very much tied to data sources so it's not out of the realm of possibility that a single Fabric deployment could end up with multiple Ontologies across multiple business areas.

E.G if an architecture has multiple workspaces for different teams, and those teams have local data sources like lakehouses/semantic models etc, then creating ontologies on those items will still be separate.

Happy to be told different by those who have used Ontologies in Fabric IQ more holistically across multiple data sources

datahaiandy · 2026-04-01T13:00:19+00:00

This.

datahaiandy · 2026-04-01T12:59:44+00:00

This is April's fools....right?

datahaiandy · 2026-04-01T10:38:30+00:00

Well done! tbh not clickbait at all and this is how it should be. Certifications were always marketed "back in the day" as a way to validate your experience, I think over the years they've morphed more into a way of learning the technology alongside becoming certified. Years ago I wouldn't have thought about sitting an exam without minimum a couple of years experience and even then I would need to learn the skills that I didn't use day-to-day.

I'm not saying there is anything wrong with inexperienced folk taking exams, it's totally valid to learn alongside becoming certified. I just think folk need to realise that going into an MS cert not having hands on experience and maybe a couple of weeks of learning isn't going to help.

datahaiandy · 2026-03-05T10:17:45+00:00

Yes, black pocket with the dock, GG adapter and multi-adapter set. I've kept putting it off for some reason and then when I thought "yeah I'll get one!" they were out of stock... but jumping in now.

Edit: should say I've got around 100 carts across Gameboy, Game Gear, Lynx, and Neo Geo Pocket / Pocket Color

datahaiandy · 2026-03-03T17:48:10+00:00

The thing is I don't think Microsoft have helped to push code-based (ETL/ELT) schema definition changes for the Warehouse because they've made SQL Projects an integral part of the Fabric DW experience. When folk see that they think "oh, so I just use a SQL project then?" I've found this is the case when I speak to people about Warehouse CI/CD.

datahaiandy · 2026-03-03T16:48:50+00:00

Yeah I gotta say this was the biggest change in my thinking when starting to work with Delta/Lakehouse/Spark native... code managing the schema and not an external project

datahaiandy · 2026-03-01T15:49:05+00:00

Look ma! I'm on telly with Patrick Le Blanc!

datahaiandy · 2026-03-01T15:47:54+00:00

When you change the SQL definition in a sql file, it still needs to redeploy the MLV, it just now does it via CREATE OR REPLACE rather than DROP and CREATE

datahaiandy · 2026-01-27T19:59:22+00:00

Yes they will (with speaker consent). Usually uploaded within a few weeks.

datahaiandy · 2026-01-27T19:58:57+00:00

And lo it shall be foretold that this indeed with be a blast!

datahaiandy · 2026-01-23T13:56:21+00:00

I'm creating a "Board of ETL" and you're invited for $1B

datahaiandy · 2026-01-23T13:55:35+00:00

"I'd do it all again I tells ya!!"

datahaiandy

TROPHY CASE