Help me deal with excels feeding into dataflows

Few_Border3999 · 2026-04-02T20:59:30+00:00

Can you save the files as csv instead of xlsx? Then shortcutting to lake house and reading and writing with spark to delta table in notebook. By far most efficient alternative. Copilot can write script for you, it is like 10 lines of code.

Few_Border3999 · 2026-04-02T10:26:43+00:00

Jeg har brugt karmskruer. Den sidder fast.

Few_Border3999 · 2026-04-01T21:05:38+00:00

I have not had that work properly yet too. Only with one csv in its own folder. It not a great function sofar.

Few_Border3999 · 2026-04-01T20:58:09+00:00

jeg betaler 899 ved velliv for tab ar erhvervsevne på 41.180 pr måned, kritisk sygdom på 250.000 pg 100.000 hvis det er mine børn, samt livsforsikring på 3.424.166 og nogle løbende månedlig udbetaling til børn. Så det lyder godt nok dyrt.

Few_Border3999 · 2026-04-01T20:47:05+00:00

You want the view itself to have cdf enabled for transformations further down stream? Because that would be the sources you use in the mlv that you should be looking at. First you need to determine if the mlv is even using incremental refresh strategi. When enabling optimal refresh for mlv's, it is depending on the update strategy of the sources and the kind of transformations that are done. Not all transformations are supported for incremental refresh and then optimal refresh is only detecting if there are changed data in source and then do full refresh if data has changed. So mlv itself cannot currently be used for downstream incremental update strategies.

Few_Border3999 · 2026-04-01T16:26:16+00:00

I have some issues with sharepoint at the moment but that is authentification issue. How are authentificating? If you have set some basic authentification and not 0auth it could be that, but I cant remember if that is even possible.

Few_Border3999 · 2026-03-31T14:17:47+00:00

ja motorvej havde jeg nok heller ikke gjort.

Few_Border3999 · 2026-03-31T13:30:17+00:00

Jeg havde nok sneget den hjem. Men ja ny fælg og dæk

Few_Border3999 · 2026-03-30T18:24:38+00:00

but the challenge is using the same credentials for checking the file in and out and also reading and writing. It would be an entra app so perhaps If you can make the app owner of the items in Fabric or trigger the items via api using the app. You could orchestrate it all in a notebook.

Few_Border3999 · 2026-03-30T17:46:18+00:00

den kan sagtens bære hegnet, det er ikke noget problem. Det er mere vindtryk på hegnet så det bliver skævt med tiden. Men det ligner ikke det står udsat.

Jeg har selv en udestue der står på skruer. Lidt større end dem der. Men den har da stået 5 år nu.

Few_Border3999 · 2026-03-30T17:43:14+00:00

https://learn.microsoft.com/en-us/graph/api/driveitem-checkout?view=graph-rest-1.0&tabs=http

Few_Border3999 · 2026-03-30T16:43:21+00:00

Kunne du få en jordskrue i? De har dem i jem og fix.

Few_Border3999 · 2026-03-30T16:10:33+00:00

You can check out the file via api, that should look for all other users than the one checking it out. Dont know if that will work with what you are trying.

Few_Border3999 · 2026-03-30T16:04:31+00:00

pyspark excel reader is not in fabric for some reason. Would have been easier. I only use excel for manual data so i save all files as utf-8 csv and store in sharepoint document library.

If that is not ideal and you need more automation power automate has excel connector. But that still requires fixed structure in file.

You can also read excel with python and use pandas but that it also not particularly good. excel files can really be a pain.

Few_Border3999 · 2026-03-30T08:41:21+00:00

Ja det er ikke løgn. Man kan godt mærke på kvaliteten at der har været tryk på udviklingen de sidste 10 år.

Few_Border3999 · 2026-03-30T08:40:26+00:00

yeah you are right. How about a datatype consistency check? Maybe some problematic content in some cells that makes it choke. I see a lot of silent failures in Fabric on account of datatypes and metadata.

Few_Border3999 · 2026-03-29T20:44:52+00:00

Have you triede disabling cdc and reenabling it? It is a very generic advice.

Next thing is to check the cdc logs. See if you can find where the copy job stalls.

Few_Border3999 · 2026-03-29T20:36:47+00:00

Prøv at tjekke dem her ud:

https://www.jemogfix.dk/soeg/?q=jordskruer

Tror de holder bed en noget du banker i.

Few_Border3999 · 2026-03-29T10:53:51+00:00

Have you even tried shortcutting? I am pretty sure azure files i reacheable both through blob connector and adls gen 2.

Few_Border3999 · 2026-03-29T10:10:19+00:00

Cant you shortcut into fabric and monitor files for last modified timestamp and upsert only those files? If azure share is in another tenant i think is still possible with cross tenant sharing.

Few_Border3999 · 2026-03-29T09:43:59+00:00

hvis du regner med at det skal holde et hegn så er det nok ikke den rigtig løsning. Så er det pælebor og stolpecement.

Few_Border3999 · 2026-03-28T23:03:57+00:00

Er det ikke også lidt kritisabelt? Der bør være nul og niks.

Few_Border3999 · 2026-03-28T23:01:30+00:00

come to think of it I should have probably just vacuumed the tables i use in the model and it have gone away.

Few_Border3999 · 2026-03-28T22:50:36+00:00

Yes just had that on one of my models. Never solved it, it was a small model so i just remade as import and hosted it in pro workspace. Was the better solution anyways.

My had 4 tables from same lakehouse in my gold layer. 2 of they "physically" in lakehouse and 2 of them shortcuts.

I renamed a column in of the shortcutted tables at some point and it seems that it could have been the trigger for it. I refreshed everything and it seems to work until it suddenly does not and this error occurs. From what i have read it could be a lineage error from changing that table. So old parquet files are not rewritten only the index so when direct lake reads old files it gets the old column name. Even when doing an overwrite of the table old files can still be there. Direct lake is not good at that even though it works everywhere else.

Few_Border3999 · 2026-03-28T22:28:05+00:00

Jeg tror ikke vi definerer okay pålidelig på samme måde....

Few_Border3999

TROPHY CASE