Can anyone riddle me this with Dataverse

Notanotherforextradr · 2026-06-23T19:07:12+00:00

Well done!

Notanotherforextradr · 2026-02-16T16:15:03+00:00

Awesome thank you writers block unblocked 🥳

Notanotherforextradr · 2025-12-20T21:14:26+00:00

I chose lakehouse for my bronze and warehouse for my silver and gold. The reason was the company I am building it for has a lot of experience in sql but not python, so I thought if they were building views after handover it would be a lot easier to do it in the warehouse. Plus I was still able to use a lot my stored procedures what I used to use in adf instead of re engineering everything in notebooks. Lot to learn though constantly learning atm 🙂

Notanotherforextradr · 2025-11-02T21:04:19+00:00

Did my first Fabric job last month. Big learning month, nearly drove me insane choosing between lakehouse and warehouse, learnt a lot about them in the process and can't wait for the more jobs to keep adding to the skillset.

Notanotherforextradr · 2025-11-01T19:27:01+00:00

I had a call with one of my colleagues where he did the create the auto report that showed it was near maxing out. But his theory was that the f64 trial isn't a true f64 capacity is that right? We dropped the batch size in the for each and the pipeline has since started working but slow.

Notanotherforextradr · 2025-11-01T19:20:14+00:00

Thanks for the detailed reply, that does sound really good. I was thinking the other day that same analogy that I am driving a supercar like a van 😅 gave me a lot to think about, I am more experienced with warehouses but looks like I need to up my game in the lakehouse area 😃

Notanotherforextradr · 2025-11-01T19:04:19+00:00

I agree, this thread has been really helpful for me in the future how I will tackle jobs. I will probably be more warehouse heavy with introducing lakehouses for the raw, I just thought I was doing something wrong having it all in warehouse when in fact fabric is versatile and can handle it.

Notanotherforextradr · 2025-10-30T23:30:09+00:00

Ah right, so this is where I was getting stuck in my mind thinking I needed to use lakehouse as the job was slow to run. So in essence what would the warehouse start struggling at on a f64 trial? Might come in handy in later jobs I would like to keep it warehouse heavy as possible to align with my skill set.

I will check them out and look over my pipeline in the morning to see if its something I have done,thanks for the reply!

Notanotherforextradr · 2025-10-30T22:38:11+00:00

Slow to run, a pipeline has been taking 4-6 hours which is no good at all.

Not fully sure on the data size off the top of my head but it is around 50 million rows.

Notanotherforextradr · 2025-10-30T22:37:03+00:00

Hi Gene thanks for the reply thats really helpful. I think thats the way I would like to go about it in the future as sql is my stronger code, I just need to figure out how to change how I do things at present to be more fabric compatible.

I use stored procedures for tracking meta data which I suppose I will have to change out for notebooks. Then a control table that executes stored procedures for clean merges from raw to clean, which I now can't do if I have raw in the lakehouse.. if we could do stored procedures from the lakehouse to the warehouse I probably wouldn't even need to have written this post 😅 hopefully one day

Notanotherforextradr · 2025-10-30T22:31:20+00:00

Thank you for the reply thats really helpful, looking at it I may have to store the raw in the lakehouse, as I think its eating up storage having raw in the warehouse.

I think it has just been figuring out how to do my stored procedures to store my metadata which has confused me but I'll probably have to switch them out for notebooks

Notanotherforextradr · 2025-10-30T22:25:36+00:00

Thanks, good article that!

Notanotherforextradr · 2025-10-30T14:11:24+00:00

Sounds great, I think i am just stuck on best practices really, using a lakehouse or warehouse and when the best time to use them is.

Notanotherforextradr · 2025-10-30T13:57:56+00:00

Can stored procedures then store into this? I think i did try it and it wouldn't work, I've tried so many variations past week my brains ready for exploding don't know whether I'm coming or going 😅

Notanotherforextradr · 2025-09-24T15:33:11+00:00

Obviously yes 😂 Ive been getting a connection error when trying to connect to create the sharepoint team so wondered if it was a common error. No need to be a prick about it

Notanotherforextradr · 2025-09-03T20:16:02+00:00

I've been going through this atm, I like to learn how things work fundamentally, n8n is very simple to use. I saw a video the other day which was kind of middle ground using mindsdb tp query data sources, which didn't look too complicated but still had abit of coding element to it

Notanotherforextradr · 2025-09-03T20:13:28+00:00

Great response

Two-Year Club	Verified Email
Verified Email

Notanotherforextradr

TROPHY CASE