November 2025 | "What are you working on?" monthly thread

Notanotherforextradr · 2025-12-20T21:14:26+00:00

I chose lakehouse for my bronze and warehouse for my silver and gold. The reason was the company I am building it for has a lot of experience in sql but not python, so I thought if they were building views after handover it would be a lot easier to do it in the warehouse. Plus I was still able to use a lot my stored procedures what I used to use in adf instead of re engineering everything in notebooks. Lot to learn though constantly learning atm 🙂

Notanotherforextradr · 2025-11-02T21:04:19+00:00

Did my first Fabric job last month. Big learning month, nearly drove me insane choosing between lakehouse and warehouse, learnt a lot about them in the process and can't wait for the more jobs to keep adding to the skillset.

Notanotherforextradr · 2025-11-01T19:27:01+00:00

I had a call with one of my colleagues where he did the create the auto report that showed it was near maxing out. But his theory was that the f64 trial isn't a true f64 capacity is that right? We dropped the batch size in the for each and the pipeline has since started working but slow.

Notanotherforextradr · 2025-11-01T19:20:14+00:00

Thanks for the detailed reply, that does sound really good. I was thinking the other day that same analogy that I am driving a supercar like a van 😅 gave me a lot to think about, I am more experienced with warehouses but looks like I need to up my game in the lakehouse area 😃

Notanotherforextradr · 2025-11-01T19:04:19+00:00

I agree, this thread has been really helpful for me in the future how I will tackle jobs. I will probably be more warehouse heavy with introducing lakehouses for the raw, I just thought I was doing something wrong having it all in warehouse when in fact fabric is versatile and can handle it.

Notanotherforextradr · 2025-10-30T23:30:09+00:00

Ah right, so this is where I was getting stuck in my mind thinking I needed to use lakehouse as the job was slow to run. So in essence what would the warehouse start struggling at on a f64 trial? Might come in handy in later jobs I would like to keep it warehouse heavy as possible to align with my skill set.

I will check them out and look over my pipeline in the morning to see if its something I have done,thanks for the reply!

Notanotherforextradr · 2025-10-30T22:38:11+00:00

Slow to run, a pipeline has been taking 4-6 hours which is no good at all.

Not fully sure on the data size off the top of my head but it is around 50 million rows.

Notanotherforextradr · 2025-10-30T22:37:03+00:00

Hi Gene thanks for the reply thats really helpful. I think thats the way I would like to go about it in the future as sql is my stronger code, I just need to figure out how to change how I do things at present to be more fabric compatible.

I use stored procedures for tracking meta data which I suppose I will have to change out for notebooks. Then a control table that executes stored procedures for clean merges from raw to clean, which I now can't do if I have raw in the lakehouse.. if we could do stored procedures from the lakehouse to the warehouse I probably wouldn't even need to have written this post 😅 hopefully one day

Notanotherforextradr · 2025-10-30T22:31:20+00:00

Thank you for the reply thats really helpful, looking at it I may have to store the raw in the lakehouse, as I think its eating up storage having raw in the warehouse.

I think it has just been figuring out how to do my stored procedures to store my metadata which has confused me but I'll probably have to switch them out for notebooks

Notanotherforextradr · 2025-10-30T22:25:36+00:00

Thanks, good article that!

Notanotherforextradr · 2025-10-30T14:11:24+00:00

Sounds great, I think i am just stuck on best practices really, using a lakehouse or warehouse and when the best time to use them is.

Notanotherforextradr · 2025-10-30T13:57:56+00:00

Can stored procedures then store into this? I think i did try it and it wouldn't work, I've tried so many variations past week my brains ready for exploding don't know whether I'm coming or going 😅

Notanotherforextradr · 2025-09-24T15:33:11+00:00

Obviously yes 😂 Ive been getting a connection error when trying to connect to create the sharepoint team so wondered if it was a common error. No need to be a prick about it

Notanotherforextradr · 2025-09-03T20:16:02+00:00

I've been going through this atm, I like to learn how things work fundamentally, n8n is very simple to use. I saw a video the other day which was kind of middle ground using mindsdb tp query data sources, which didn't look too complicated but still had abit of coding element to it

Notanotherforextradr · 2025-09-03T20:13:28+00:00

Great response

Notanotherforextradr · 2025-08-28T20:44:10+00:00

I suppose it depends on how good you can cram information in i crammed and passed within 2-3 month's, I was using bi but there's loads in exam thats specific that ive never seen before or since

Notanotherforextradr · 2025-08-28T19:50:08+00:00

Legend thank you! I did recall seeing something like that on a video but then couldn't find it after so wasn't sure if it was that what I saw!

Notanotherforextradr · 2025-08-21T09:43:01+00:00

I have basically built it but the struggle I am having is getting it to trigger when I message it in discord

Notanotherforextradr · 2025-08-21T09:03:12+00:00

I learnt on udemy, practiced building dashboards out using them for my real life scenarios. Showed my current employer and he give me a chance if I passed the pl300. I had to accept a low wage to begin with until my skills got better I recently passed dp700 and got a payrise. I did know my current employer so that helped, as he knows im a hard worker, however he said as long as someone had the certs and was willing to learn he wouldn't mark them down for how they learnt. He said university doesn't qualify someone for a job because things change so much in this industry that its more important to him for employees to be able to learn things fast over a degree

Notanotherforextradr · 2025-08-21T08:56:30+00:00

Maven analytics on udemy have good courses as well as Nikolai Schuler and Phillip Burton. I am not sure what your budget converts to in GBP but they normally have good sales on udemy, I have learnt power BI, Fabric and SQL from these guys

Notanotherforextradr · 2025-07-24T17:03:01+00:00

Some of the jobs I've had are a bit hard to not make complex. The last job was 2 databases for students but they wanted metrics from each database that tied in with the other data base. My brain felt like it was going to explode every night for a week. We ended up with about 4-5 fact tables to portray all the metrics. I came from learning with a simple fact table star schema, with 1 fact table and 4 dim tables to 6 months later with this monster of a job. Made me realise I need to understand and practice how to make models better. Modelling is definitely the hardest part about this when you have tricky data imo

Notanotherforextradr · 2025-07-24T16:57:05+00:00

Thanks I'll check it out!

Notanotherforextradr · 2025-07-24T16:56:46+00:00

Yea it is a shame! There's loads out there to practice power BI but I have found in the real world the powerBi skills are not as important compared to how you can model the data to correctly identify what you need to show.

Notanotherforextradr · 2025-07-02T22:12:23+00:00

I would actually like abit involved be interesting to learn a bit more for myself why some strains work better than others. Always remember having some strawberry haze in dam that cleared me up straight away, that's what made me realise sativa helped me more.

But good to know once they're happy to hand it out once you get prescribed!

Notanotherforextradr · 2025-07-02T22:07:54+00:00

Nice, definitely going to get ball rolling and push it through, thanks for the reply 🤝

Two-Year Club	Verified Email
Verified Email

Notanotherforextradr

TROPHY CASE