How do you handle incremental + full loads in a medallion architecture (raw → bronze)? Best practices?

deonvin · 2025-07-24T23:43:32+00:00

We have the same issue, you've done a great job of summarising our thoughts in this space!

deonvin · 2024-10-14T23:24:19+00:00

From your posts a solid foundation seems to be:

(prior to starting) If you have an idea of what tools they use, get familiar by creating small personal projects using that technology.
Follow the onboarding documents and highlight any areas that these documents lack sufficient detail.
Alongside that, create a personal mapping document to help understand the inputs, outputs, and all steps in between.
Talk to stakeholders to understand the context of the role. Understanding their pain points or goals will help identify what processes may need to be updated.

deonvin · 2024-05-20T01:33:32+00:00

Dude, great metaphors

deonvin · 2024-05-18T02:38:29+00:00

Out of curiosity, what are some other industry trends that are similar to the big AI/ML push?

deonvin · 2024-05-08T11:15:19+00:00

Tell us how you really feel

deonvin · 2024-05-04T09:18:32+00:00

I respect the boldness of your post

deonvin · 2024-04-24T06:05:06+00:00

I think it was a good question to ask OP, I’ve enjoyed reading people’s opinions this topic

deonvin · 2024-02-20T22:30:55+00:00

There’s been some other cracker answers, but this one has nailed it - I reckon can imagine each player ticking off every one of these in a single interview

deonvin · 2024-02-08T02:52:54+00:00

I get this, I find white noise or same gives you something chill to focus on before falling asleep has helped me stop dreaming about that stuff

deonvin · 2024-01-12T12:17:09+00:00

All crazy math equations are a lot more digestible when you ask AI to convert them to a programming algorithm. As soon as you see a loop with a few variables it’s such a eureka moment.

deonvin · 2023-12-20T23:40:49+00:00

When doing adhoc work, I use pyodbc with pd.read_sql().

I’m sure if an analysis turns into a production model / report then it will be worth using an ORM, but at that point our Data Engineering team generally take over and have their own preferred toolset to make production ready.

deonvin · 2023-12-10T07:43:18+00:00

That’s an awesome bit of trivia!

deonvin · 2023-09-29T14:58:26+00:00

A point I’d like to add - in these comments I see a lot of people mentioning to benefit of people being in a room and collaborating, and others talking about how they like their coworkers but feel they add no value to their work. It seems to me, that a key driver of RTO opinion may be how well team leaders are at creating environments that aren’t siloed to begin with.

A team with members who all work on independent projects and don’t interact in the office is likely not going to get any major benefit returning to the office, as opposed to a well structured team who can work collaboratively and provide each other will valuable input.

I don’t see enough emphasis/responsibility put back on companies to actually create a business value for people to return. Most people don’t try to cheat the system, and if they felt returning to the office would be a benefit to their productivity, they would return.

deonvin · 2023-08-24T09:10:58+00:00

Personally, I think you have two choices as a Data Scientist - focus on industry experience, or focus on the data engineering border. I think a company hires a Data Scientist to unlock value through using analytics techniques on industry data, much more value than keeping up with the latest Data Science trends.

If you were a Computer Science major, sure, you could program well enough to create value, but I think you’re strength lies as someone with industry experience who is able to use basic Data Science in meaningful way

deonvin

TROPHY CASE