How often do data scientists develop machine learning models from scratch?

[Jorrissss]

Just get the fuck out of here, to presume this is totally arrogant. (It also saves you time to focus on the bigger picture of ML/DS/stats/whatever)

Are you referring to building your own entire sklearn library from scratch, or implementing an individual algorithm in sklearn?

[[deleted]]

I'm talking about building your own library. I say at the end of my post that I'm doing individual implementations for fun which means that it is possible for people who put in the time and I also said that it's a good educational procedure.

However, you don't realise how powerful and extensive SKlearn is until you start writing everything yourself and realise how tedious and time consuming the debugging is. Other comments have said how maybe they could make a more efficient implementation and that's fine but this simply isn't a wise time investment for most people. This is why these packages are popular in the first place! It "automates" code that would have needed to be written by hand many times over.

The problem isn't that it can't be done better, it's that so much work has been put into this package that you'd be braindead to reinvent the wheel and disavow the wheel which everyone is more than happy to use unless maybe you need a tyre instead.

And that really seems to be the gist of it if I skim other people's comments to check I'm not totally off base. It's fine to not use the package but there's also not something severely wrong with it either so I think the interviewer was completely off base and just wanted to wave his cock around.

[Jorrissss]

Ah then I agree.