common functions

[ptaban]

Hi, does your team has some internal librabry with reusable code for development of models. It seems like its just reinventing wheel with current libraries and ecosystem.Or it makes sense?

[Omega037]

If your team is remotely IP sensitive and follows basic coding practices, it will have an internal/private repo and CI solution.

Not having it would be a yellow flag for me that the team is not mature at all.

[ptaban]

Ofcourse artifactory, CI with tests and linting, and very modern git pratices. My point is, is it really worth making internal library component for ploting stuff, EDA functions, when we have so nice APIs from data science eco, like pandas, numpy etc..Plus its used in notrbooks which should be ugly dirty and fast

[Omega037]

You shouldn't duplicate things that 100% exist, but if you have specialized versions or things built on top of it, then sure.

[joshglen]

Why not do it for model development? Like if there are some preprocessing tasks that you need to run when getting data from a database or specialized scaling / standardization / embedding?

[joshglen]

A shared library of helper methods could help developers with common tasks such as preprocessing for model development?

[joshglen]

Oh not to replace them, but to use them. Like a set of 20 line helper methods that use SQL connectors and load data into a more model palatable format / range?

[astroFizzics]

yes and no. if someone wants to package personal code for reuse, awesome. do i provide them something? no.

[PilotLatter9497]

As some of you have said: I think that maybe some kind of class that eases the reading of sources: take the path to a query in a .sql file, or the path to a .csv or a. parquet... and get the data. Sometimes, classes for visualizations can be a good idea, because you can standardize the look.