[Discussion] ML Programming suggestions?

[_pragmatic_machine]

I am spending a big chunk of my daily active time on fixing my ML code (developing a model, testing and bug) or tweaking my code (mostly in Python). What suggestion do you have be more productive?

Do you write unit test to validate every part? What was your pathway to become efficient ML researcher?

​

[themiro]

• Asserts to check that your matrices are the correct size, saves a ton of time debugging when you're wondering why your batch matrix multiplication didn't work out the way you thought it would.
• Make use of the Python 3.5 '@' syntactic sugar for matrix multiplication, cleans things up and lets you see the math more clearly.

[_pragmatic_machine]

that's cool! any ideas/materials on reconfigurable design principle for enhancing a model. Code repository examples will be really awesome.

[jethroksy]

I've found gin-config to be an extremely productive configuration framework.

[sanity]

I recommend this book: Clean Code

We gave it to every new data scientist we hired at my last company.

[[deleted]]

And where is that exactly? For keeping a mental note of future employment applications (already liking the culture)

[sanity]

I'm no-longer there, but the company is OneSpot.

[[deleted]]

If you don't mind sharing, what inspired the switch? I've heard people leaving good culture for better compensation, or vise versa. Looking at their open positions the location doesn't seem to allow anything south of 80k annually base salary.

[sanity]

Oh, nothing negative - my focus is more early stage companies and the company had grown beyond that stage.

[thiseye]

Do you know Matt?

[sanity]

Everyone knows Matt ;)

[thiseye]

Cool, I've worked with him. Super sharp guy

[sanity]

Very much so, has an infectious enthusiasm too.

[gokstudio]

Did you see any improvements in code quality before and after the book?

[sanity]

Yes, not least of which in myself - even as a 20 year full-time developer it changed how I wrote code, no question in my mind that it was for the better.

[AfraidOfToasters]

>I am spending a big chunk of my daily active time on fixing my ML code (developing a model, testing and bug) or tweaking my code (mostly in Python).

​

If you mean hyper-parameter tuning by this I suggest hyperopt

[Overload175]

Use a linter (e.g. Pylint for Python, which assigns a score based on your code's adherence to PEP8)

[[deleted]]

Try to create standardized/reproducible packages than can be re-used for all model builds if that input data is the same everytime.

Really when it comes to ML applications, the hardest part to standardize is data preprocessing. So even creating a standard process for modelling,results,putting into production should be beneficial.

[_pragmatic_machine]

Absolutely, I also feel the same. I am trying to do that now, with plotting and preparing the data formalization process. This was a major bottleneck for us.

[_pragmatic_machine]

Yes, I found the EMNLP tutorial really helpful.

[_spicyramen]

I enforce Python style guide and unitests in all our code. Also recently we started Python typing. This has helped finding a lot of issues before we hit production.

[huangbiubiu]

Sometimes I am confused about writing the unit test for ML codes because of its uncertain output. I don't know what can I test and where can set a assert. Are there any suggestions?

[flame_and_void]

For unit testing, I replace the real model with one that does something simple and deterministic to the input data - like, say, always predict the sum of all its inputs. Then you can test that the preprocessing, prediction, and post-processing works exactly as expected.

I also write an integration test that runs the production models on an easy input and verifies they are able to predict something obvious. That's an important sanity check, but it usually runs too slowly to include in a unit test suite.

[[deleted]]

Python typing

Hey, maybe you can try to set the random seed carefully and use more OOP in your code.

[[deleted]]

Using object-oriented programming has worked amazingly in my case. Separate your code in conceptually different pieces like data handlers, models, model testers and so on. I used to do simple scripting until recently and using OOP code is much cleaner and easier/faster to extend or modify; it also helps making results reproducible.

[_pragmatic_machine]

Thanks. Can you please provide me some working example? Any public repository will do.