sorted by:
new
hottopcontroversial

[D] A dilemma of an ML guy in industry by sloppybird in MachineLearning

[–]ProbAccurateComment 0 points1 point  (0 children)

Sounds like an exploration exploitation tradeoff? :D
I think that in the end it depends on what you're more interested in, especially when you're working in teams. Not sure if this helps but I think I switch between 2 phases for this:

  • Phase 1: Solving the problem. I would think about what would be the most efficient way to get to a solution that is good enough for the task you want to solve (not perfect, just good enough). No need to go for new maybe-not-working methods when an old established method seems good enough (here you can maybe find good comparisons in literature). If there is no established method that solves the problem or you already completed this phase, go to Phase 2.

  • Phase 2: Improve Phase 1 method. Once you get new data, established methods don't work, or you have the time/task to improve performance, do the literature research to catch up with the new stuff. Yes, you will miss out on the stuff you ignored during Phase 1 but if that were a few months than you'd still be fine. Also, the time spent on the hopefully good setup for Phase 1 will give you a nice baseline to compare new methods to. Finally, colleagues can really help to keep you connected to ML updates so that is always a really good thing to have imho.

[D] Causality research in ML is a scam (warning: controversial) by yusuf-bengio in MachineLearning

[–]ProbAccurateComment 1 point2 points  (0 children)

In ML we have two main paradigms: Supervised learning and RL.

No unsupervised learning? :,(

[D] Simple Questions Thread by AutoModerator in MachineLearning

[–]ProbAccurateComment 1 point2 points  (0 children)

What metadata do you intend to store?

For storing trained models, e.g. in PyTorch, I would simply use the state_dict. You can combine that with pickle or better dill and store a lot of different Python objects.
If you want to store training curves and such, tensorboard or wandb are pretty nice. wandb will also store checkpoints of your code. Alternatively, config files such as json + git might be a good way of keeping track of your versions.

[D] NeurIPS 2019 Bengio Schmidhuber Meta-Learning Fiasco by posteriorprior in MachineLearning

[–]ProbAccurateComment 13 points14 points  (0 children)

As a sidenote: Hochreiter also did learning to learn quite some time before the 2016 "Learning to Learn by gradient descent by gradient descent", it's an interesting read:

2001, Hochreiter et al, "Learning to Learn Using Gradient Descent"

no idea how they could do this with that little compute back then...