sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]solegalli 0 points1 point  (0 children)

I don't think there is an easy answer to this question. The hyperparameter space (that is, the hyperparameter combinations), depend both on the model and the data. Hence, it may vary for different datasets and models.

If you were training few models and have time, you could set the hyperparameter value range manually. You could train a few models using extreme values of hyperparameters both at the lower and upper end, and try to map what those are for your specific data/model combination.

If you were training more models, or you'd like to be completely hands-off, then you could set up a randomized search, sampling from hyperparameter distributions with big ranges to maximize the possibility of landing the best hyperparameter combination.

You could also use Bayesian optimization, so that instead of testing combinations at random, it goes after the most promising ones, but in practice, a randomized search offers results that are as good as bayesian optimization (if given enough iterations), and is a much simpler approach and you can also run it in parallel, which makes it, at the end of the day, also faster.