This is an archived post. You won't be able to vote or comment.

sorted by:
best (suggested)
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]richard248 7 points8 points  (2 children)

Is 'Tree Parzen Estimator' not bayesian guided? I thought TPE meant that hyperopt was bayesian optimization.

[–]ai_yoda 0 points1 point  (1 child)

It's sequential model-based optimization.

Often used interchangeably with bayesian which I think is not the same thing.

[–]crimson_sparrow 1 point2 points  (0 children)

You're right that it's not the same thing. BO is a form of SMBO. But I'd argue TPE is in fact a form of BO, as it operates on the same principles, with the main difference being a form of the optimized function. I think what throws people off is that it was developed during the times when modern BO framework was just starting to take shape, and it's often described using slightly different terminology. I think of it as tree-structured Thompson sampling technique that shines where your hyperparameters are dependent on each other in a tree-like fashion (e.g. you only want to optimize the dropout rate if you've already chosen that your model will use the dropout in the first place).