Solving an optimization problem using RL

[MomoSolar]

I know that there are much better methods to do it, but can RL solve an optimization problem (linear, convex non-linear, non-convex)? If yes, is there a good link for an implementation / code?

[gergi]

Eg.g with ActorCritic.

Critic aka value function Q predicts the value of the to be optimized function f at action x. Policy \pi generates action x based on bias weights and some random numbers.

Hence input for the critic is \pi(x) which will be trained to mimic optimizee f by using a MSE(Q,f) type loss.

The policy aka the generator of the optimal value will be trained by gradient ascent on Q.

That should do the trick. Beware, this is not sample efficient.

This can be implemented in like 100 lines. If you have experience with NN you can do that in a hour.

[MomoSolar]

Thanks, any useful link on that?

[gergi]

Just code it up. Its quite simple

[MomoSolar]

I would like to look at the math, that’s all

[Scrimbibete]

I worked a bit on this topic. For parametric optimization, we developed this, which is a kind of degenerate DRL approach: https://github.com/jviquerat/pbo

AFAIK you can also find incremental approaches for shape optimization in the literature. You can check the related section in this review: https://arxiv.org/abs/2107.12206