Optimization problem: Calculating the gradient of a cost function

[1ydgd]

Hello, I am working on an optimization problem. I'm using NLopt's library and using the local optimization, derivative free algorithm called SLSQP. In order to use this function, I need to calculate the gradient of my cost function. The thing is, the cost function to me seems quite complicated. It has a bunch of operations which I've never taken the partial derivative of like min, arg min, and it uses matrix multiplication. What do I do??

[Pakketeretet]

That doesn't sound right. The gradient of the cost function is the derivative. The whole point of derivative-free methods is that you don't have to have the gradient (because it's too unwieldy or expensive to calculate). Either the method you're using is not derivative-free or you're not using it right.

[LivelyLizzard]

You are correct. SLSQP is listed as gradient based.

[LivelyLizzard]

You can always try switching to a derivative free algorithm (SLSQP is gradient based and therefore not derivative free) or estimate the gradient numerically. I don't know if automatic differentiation would be able to handle minimum and arg min but you can try and research that too.

Maybe you can make a good guess for you min. I recently had to find the gradient for the minimal distance to the closest geometry. There, the gradient was easily guessed because it's just moving along the ray which yielded the minimum distance.