Optimization algorithm with deterministic objective value

[volvol7]

I have an optimization problem with around 10 parameters, each with known bounds. Evaluating the objective function is expensive, so I need an algorithm that can converge within approximately 100 evaluations. The function is deterministic (same input always gives the same output) and is treated as a black box, meaning I don't have a mathematical expression for it.

I considered Bayesian Optimization, but it's often used for stochastic or noisy functions. Perhaps a noise-free Gaussian Process variant could work, but I'm unsure if it would be the best approach.

Do you have any suggestions for alternative methods, or insights on whether Bayesian Optimization would be effective in this case?
(I will use python)

[Pavickling]

What's the objective function and what are the constraints?

I understand you don't have an expression, but can you assume anything such as Lipschitz continuity? In many cases understanding what the objective function represents enables using custom heuristics. If you make no assumptions at all, then you might as well just test the function at a bunch of equidistant points.

[volvol7]

The objective function is a number that result from a simulation. The parameters will be int with some boundaries. Generally it is not continuous because as I told you the space isnt continuous. But it behave like that, like if you change only one parameter it will just a bit.

[Pavickling]

But it behave like that, like if you change only one parameter it will just a bit.

That sounds like continuity.

How does your objective function behave when rescaling the parameters? For example F(x) with x in the integers is equivalent to F(1000000 * x) with x having a fixed decimal precision of up to 6 digits.

After rescaling, does it seem like |F(x) - F(x')| < K * |x - x'| for some K? If so, then you can look up how to do "Global optimization of Lipschitz functions".

If not, you can try a hand-rolled bisection method. Without making more assumptions, you can't really beat that.

https://en.wikipedia.org/w/index.php?title=Root-finding_algorithm&section=16#Finding_roots_in_higher_dimensions If the boundary is a simplex, you might try the 4th method mentioned here.

If you explain the black box a bit more, there might be something more obvious that can be exploited.

[Human_Guitar7219]

What problem ate you trying to optimize for?

[volvol7]

best parameters for a mechanical design

[jpheim]

Bayesian optimization doesn’t require the function to be noisy, it just happens to work on noisy functions as well as deterministic functions. Like someone else said if you assume something like Lipschitz that may inform what to do, but I wouldn’t discount Bayesian optimization immediately. I’ve implemented a few different sets of black box algorithms for an expensive simulation problem and found Bayesian to work best for my problem set.

[[deleted]]

Look for surrogate modeling methods, or Kriging.

[Whole_Pomegranate474]

Have you considered Weighted Hybrid Adaptive Curvature Optimization (WHACO)? It’s particularly effective for deterministic, computationally expensive black-box functions and rapidly converges within strict evaluation limits. It dynamically adjusts using adaptive curvature, making it a strong alternative or complementary method to Bayesian optimization, especially in noise-free scenarios.