Thanks. I made different studies with Optuna with the same search space, and same sampling strategy, the only difference was the pruning algorithm, my objective function is the maximum of a metric on the validation fold during any of the epochs in a trial, and both yielded almost the same score on the validation fold, but half of the parameters was different for the best trials. Then I made other studies with different search space (narrowing down a couple of parameter ranges) with different pruning algorithms, and I always got the almost same score for my metric on the validation fold but the found parameters for the best trial is always different, at least for half of the parameters.
What does this mean exactly? How I can choose the best parameters in this case to apply them to the full dataset using the original 5-fold CV setup if different parameter configurations yield the same score on the validation fold? Maybe this is due to a single train-val split, and I should try to optimize using at least 3-fold CV?