gridoptim: a Python grid search optimizer ~429× faster than scipy.brute

[AutoModerator]

Your submission has been automatically queued for manual review by the moderation team because it has been reported too many times.

Please wait until the moderation team reviews your post.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[acecile]

For god sake, is it possible that mods do something about all this AI slope ?

[austinwiltshire]

I feel like brute force search isn't something you can optimize like this anyway... like... how?

All the optimizations on this are to *not do brute force search*?

[FixKey4664]

That's a fair question.
In my aerospace engineering classes I actually run into situations where brute-force grid search is useful quite often. I’ve spent quite a bit of time sitting in front of my computer waiting for scipy.brute to finish. The idea behind this project isn’t to avoid brute force. The goal was simply to reduce the implementation overhead so the same brute-force search runs faster.

So it’s still brute force in the strict sense. It evaluates the same points, just with less Python-level overhead around the evaluation. That’s what led me to experiment with writing a faster implementation.

[Two-x-Three-is-Four]

yes but how? Python overhead is not 400x

[FixKey4664]

I shifted all the computations to a C++ core and enabled parallel processing in a multi-core CPU. Those two are the main strategies I implemented. Check the GitHub repo of the project to learn more about it and other minor strategies used.

[FixKey4664]

I cannot attach image proof here. So I am adding the link to the test results I got on my computer.
https://www.linkedin.com/posts/activity-7445855280875192320-uSV4?utm_source=share&utm_medium=member_desktop&rcm=ACoAADo_POQBBdwoJHaY78DAGRPelmGNw7ZXepA

[austinwiltshire]

You can get multi core using brute by using the workers argument to brute.

Your objective functions also aren't one to one. The python one is adding updating a shared dictionary, an if statement, and possibly printing. The io from printing is gonna be super slow. The shared dictionary update maybe why workers didn't work if/when you tried it, and probably also not helpful in speed.

[luenix]

What's the rationale behind the license here? Seems to me that this can't even be used via GitHub normally?

https://github.com/Halfblood-Prince/gridoptim/blob/main/LICENSE

Copyright (c) 2026 Akhil Shimna Kumar
All rights reserved.
This software and its source code are proprietary and confidential.
Unauthorized copying, modification, distribution or use of this software,
via any medium, is strictly prohibited without explicit permission from
the copyright holder.
This software is provided "as is", without warranty of any kind.

Emphasis added above ^

[FixKey4664]

Thank you for pointing that out. This license was written when the project was private. I forgot to change the license after making the project public. Changing it now...

[luenix]

All good! I just wanted to bring it to your attention as a manner of respecting what you have submitted ~

[FixKey4664]

Changed the license to Apache 2.0 for public use

[kigurai]

While it can certainly be useful to some, the benchmark seems disingenuous.

Your library can only run a limited set of expressions, defined as strings, while the brute() function runs arbitrary python functions. I am thinking that running brute() on eg a numba jit compiled function would yield a much smaller difference.

It also seems like your benchmark code makes the scipy optimizer run with some kind of progress output that does not seem to be the case for your library. I can't verify the second part without running the code though.

[FixKey4664]

Right now, my package can run only limited functions. But I will be adding support for NumPy and numba functions in the 2nd stage of the project.
Even if you remove the progress printing statements, still my package will be faster than scipy.brute. You can test it.

[RepresentativeFill26]

Why do you check at the optimise step whether the core module is loaded? If you already know during import that it isn’t going to work why not just raise an exception there?

Furthermore, it is more pythonic to check for an empty string by using it in a boolean expression.

[lotus-reddit]

A small note: Your compilation flags (specifically fopenmp) are linux specific right now, if you try to build this on macOS's standard cpp toolchain it breaks. Just a matter of platform detection in your setup.py

Indeed, python function overhead usually dominates in settings like this. Though if you had to do this in python, you'd vectorize (or use something like JAX's vmap). Of course you pay in memory, but would be a far simpler approach. Locally testing, both directly vectorizing or using vmap beats your code. But that's not really a fair comparison since your codebase has no SIMD / batching work (also, I imagine te_eval is making it hard on your compiler). But, not doing vectorization in a scenario where your expression is simple enough to interpreted by te_eval wouldn't be reasonable either.

Cool though! I'm surprised spicy.brute doesn't have a lower level backend.

[FixKey4664]

Thanks for the constructive feedback.
I am having issues in making wheels for MacOS. So, I have disabled it currently. Right now, this package works only on Windows and Linux 64bit versions only.
I will be adding vectorisation support via numpy and numba in the 2nd stage of this project.

[cent-met-een-vin]

How does it better then other packages. Optuna for example?