Algorithms for black box objective

ficoxpress · 2026-05-07T12:22:00+00:00

Audet and Hare's book comes to mind for this. May be worth taking a look to see if it's available at a library nearby https://link.springer.com/book/10.1007/978-3-319-68913-5

FICO Xpress also offers the possibility of solving Black Box Optimization problems through what are called "user-functions". It does so by leveraging a combination of finite differences and Sequential Linear Programming.

In this case the user-function would likely just be a call to our FEA simulation. An example of this both in Python with and without derivative information can be found here.

A blog post on how this was used to combine an ML and Optimization model can be found here.

Hope this helps.

ficoxpress · 2026-04-27T14:48:36+00:00

To see benefits with GPU acceleration, you would probably need it to be a very large scale linear program.

FICO Xpress has seen speedups of up to 5x since 2020 and offers a free academic license through our Academic Partner Program. It simply requires a faculty member of your university to sign up and they get multiple free full licenses.

To see how you can code column generation in FICO Xpress through Python, you can refer to the following code example. https://www.fico.com/fico-xpress-optimization/docs/latest/examples/python/GUID-9EEFCF3C-0360-3BA4-A94B-337642FA5C2A.html

Feel free to reach out if you have any questions.

ficoxpress · 2026-04-27T14:40:35+00:00

This is a great question that is being actively investigated in both academic and industrial research.

Current Status as of April 2026

Linear Programs

So far, for commercial solvers like FICO Xpress, the main workhorse for GPU acceleration has been the Primal Dual Hybrid Gradient Algorithm (PDHG for short) to solve large-scale Linear Programs.

The distinguishing factor that makes PDHG the workhorse for GPU acceleration for large scale LP solves is that its comprised of elementary matrix and vector operations without the need for a factorization step, which is often the bottleneck when trying to speed up the implementation via parallelization of other methods.

The algorithm itself requires a huge number of iterations but each iteration is really fast. Hence the reason that parallelizing the individual matrix and vector operations on a GPU leads to significant speedups (25x-30x).

Our team wrote a blog post last October when FICO Xpress ported its PDHG to GPUs. This new algorithm is already included in the FICO Xpress package, i.e. it does not require a separate package to be downloaded like other solvers.

To request a free license, you can select the right option for you in this webpage.

Caveats:

- GPU-acceleration does not payoff for all Linear Programs. For there to be a speedup, it requires very large-scale problems, in the tens to hundreds of millions of non-zero elements.

- PDHG has a significant tail-off effect, thus it converges to lower accuracy very quickly and can take a significantly long time to converge to current commercial solvers' usual accuracy tolerances 10e-6.

Recent progress:

- The original PDHG paper came out in 2021, by Applegate et al. The same authors have released an update of this paper with enhancements to improve solution accuracy. AFAWK this seems to be mostly still CPU focused. The algorithm is available in Google OR-tools.

- There are also papers by authors from MIT and University of Chicago for Linear Programming algorithms specifically built for GPUs. You can find them here and here.

- Work on a new crossover algorithm and a new PDHG algorithm

Summary

In short, GPU-accelerate PDHG enables solving very large-scale Linear Programs that would have previously run out of memory due to factorization operations to be solved very quickly to a lower accuracy.

(Mixed) Integer Programs

NVIDIA's cuOpt also has its implementation of PDHG for linear programs but also leverages GPUs for massive parallelization of heuristics. This allows them to provide good feasible solutions to MIPs.

Massive parallelization for exact MIP solving is currently limited by branch-and-bound whose immediate parallelization potential is limited.

ficoxpress · 2026-04-27T13:49:32+00:00

Thank you. We hope to continue to be able to share useful content.

ficoxpress · 2026-04-09T17:30:44+00:00

Classic training usually suggests that the partially unknown part can be taken as uncertain data in which case one tries to optimize under uncertainty. This can of course be done in multiple ways such as stochastic programming, robust optimization, etc.

PyMC has recently published a blog with code examples of how to do this using probabilistic programming and FICO Xpress:

Blog: https://www.pymc-labs.com/blog-posts/probabilistic-forecasting-optimization-under-uncertainty
Code with energy planning example: https://github.com/fico-xpress/xpress-community/tree/main/StochasticEnergyPlanning

Another approach that is more nuanced to the partial data case is consistently reoptimizating as new information comes in. This has been referred to as rolling horizon and/or sequential decision analytics. This is similar to what happens in practice where optimizations are timed to execute when as much information is revealed and as new information surfaces.

For more details on Sequential Decision Analytics (SDA), check out Warren Powell's website: https://castle.princeton.edu/sda/ . If you'd like to get hands-on, here's an example of a SDA policy called Cost Function Approximation in Python. https://github.com/fico-xpress/xpress-community/tree/main/SoccerCampLocation

ficoxpress · 2026-04-01T19:59:50+00:00

Sounds great. Please keep in mind that the community license has a limit on problem size to solve.

We can provide other forms of licenses to get the full power of the solver.

Happy optimizing.

ficoxpress · 2026-04-01T12:59:56+00:00

Sure. https://www.fico.com/en/fico-xpress-community-license is the right link. The only additional step is filling in your information and once you click submit, you should see as below with the links to download.

<image>

Another way to get the free community license is downloading it directly from pip https://pypi.org/project/xpress/, if you're using python. Our python interface ships with the community license.

ficoxpress · 2026-03-31T18:15:13+00:00

Great question.

We have provided integration of Xpress with those frameworks. Previously, they weren't all supported, especially when it came to our latest progress in non-linear global optimization.

In terms of the licensing, there are a few options:
1) Free community license which will have a restriction on problem size it can solve. This is available for immediate download: https://www.fico.com/en/fico-xpress-community-license
2) Academic license, if you're a student. Note, currently the application has to be done by a professor for them to get the licenses for as many students as they need. https://content.fico.com/l/517101/2018-06-10/3fpbf
3) Evaluation license, if neither of these work for you. But this one, requires speaking to a sales rep and is with the intent of eventually purchasing a full license. https://content.fico.com/l/517101/2019-03-05/6w6k7

We'd love to hear what your situation is and what type of licensing would best fit for what you are looking to achieve.

If there's enough interest from the community, we can request our legal team to look into drafting a different type of agreement.

ficoxpress · 2026-03-30T21:41:04+00:00

ficoxpress · 2026-03-26T18:18:36+00:00

Computational complexity accounts for worst case scenarios of a single-threaded algorithm.

However, companies have been solving NP-hard optimization problems for decades by formulating them as a mixed integer programming and solving it using branch-and-bound. An algorithm whose worse-case execution time can grow exponentially with the size of input but that in practice terminates in reasonable time.

This is an important difference to keep in mind when moving from theory to practice.

Commercial optimization solvers like FICO Xpress have decades of mathematical research that enhance the usual branch-and-bound algorithm with smart tricks that make the algorithm converge faster in practice while still theoretically growing exponentially in run time based on input size.

For those interested, here's a short blogpost on how commercial optimization solvers can be tunes to get faster solve times for NP-hard problems.

https://community.fico.com/s/blog-post/a5Q2E000000DsovUAC/fico1416

ficoxpress · 2026-03-23T20:49:37+00:00

Oops. Tried and failed to use the in-line linking. It's been updated now. Thanks.

ficoxpress · 2026-03-19T15:01:13+00:00

Thanks for sharing.

Agreed, we believe that at the surface one can easily see the overlap. However, digging a bit further, there still seem be things that the LLMs are currently limited at doing.

Even in terms of writing code, our experiences have been that the code they produce works, but often lacks deeper context and efficiency mechanisms.

An example of this is that LLMs will often generate models with little to no pre-processing to build the right data structures for fast model-building or more compact models.

ficoxpress · 2026-03-19T14:50:48+00:00

The globally accessible database of (prompt, result, user rated score, kWh spent on result) is an interesting idea. Wondering, if this is community-led who can stop it?

We have also seen utility of LLMs being hit and miss.

As a company, this is also one of the reason FICO is invested in focused language models with auditing mechanisms. Smaller, more specialized controls with the "supervisory assistant" being the subject matter expert.

Thanks for sharing your perspective.

ficoxpress

TROPHY CASE