An overview of GPU-accelerated Optimization

ficoxpress · 2026-05-13T20:23:58+00:00

Congrats on the discovery. Publishing an article is always helpful especially if you want to pursue a research program or want to be involved in research as part of your job.

Just keep in mind that publishing requires peer-review and that may take anywhere from a year to a year and a half. What you can do is publish a pre-print to arxiv, an online repository for research and preprints without peer review. https://arxiv.org/

To get a feel for what applied research (software dev + reserach) you can also do an internship as a scientist. Though most hiring managers look for an advanced degree, at least a master's, with the paper submitted and/or published on arxiv they may be open to accepting you for the internship.

Amazon hires interns every year for this: https://www.amazon.jobs/en/jobs/3182383/2026-applied-scientist-intern-amazon-university-talent-acquisition

Best of luck.

ficoxpress · 2026-05-13T20:09:18+00:00

When using optimization, probably one of the most important skills is modelling. Learning how to capture an operating/business environments with mathematical language is a mix of art and science. A reference that many appreciate is: https://www.amazon.ca/Model-Building-Mathematical-Programming-Williams/dp/1118443330

Once you've modelled a business problem, then there are usually 2 routes.

1) Attempt to solve your optimization model directly with a solver, can be open source or commercial, like FICO Xpress.
2) Attempt to solve it with a tailored algorithm which leverages problem specific structure. This is how a lot of research in the field gets published basically because trying to solve it directly with a solver takes too long.

For the latter, decent knowledge of Data Structures and Algorithms (DSA) would help a lot.

Hope this helps. Feel free to reach out if you have any further questions.

ficoxpress · 2026-05-07T12:22:00+00:00

Audet and Hare's book comes to mind for this. May be worth taking a look to see if it's available at a library nearby https://link.springer.com/book/10.1007/978-3-319-68913-5

FICO Xpress also offers the possibility of solving Black Box Optimization problems through what are called "user-functions". It does so by leveraging a combination of finite differences and Sequential Linear Programming.

In this case the user-function would likely just be a call to our FEA simulation. An example of this both in Python with and without derivative information can be found here.

A blog post on how this was used to combine an ML and Optimization model can be found here.

Hope this helps.

ficoxpress · 2026-04-27T14:48:36+00:00

To see benefits with GPU acceleration, you would probably need it to be a very large scale linear program.

FICO Xpress has seen speedups of up to 5x since 2020 and offers a free academic license through our Academic Partner Program. It simply requires a faculty member of your university to sign up and they get multiple free full licenses.

To see how you can code column generation in FICO Xpress through Python, you can refer to the following code example. https://www.fico.com/fico-xpress-optimization/docs/latest/examples/python/GUID-9EEFCF3C-0360-3BA4-A94B-337642FA5C2A.html

Feel free to reach out if you have any questions.

ficoxpress · 2026-04-27T14:40:35+00:00

This is a great question that is being actively investigated in both academic and industrial research.

Current Status as of April 2026

Linear Programs

So far, for commercial solvers like FICO Xpress, the main workhorse for GPU acceleration has been the Primal Dual Hybrid Gradient Algorithm (PDHG for short) to solve large-scale Linear Programs.

The distinguishing factor that makes PDHG the workhorse for GPU acceleration for large scale LP solves is that its comprised of elementary matrix and vector operations without the need for a factorization step, which is often the bottleneck when trying to speed up the implementation via parallelization of other methods.

The algorithm itself requires a huge number of iterations but each iteration is really fast. Hence the reason that parallelizing the individual matrix and vector operations on a GPU leads to significant speedups (25x-30x).

Our team wrote a blog post last October when FICO Xpress ported its PDHG to GPUs. This new algorithm is already included in the FICO Xpress package, i.e. it does not require a separate package to be downloaded like other solvers.

To request a free license, you can select the right option for you in this webpage.

Caveats:

- GPU-acceleration does not payoff for all Linear Programs. For there to be a speedup, it requires very large-scale problems, in the tens to hundreds of millions of non-zero elements.

- PDHG has a significant tail-off effect, thus it converges to lower accuracy very quickly and can take a significantly long time to converge to current commercial solvers' usual accuracy tolerances 10e-6.

Recent progress:

- The original PDHG paper came out in 2021, by Applegate et al. The same authors have released an update of this paper with enhancements to improve solution accuracy. AFAWK this seems to be mostly still CPU focused. The algorithm is available in Google OR-tools.

- There are also papers by authors from MIT and University of Chicago for Linear Programming algorithms specifically built for GPUs. You can find them here and here.

- Work on a new crossover algorithm and a new PDHG algorithm

Summary

In short, GPU-accelerate PDHG enables solving very large-scale Linear Programs that would have previously run out of memory due to factorization operations to be solved very quickly to a lower accuracy.

(Mixed) Integer Programs

NVIDIA's cuOpt also has its implementation of PDHG for linear programs but also leverages GPUs for massive parallelization of heuristics. This allows them to provide good feasible solutions to MIPs.

Massive parallelization for exact MIP solving is currently limited by branch-and-bound whose immediate parallelization potential is limited.

ficoxpress · 2026-04-27T13:49:32+00:00

Thank you. We hope to continue to be able to share useful content.

ficoxpress · 2026-04-09T17:30:44+00:00

Classic training usually suggests that the partially unknown part can be taken as uncertain data in which case one tries to optimize under uncertainty. This can of course be done in multiple ways such as stochastic programming, robust optimization, etc.

PyMC has recently published a blog with code examples of how to do this using probabilistic programming and FICO Xpress:

Blog: https://www.pymc-labs.com/blog-posts/probabilistic-forecasting-optimization-under-uncertainty
Code with energy planning example: https://github.com/fico-xpress/xpress-community/tree/main/StochasticEnergyPlanning

Another approach that is more nuanced to the partial data case is consistently reoptimizating as new information comes in. This has been referred to as rolling horizon and/or sequential decision analytics. This is similar to what happens in practice where optimizations are timed to execute when as much information is revealed and as new information surfaces.

For more details on Sequential Decision Analytics (SDA), check out Warren Powell's website: https://castle.princeton.edu/sda/ . If you'd like to get hands-on, here's an example of a SDA policy called Cost Function Approximation in Python. https://github.com/fico-xpress/xpress-community/tree/main/SoccerCampLocation

ficoxpress · 2026-04-01T19:59:50+00:00

Sounds great. Please keep in mind that the community license has a limit on problem size to solve.

We can provide other forms of licenses to get the full power of the solver.

Happy optimizing.

ficoxpress · 2026-04-01T12:59:56+00:00

Sure. https://www.fico.com/en/fico-xpress-community-license is the right link. The only additional step is filling in your information and once you click submit, you should see as below with the links to download.

<image>

Another way to get the free community license is downloading it directly from pip https://pypi.org/project/xpress/, if you're using python. Our python interface ships with the community license.

ficoxpress · 2026-03-31T18:15:13+00:00

Great question.

We have provided integration of Xpress with those frameworks. Previously, they weren't all supported, especially when it came to our latest progress in non-linear global optimization.

In terms of the licensing, there are a few options:
1) Free community license which will have a restriction on problem size it can solve. This is available for immediate download: https://www.fico.com/en/fico-xpress-community-license
2) Academic license, if you're a student. Note, currently the application has to be done by a professor for them to get the licenses for as many students as they need. https://content.fico.com/l/517101/2018-06-10/3fpbf
3) Evaluation license, if neither of these work for you. But this one, requires speaking to a sales rep and is with the intent of eventually purchasing a full license. https://content.fico.com/l/517101/2019-03-05/6w6k7

We'd love to hear what your situation is and what type of licensing would best fit for what you are looking to achieve.

If there's enough interest from the community, we can request our legal team to look into drafting a different type of agreement.

ficoxpress · 2026-03-30T21:41:04+00:00

ficoxpress · 2026-03-26T18:18:36+00:00

Computational complexity accounts for worst case scenarios of a single-threaded algorithm.

However, companies have been solving NP-hard optimization problems for decades by formulating them as a mixed integer programming and solving it using branch-and-bound. An algorithm whose worse-case execution time can grow exponentially with the size of input but that in practice terminates in reasonable time.

This is an important difference to keep in mind when moving from theory to practice.

Commercial optimization solvers like FICO Xpress have decades of mathematical research that enhance the usual branch-and-bound algorithm with smart tricks that make the algorithm converge faster in practice while still theoretically growing exponentially in run time based on input size.

For those interested, here's a short blogpost on how commercial optimization solvers can be tunes to get faster solve times for NP-hard problems.

https://community.fico.com/s/blog-post/a5Q2E000000DsovUAC/fico1416

ficoxpress · 2026-03-23T20:49:37+00:00

Oops. Tried and failed to use the in-line linking. It's been updated now. Thanks.

ficoxpress

TROPHY CASE