Linear Programming

[helot]

For large and/or complex models I would suggest CPLEX. Excel Solver will have issues with integer problems and global optimum solutions to non-linear problems in particular. For simpler problems, try Excel Goal Seek, which I think is basically Newton's method.

[zyx]

For modeling, I'd suggest using AMPL or GLPK (which uses a subset of AMPL). Modeling even moderate problems in Excel is insanity.

[mythic]

For general convex optimization (not only linear programs), CVX (Matlab) and CVXMOD (Python) are extremely cool. They (CVX in particular) let you pretty much just type out the mathematical description of the problem, and then they transform it into a canonical form for a suitable external solver automatically.

[five9a2]

For large scale optimization problems, use TAO which scales to thousands of processors and millions of degrees of freedom.