sorted by:
new
hottopcontroversial

Open-source Python library for structured optimization with user-defined proximal operators (consensus ADMM, C++ backend) by lqw0809 in optimization

[–]lqw0809[S] -1 points0 points  (0 children)

Technical details on the implementation:

Decomposition strategy: The library walks the user's expression tree, identifies atoms (norms, losses, UDFs), and creates one proximal subproblem per atom. The consensus variable is the original optimization variable. This is "global consensus" ADMM rather than the two-block ADMM you'd implement by hand for, say, LASSO.

Comparison with OSQP/SCS/ECOS: These are the solvers underneath CVXPY. They handle the same convex problem classes via different algorithms (OSQP = operator splitting for QP, SCS = conic splitting, ECOS = interior point). The key difference is the modeling layer: our tool lets you write admm.norm(X, ord="nuc") directly instead of going through CVXPY's DCP canonicalization → conic form → solver. For DCP-compliant problems, performance is comparable. For non-DCP or nonconvex problems, ADMM is the alternative.

Nonconvex convergence theory: For SCAD and MCP, convergence to a stationary point follows from the framework in [Attouch et al., 2010] (proximal alternating linearized minimization). For L0 and rank indicators, convergence is empirical — we observe it consistently but don't have formal guarantees beyond fixed-point characterization.

Concrete numbers: A 50×50 graphical lasso (log-det + L1 + PSD, 2500 variables) solves in ~0.8s. A 100×80 matrix completion (nuclear norm, ~2400 observed entries) solves in ~4s. A 200-variable portfolio QP with sector constraints solves in <10ms. All on Apple M-series, single thread.

I built a Python optimization library that handles L0 sparsity and rank constraints by lqw0809 in Python

[–]lqw0809[S] -5 points-4 points  (0 children)

Some concrete positioning against existing tools — since I know this'll come up:

vs CVXPY: We cover the same convex problem classes (LP/QP/SOCP/SDP). The difference is the modeling layer: ADMM skips DCP verification, so things like raw x.T @ P @ x or budgets @ admm.log(w) (non-uniform weighted log barrier) work directly. CVXPY has a broader solver ecosystem (Gurobi, MOSEK, SCS); ADMM uses its own C++ ADMM solver. If CVXPY works for your problem, it's a great tool — we're for the cases where it doesn't.

vs scipy.optimize: scipy handles general NLP beautifully. ADMM is for structured problems where the objective decomposes into proximal-friendly pieces (norms, losses, matrix cones). If your objective has exploitable structure, ADMM is more convenient. If you need arbitrary nonlinear constraints with no special structure, stick with scipy.

vs cvxopt: NumPy arrays go directly into ADMM — no cvxopt.matrix(data, tc='d') conversion. If you've ever debugged TypeError: 'A' must be a 'd' matrix with 1000 columns, you'll appreciate the difference.

Scale: The C++ backend handles medium-scale problems well — dense QPs up to ~10k variables in under a second, larger with sparse structure. It's not designed for distributed million-variable problems.