Differentiation¶
cuPIQP computes reverse-mode vector-Jacobian products (VJPs) through a solved QP using implicit differentiation of the KKT conditions. The backward pass reuses the KKT factorization from the forward solve instead of constructing the full solution Jacobian.
If an outer scalar loss \(\ell\) depends on the solution \(x^\star\), call
backward() with the upstream gradient \(\partial \ell / \partial x^\star\). The returned
data object contains the corresponding gradients with respect to P, c, A, b,
G, h_l, h_u, x_l, and x_u. A bound side that was omitted at setup() has no
gradient block (its gradient is absent / (B, 0)), mirroring the inputs you provided.
Basic workflow¶
Enable gradient support before setup() by setting solver.settings.enable_grad=True, because setup allocates the backward
buffers and prepares backend-specific kernels:
import cupy as cp
from cupiqp import DenseSolver, Status
P = cp.eye(2)
c = cp.array([-1.0, 2.0])
solver = DenseSolver()
solver.settings.enable_grad = True # enable gradient computation
solver.setup(P=P, c=c)
solver.solve()
assert solver.result.info.status.numpy()[0] == Status.CUPIQP_SOLVED
x_star = solver.result.x
# l = 0.5 * ||x*||^2, so dl/dx* = x*
grad_data = solver.backward(grad_x=x_star)
grad_c = grad_data.c
grad_P = grad_data.P
For a single problem, solution and gradient arrays still carry a leading batch
dimension, so x_star and grad_c above have shape (1, 2).
Upstream gradients¶
backward() accepts upstream gradients for every solution variable:
grad_data = solver.backward(
grad_x=grad_x,
grad_y=grad_y,
grad_z_l=grad_z_l,
grad_z_u=grad_z_u,
grad_z_bl=grad_z_bl,
grad_z_bu=grad_z_bu,
grad_s_l=grad_s_l,
grad_s_u=grad_s_u,
grad_s_bl=grad_s_bl,
grad_s_bu=grad_s_bu,
)
Each upstream gradient must have the same shape as its corresponding field in solver.result.
Omitted upstream gradients are treated as zero, so losses that depend only on x usually need
only grad_x.
The returned object matches the selected backend:
| Solver | Return type | Matrix-gradient storage |
|---|---|---|
DenseSolver |
DenseData |
dense arrays with a leading batch dimension |
SparseSolver |
SparseData |
values at the original CSR structural nonzeros |
MultistageSolver |
MultistageData |
block-structured matrices and vectors |
All returned gradients are expressed in the original, unscaled problem coordinates; the backward pass handles any solver preconditioning internally.
Repeated solves¶
The backward pass can follow each solve in a fixed-structure update loop:
for c_k in costs:
solver.update(c=c_k)
solver.solve()
grad_x = outer_loss_gradient(solver.result.x)
grad_data = solver.backward(grad_x=grad_x)
As with the forward solution, every problem in a batch is differentiated independently
and gradients carry the leading batch dimension (B, ...).
backward() needs a converged solve
backward() differentiates the converged QP, so it requires the preceding solve()
to have reached CUPIQP_SOLVED for every batch element; otherwise it raises. An
update() marks the cached solution unsolved, so you must solve() again before the
next backward() (as the loop above does).
Gradient buffers are reused
Each solver returns the same internal gradient-data object on every backward()
call. Its buffers are overwritten by the next backward pass. Copy any gradient that
must be retained:
Differentiability at active-set changes
The VJP describes the local solution map at the converged QP. At points where the active set changes, that map can be nonsmooth and the gradient may be discontinuous or undefined. Gradient smoothing replaces it with a smooth surrogate.
Gradient smoothing¶
At a converged solution the complementarity product s * z is driven to ~0, so each
constraint is either strictly active or strictly inactive. Across an active-set change
that hard switch makes the VJP jump or become undefined. Following
qpax, gradient smoothing instead differentiates at
a relaxed point where s * z = mu for a small fixed mu > 0, which keeps the
solution map smooth through active-set transitions.
Enable it (with enable_grad) before setup(); the mode is fixed at setup:
solver.settings.enable_grad = True
solver.settings.gradient_smoothing = True
solver.settings.gradient_smoothing_mu = 1e-3 # smoothing amount (default)
solver.setup(P=P, c=c, x_l=x_l)
solver.solve()
grad = solver.backward(grad_x=grad_x) # smoothed surrogate VJP
How it works: after the normal solve, backward() runs a short damped-Newton relaxation
(gradient_smoothing_tol, gradient_smoothing_max_iter) from the converged point to a
nearby point satisfying s * z = mu, then applies the ordinary implicit-diff VJP there.
The settings are summarized in Settings.
Smoothing is fixed at setup
settings.gradient_smoothing is read at setup() (it gates the relaxation buffers), so
it cannot be toggled afterwards — changing it before a later backward() raises. The
amount gradient_smoothing_mu and the inner-solve controls gradient_smoothing_tol /
gradient_smoothing_max_iter may be changed freely between backward() calls.
It is a surrogate gradient, not the derivative of the forward map
The forward solve() still returns the true optimum x*; only the gradient is
changed. backward() returns the derivative of the relaxed solution map x_mu,
not of x*. Consequently a finite difference of the forward solution will not match
the smoothed gradient near an active-set change (they agree only as mu -> 0, away
from boundaries).
Smoothness and bias are qualified
Smoothing removes the active-set switch under the usual regularity conditions (a
nonsingular relaxed KKT Jacobian); it does not make every QP globally smooth. The bias
is mu-controlled: roughly O(mu) away from transitions and O(sqrt(mu))
in the primal at a transition. cuPIQP's factor keeps the solver's rho/delta
regularization, so the result is qpax-style rather than bit-exact qpax.
The relaxed point is smooth in all of (x, y, s, z) (since s * z = mu > 0), so
cotangents on any solution variable — primal x, equality dual y, and the
inequality/box duals and slacks — return the corresponding relaxed-map VJP, exactly as in
the standard path. Smoothing is available on DenseSolver, SparseSolver,
MultistageSolver, and the PyTorch layer.
The example notebook examples/differentiable_qp_2d.ipynb works this through on a 2-D box QP,
visualizing how smoothing rounds the gradient's discontinuity at the active-set boundaries.
See the solver API for the full backward() signature and
Settings for enable_grad and solver
tolerances.