Skip to content

Solvers

All solver classes share the setup / solve / update workflow and use the same Settings. See Re-solving with new data for the fixed-structure update pattern and Differentiation for the backward() workflow. They differ only in the accepted storage format for P, A, G and the KKT factorization used. See Backends for guidance on choosing one.

DenseSolver

DenseSolver

DenseSolver(
    dtype: Union[type[float32], type[float64]] = wp.float64,
    stream=None,
)

Bases: SolverBase

GPU solver for general dense convex quadratic programs that solves a QP - or a whole batch of QPs - of the form

\[ \begin{aligned} \min_{x}\quad & \tfrac{1}{2}\, x^\top P x + c^\top x \\ \text{s.t.}\quad & A x = b, \\ & h_l \le G x \le h_u, \\ & x_l \le x \le x_u, \end{aligned} \]

using the proximal interior-point method, running entirely on the GPU.

Inputs. P, A, G and every vector (c, b, h_l, h_u, x_l, x_u) must be dense arrays that live on the GPU: a warp.array, a cupy.ndarray, a CUDA torch.Tensor, a CUDA JAX array, or any other object exposing DLPack or the __cuda_array_interface__ protocol. Values are copied into the solver's own Warp buffers and cast to the solver dtype.

cuPIQP is GPU-only: CPU data (numpy.ndarray, CPU torch tensors, CPU JAX arrays) is rejected with a TypeError rather than copied to the device behind your back.

Batching. DenseSolver solves B independent QPs in a single GPU call. Every argument of setup and update is either batched, with a leading batch axis (P of shape (B, n, n), c of shape (B, n), ...), one value per problem, or shared, in the single-problem shape ((n, n), (n,), ...), one value for every problem. setup reads B from the batched arguments and fixes the structure shared by all problems - the dimensions and which constraint blocks and bound sides exist.

Results are Warp arrays. solver.result.x is a (B, n) warp.array on the GPU; use .numpy() for a host copy, or view it zero-copy from another framework (cupy.asarray(x), torch.from_dlpack(x), jax.dlpack.from_dlpack(x)).

Parameters:

Name Type Description Default
dtype (float64, float32)

Floating-point precision used throughout the solve. wp.float32 is faster and uses less memory but converges to looser tolerances; the default convergence tolerances are chosen to match the dtype.

wp.float64
stream Stream

CUDA stream to run on. By default the solver creates and owns one; pass a warp.Stream to run on it instead (see the stream property).

None

Examples:

A small inequality-constrained QP (one row is one-sided via -inf):

import numpy as np
import warp as wp
from cupiqp import DenseSolver, Status

P = wp.from_numpy(np.eye(2), dtype=wp.float64)
c = wp.from_numpy(np.array([-1.0, -4.0]), dtype=wp.float64)
G = wp.from_numpy(np.array([[1.0, 1.0]]), dtype=wp.float64)  # constrain x1 + x2
h_l = wp.from_numpy(np.array([-np.inf]), dtype=wp.float64)   # no lower bound on the row
h_u = wp.from_numpy(np.array([1.0]), dtype=wp.float64)       # x1 + x2 <= 1

solver = DenseSolver()
solver.setup(P=P, c=c, G=G, h_l=h_l, h_u=h_u)   # all shared: one problem
solver.solve()

print(Status(solver.result.info.status.numpy()[0]).name)    # CUPIQP_SOLVED
x = solver.result.x.numpy()[0]              # bring the solution to the host

A batch of 8 problems that differ only in c:

rng = np.random.default_rng(0)
solver = DenseSolver()
c_batch = wp.from_numpy(rng.standard_normal((8, 2)), dtype=wp.float64)  # batched c: B = 8
solver.setup(P=P, c=c_batch, G=G, h_l=h_l, h_u=h_u)   # P, G, h_l, h_u shared by all 8
solver.solve()
c_new = wp.from_numpy(rng.standard_normal((8, 2)), dtype=wp.float64)
solver.update(c=c_new)                      # new values, same structure
solver.solve()
See Also

SparseSolver: solver for general sparse problems.

MultistageSolver: structure-exploiting solver for multistage optimization (e.g. optimal-control) problems.

Notes

setup can be called only once per instance; for a different structure (dimensions, constraint blocks, bound sides), create a new DenseSolver. update reuses all GPU allocations; bound values may switch between finite and +/-inf. Solver behaviour (tolerances, verbosity, iteration cap, ...) is configured through solver.settings.

Source code in cupiqp/solver.py
def __init__(self, dtype: Union[type[wp.float32], type[wp.float64]] = wp.float64, stream=None):
    wp.init()
    if dtype is not wp.float32 and dtype is not wp.float64:
        raise TypeError(
            f"Solver dtype must be wp.float32 or wp.float64; got {dtype!r}."
        )
    # One CUDA stream per solver instance: every public entry point makes
    # it Warp's current stream, so all kernels, copies, library calls and
    # CUDA graphs of this solver are issued and captured on it. A stream
    # given to the constructor is borrowed (never destroyed here).
    # Entering, the solver stream waits for the caller's current stream,
    # except while it is being captured: that wait would depend on
    # uncaptured work, and the caller already orders the captured stream.
    if stream is not None and not isinstance(stream, wp.Stream):
        raise TypeError(
            "stream must be a warp.Stream or None; got "
            f"{type(stream).__name__}. Order other frameworks' streams "
            "with the solver through cupiqp.torch, or with "
            "warp events on solver.stream."
        )
    self._stream = stream if stream is not None else wp.Stream("cuda")
    self._owns_stream = stream is None
    # All solver data lives on the device of the solver stream.
    self._device = self._stream.device
    self._dtype = dtype
    self._settings = Settings.for_dtype(dtype)
    self._data: Data = None
    self._result = Result()    # store the values of primal, dual and slack variables of current iteration, and other information
    self._step = Variables()   # used to store the step direction of primal and dual variables
    self._res_nr = Variables()  # used to store the non-regularized residuals
    self._res = Variables()  # used to store the regularized residuals
    self._prox_vars = Variables()  # used to store the proximal variables
    self._kkt_system = KKTSystem()
    self._preconditioner = None
    self._setup_done = False

    # Gradient smoothing (qpax-style relaxed-KKT differentiation) state.
    # The mode is frozen at setup(); the relaxation buffers (see setup) are
    # only allocated when it is on. The backward outer products read the
    # linearization point passed in by backward(): ``self._result`` (the true
    # optimum) in standard mode, or ``self._result_smoothed`` (the user-space
    # relaxed iterate) in smoothed mode.
    self._grad_smoothing = None
    self._result_scaled = Variables()
    self._result_smoothed = Variables()

setup

setup(
    P: CudaArray,
    c: CudaArray,
    A: Optional[CudaArray] = None,
    b: Optional[CudaArray] = None,
    G: Optional[CudaArray] = None,
    h_u: Optional[CudaArray] = None,
    h_l: Optional[CudaArray] = None,
    x_u: Optional[CudaArray] = None,
    x_l: Optional[CudaArray] = None,
) -> None

Fix the problem structure, load the data of every problem of the batch, and allocate all GPU memory.

Each argument is either shared - the shape of a single problem, e.g. P of shape (n, n) or c of shape (n,), used by every problem - or batched, with a leading batch axis, e.g. P of shape (B, n, n) or c of shape (B, n). The batch size B is read from the batched arguments, which must agree; if every argument is shared, B = 1. The dimensions and which constraint blocks and bound sides are present become the structure of the solver. Call :meth:solve right away, or change values with :meth:update first. Call setup() once per solver instance.

Parameters:

Name Type Description Default
P GPU array

Quadratic cost, (n, n) or (B, n, n). Must be symmetric positive semidefinite. Required.

required
c GPU array

Linear cost, (n,) or (B, n). Required.

required
A GPU array

Equality constraints A x = b: A of shape (p, n) or (B, p, n), b of shape (p,) or (B, p). Provide both or neither.

None
b GPU array

Equality constraints A x = b: A of shape (p, n) or (B, p, n), b of shape (p,) or (B, p). Provide both or neither.

None
G GPU array

Inequalities h_l <= G x <= h_u: G of shape (m, n) or (B, m, n), bounds (m,) or (B, m). At least one bound is required when G is given; an omitted side is absent for the lifetime of the solver. Use -inf / +inf entries for one-sided rows.

None
h_l GPU array

Inequalities h_l <= G x <= h_u: G of shape (m, n) or (B, m, n), bounds (m,) or (B, m). At least one bound is required when G is given; an omitted side is absent for the lifetime of the solver. Use -inf / +inf entries for one-sided rows.

None
h_u GPU array

Inequalities h_l <= G x <= h_u: G of shape (m, n) or (B, m, n), bounds (m,) or (B, m). At least one bound is required when G is given; an omitted side is absent for the lifetime of the solver. Use -inf / +inf entries for one-sided rows.

None
x_l GPU array

Box bounds x_l <= x <= x_u, (n,) or (B, n). An omitted side is absent for the lifetime of the solver.

None
x_u GPU array

Box bounds x_l <= x <= x_u, (n,) or (B, n). An omitted side is absent for the lifetime of the solver.

None

Raises:

Type Description
RuntimeError

If setup() has already been called on this instance.

TypeError

If an input is not a GPU dense array of the solver dtype.

ValueError

If an input has the wrong rank or shape, the batched inputs disagree on the batch size, or only one of A / b is given.

Notes

To set up B problems that are all equal for now, give at least one argument its batch axis, e.g. cupy.broadcast_to(c, (B, n)).

Source code in cupiqp/dense/dense_solver.py
def setup(
    self,
    P: CudaArray,
    c: CudaArray,
    A: Optional[CudaArray] = None,
    b: Optional[CudaArray] = None,
    G: Optional[CudaArray] = None,
    h_u: Optional[CudaArray] = None,
    h_l: Optional[CudaArray] = None,
    x_u: Optional[CudaArray] = None,
    x_l: Optional[CudaArray] = None
) -> None:
    """Fix the problem structure, load the data of every problem of the
    batch, and allocate all GPU memory.

    Each argument is either **shared** - the shape of a single problem,
    e.g. ``P`` of shape ``(n, n)`` or ``c`` of shape ``(n,)``, used by
    every problem - or **batched**, with a leading batch axis, e.g. ``P``
    of shape ``(B, n, n)`` or ``c`` of shape ``(B, n)``. The batch size
    ``B`` is read from the batched arguments, which must agree; if every
    argument is shared, ``B = 1``. The dimensions and which constraint
    blocks and bound sides are present become the structure of the
    solver. Call :meth:`solve` right away, or change values with
    :meth:`update` first. Call ``setup()`` once per solver instance.

    Parameters
    ----------
    P : GPU array
        Quadratic cost, ``(n, n)`` or ``(B, n, n)``. Must be symmetric
        positive semidefinite. Required.
    c : GPU array
        Linear cost, ``(n,)`` or ``(B, n)``. Required.
    A, b : GPU array, optional
        Equality constraints ``A x = b``: ``A`` of shape ``(p, n)`` or
        ``(B, p, n)``, ``b`` of shape ``(p,)`` or ``(B, p)``. Provide both
        or neither.
    G, h_l, h_u : GPU array, optional
        Inequalities ``h_l <= G x <= h_u``: ``G`` of shape ``(m, n)`` or
        ``(B, m, n)``, bounds ``(m,)`` or ``(B, m)``. At least one bound
        is required when ``G`` is given; an omitted side is absent for the
        lifetime of the solver. Use ``-inf`` / ``+inf`` entries for
        one-sided rows.
    x_l, x_u : GPU array, optional
        Box bounds ``x_l <= x <= x_u``, ``(n,)`` or ``(B, n)``. An omitted
        side is absent for the lifetime of the solver.

    Raises
    ------
    RuntimeError
        If ``setup()`` has already been called on this instance.
    TypeError
        If an input is not a GPU dense array of the solver dtype.
    ValueError
        If an input has the wrong rank or shape, the batched inputs
        disagree on the batch size, or only one of ``A`` / ``b`` is given.

    Notes
    -----
    To set up ``B`` problems that are all equal for now, give at least one
    argument its batch axis, e.g. ``cupy.broadcast_to(c, (B, n))``.
    """
    # Every non-None input must be a GPU dense array. cupiqp does not
    # silently do H2D copies.
    P, c, A, b, G, h_u, h_l, x_u, x_l = (
        _check_dense(name, v, ndim, self._dtype)
        for name, v, ndim in (("P", P, 2), ("c", c, 1), ("A", A, 2), ("b", b, 1),
                              ("G", G, 2), ("h_u", h_u, 1), ("h_l", h_l, 1),
                              ("x_u", x_u, 1), ("x_l", x_l, 1))
    )
    if (A is None) != (b is None):
        raise ValueError("A and b must either both be provided or both be None.")
    with wp.ScopedStream(self._stream, sync_enter=not self._stream.is_capturing):
        self._setup_impl(P, c, A, b, G, h_u, h_l, x_u, x_l)
        if self.settings.enable_grad:
            self._init_grad_data()

solve

solve() -> None

Solve the QP set up by setup().

Runs the proximal interior-point iterations on the GPU and writes the solution (primal x, dual, and slack variables) and per-problem diagnostics to solver.result. The call is asynchronous: it queues the work on solver.stream and returns without waiting for it.

Every result is a Warp array at a fixed address: solver.result.x and the other variables, and the per-problem diagnostics in solver.result.info (status holds one Status code per problem; CUPIQP_SOLVED means converged to tolerance). Work queued on solver.stream afterwards sees the finished solve without any synchronization; .numpy() waits for the solve and copies to the host::

solver.solve()
x = solver.result.x.numpy()
status = solver.result.info.status.numpy()
print(Status(status[0]).name)       # CUPIQP_SOLVED

After setup() you may solve() repeatedly, optionally calling update() in between to change the numerical data. Set solver.settings.verbose = True to print a per-iteration log (this makes the solve wait on the host every iteration).

CUDA graph capture: solve() and update() may be recorded into your own CUDA graph together with your own work, also inside your own wp.capture_while / wp.capture_if; every launch of the graph then runs a complete solve on the data the graph sets::

solver.solve()          # once before capturing: compiles every
                        # kernel and puts the settings on the GPU
with wp.ScopedStream(solver.stream), wp.ScopedCapture() as capture:
    wp.launch(build_qp, ..., outputs=[c_buf])
    solver.update(c=c_buf)
    solver.solve()
    wp.launch(use_solution, ..., inputs=[solver.result.x])
wp.capture_launch(capture.graph, stream=solver.stream)

Capture solver.stream and make it the current stream, as above, so that your work and the solver's are recorded on one stream. Pass the same input arrays to update() at every launch (the graph reads them by address), and keep the settings of the last solve before the capture. Verbose mode and the sparse backend are not supported inside a capture; iterative refinement (iterative_refinement_always_enabled) is, its loop is decided on the device like the interior-point loop.

Source code in cupiqp/solver.py
def solve(self) -> None:
    """Solve the QP set up by ``setup()``.

    Runs the proximal interior-point iterations on the GPU and writes the
    solution (primal ``x``, dual, and slack variables) and per-problem
    diagnostics to ``solver.result``. The call is asynchronous: it queues
    the work on ``solver.stream`` and returns without waiting for it.

    Every result is a Warp array at a fixed address:
    ``solver.result.x`` and the other variables, and the per-problem
    diagnostics in ``solver.result.info`` (``status`` holds one ``Status``
    code per problem; ``CUPIQP_SOLVED`` means converged to tolerance).
    Work queued on ``solver.stream`` afterwards sees the finished solve
    without any synchronization; ``.numpy()`` waits for the solve and
    copies to the host::

        solver.solve()
        x = solver.result.x.numpy()
        status = solver.result.info.status.numpy()
        print(Status(status[0]).name)       # CUPIQP_SOLVED

    After ``setup()`` you may ``solve()`` repeatedly, optionally calling
    ``update()`` in between to change the numerical data. Set
    ``solver.settings.verbose = True`` to print a per-iteration log (this
    makes the solve wait on the host every iteration).

    CUDA graph capture: ``solve()`` and ``update()`` may be recorded into
    your own CUDA graph together with your own work, also inside your own
    ``wp.capture_while`` / ``wp.capture_if``; every launch of the graph
    then runs a complete solve on the data the graph sets::

        solver.solve()          # once before capturing: compiles every
                                # kernel and puts the settings on the GPU
        with wp.ScopedStream(solver.stream), wp.ScopedCapture() as capture:
            wp.launch(build_qp, ..., outputs=[c_buf])
            solver.update(c=c_buf)
            solver.solve()
            wp.launch(use_solution, ..., inputs=[solver.result.x])
        wp.capture_launch(capture.graph, stream=solver.stream)

    Capture ``solver.stream`` and make it the current stream, as above,
    so that your work and the solver's are recorded on one stream. Pass
    the same input arrays to ``update()`` at every launch (the graph
    reads them by address), and keep the settings of the last solve
    before the capture. Verbose mode and the sparse backend are not
    supported inside a capture; iterative refinement
    (``iterative_refinement_always_enabled``) is, its loop is decided on
    the device like the interior-point loop.
    """
    with wp.ScopedStream(self._stream, sync_enter=not self._stream.is_capturing):
        self._solve_impl()

update

update(
    P: Optional[CudaArray] = None,
    c: Optional[CudaArray] = None,
    A: Optional[CudaArray] = None,
    b: Optional[CudaArray] = None,
    G: Optional[CudaArray] = None,
    h_u: Optional[CudaArray] = None,
    h_l: Optional[CudaArray] = None,
    x_u: Optional[CudaArray] = None,
    x_l: Optional[CudaArray] = None,
    check_validity: bool = False,
) -> None

Set new numerical data, per problem or shared, then solve().

Any argument left as None keeps its current value. Each argument is either batched - a leading batch axis, e.g. P of shape (B, n, n) or c of shape (B, n), one entry per problem - or unbatched - the single-problem shape, e.g. (n, n) or (n,), in which case the same value is used for every problem. Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

Parameters:

Name Type Description Default
P GPU array

New values for the corresponding problem block.

None
c GPU array

New values for the corresponding problem block.

None
A GPU array

New values for the corresponding problem block.

None
b GPU array

New values for the corresponding problem block.

None
G GPU array

New values for the corresponding problem block.

None
h_u GPU array

New values for the corresponding problem block.

None
h_l GPU array

New values for the corresponding problem block.

None
x_u GPU array

New values for the corresponding problem block.

None
x_l GPU array

New values for the corresponding problem block.

None
check_validity bool

Unused by this backend: shapes are always checked (metadata only, no GPU sync).

False
Source code in cupiqp/dense/dense_solver.py
def update(
    self,
    P: Optional[CudaArray] = None,
    c: Optional[CudaArray] = None,
    A: Optional[CudaArray] = None,
    b: Optional[CudaArray] = None,
    G: Optional[CudaArray] = None,
    h_u: Optional[CudaArray] = None,
    h_l: Optional[CudaArray] = None,
    x_u: Optional[CudaArray] = None,
    x_l: Optional[CudaArray] = None,
    check_validity: bool = False
) -> None:
    """Set new numerical data, per problem or shared, then ``solve()``.

    Any argument left as ``None`` keeps its current value. Each argument
    is either **batched** - a leading batch axis, e.g. ``P`` of shape
    ``(B, n, n)`` or ``c`` of shape ``(B, n)``, one entry per problem - or
    **unbatched** - the single-problem shape, e.g. ``(n, n)`` or ``(n,)``,
    in which case the same value is used for every problem. Bound values
    may switch between finite and ``+/-inf``, but only bound sides given
    at ``setup()`` can be updated.

    Parameters
    ----------
    P, c, A, b, G, h_u, h_l, x_u, x_l : GPU array, optional
        New values for the corresponding problem block.
    check_validity : bool, default: False
        Unused by this backend: shapes are always checked (metadata only,
        no GPU sync).
    """
    P, c, A, b, G, h_u, h_l, x_u, x_l = (
        as_warp_array(v, name, self._dtype)
        for name, v in (("P", P), ("c", c), ("A", A), ("b", b), ("G", G),
                        ("h_u", h_u), ("h_l", h_l), ("x_u", x_u), ("x_l", x_l))
    )
    with wp.ScopedStream(self._stream, sync_enter=not self._stream.is_capturing):
        self._update_impl(P, c, A, b, G, h_u, h_l, x_u, x_l, check_validity)

backward

backward(
    grad_x=None,
    grad_y=None,
    grad_z_u=None,
    grad_z_l=None,
    grad_z_bu=None,
    grad_z_bl=None,
    grad_s_u=None,
    grad_s_l=None,
    grad_s_bu=None,
    grad_s_bl=None,
)

Compute gradients of an outer scalar :math:L w.r.t. problem data, given upstream cotangents on the solution variables, by implicit differentiation of the latest solve() through its cached KKT factor.

Cotangents are GPU arrays of shape (B, k) in the solver dtype; missing ones are zero. Returns the backend's Data subclass holding the gradients in user (unscaled) space as Warp arrays, overwritten by the next backward.

Problems of the batch whose solve did not end with status CUPIQP_SOLVED follow settings.gradient_on_failure: "differentiate" (default) differentiates their final iterate like that of any other problem, "zero" returns zero gradients for them. A problem whose KKT factorization or adjoint solve failed always gets zero gradients. solver.grad_valid says which problems' gradients are valid; nothing is read on the host.

Raises:

Type Description
RuntimeError

If no solve() of the current data is cached: before the first solve, after an update() without a new solve, or after a :meth:backward_from_state, which replaces the cached factor.

Source code in cupiqp/solver.py
def backward(self,
         grad_x=None, grad_y=None,
         grad_z_u=None, grad_z_l=None, grad_z_bu=None, grad_z_bl=None,
         grad_s_u=None, grad_s_l=None, grad_s_bu=None, grad_s_bl=None):
    r"""Compute gradients of an outer scalar :math:`L` w.r.t. problem
    data, given upstream cotangents on the solution variables, by
    implicit differentiation of the latest ``solve()`` through its cached
    KKT factor.

    Cotangents are GPU arrays of shape ``(B, k)`` in the solver dtype;
    missing ones are zero. Returns the backend's ``Data`` subclass holding
    the gradients in user (unscaled) space as Warp arrays, overwritten by
    the next backward.

    Problems of the batch whose solve did not end with status
    ``CUPIQP_SOLVED`` follow ``settings.gradient_on_failure``:
    ``"differentiate"`` (default) differentiates their final iterate
    like that of any other problem, ``"zero"`` returns zero gradients for them. A problem
    whose KKT factorization or adjoint solve failed always gets zero
    gradients. ``solver.grad_valid`` says which problems' gradients are
    valid; nothing is read on the host.

    Raises
    ------
    RuntimeError
        If no ``solve()`` of the current data is cached: before the first
        solve, after an ``update()`` without a new solve, or after a
        :meth:`backward_from_state`, which replaces the cached factor.
    """
    with wp.ScopedStream(self._stream, sync_enter=not self._stream.is_capturing):
        self._check_backward_preconditions("backward()")
        if self._factor_revision != self._data_revision:
            raise RuntimeError(
                "backward() differentiates the latest solve() of the current data "
                "through its cached KKT factor, and there is none: call solve() "
                "after update(), and again after backward_from_state()."
            )
        if self._grad_smoothing:
            self._run_smoothing_steps(self._result_scaled, self._result.info.rho,
                                      self._result.info.delta)
            point = self._result_smoothed
        else:
            point = self._result
        return self._backward_impl(
            (grad_x, grad_y, grad_z_u, grad_z_l, grad_z_bu, grad_z_bl,
             grad_s_u, grad_s_l, grad_s_bu, grad_s_bl),
            point, self._result.info.status)

SparseSolver

SparseSolver

SparseSolver(
    dtype: Union[type[float32], type[float64]] = wp.float64,
    stream=None,
)

Bases: SolverBase

GPU solver for general sparse convex quadratic programs that solves a QP - or a whole batch of QPs - of the form

\[ \begin{aligned} \min_{x}\quad & \tfrac{1}{2}\, x^\top P x + c^\top x \\ \text{s.t.}\quad & A x = b, \\ & h_l \le G x \le h_u, \\ & x_l \le x \le x_u, \end{aligned} \]

using the proximal interior-point method with a sparse LDL^T factorization, running entirely on the GPU. It is built for large, structurally sparse P / A / G.

Matrices are CSR triples. P, A and G are given to setup as (indptr, indices, values). The pattern (indptr, indices) may be on the host or the device; it is shared by every problem of the batch and fixed by setup. The values must be on the device; update then takes the values alone.

Batching. Every value array of setup and update is either batched, with a leading batch axis (matrix values (B, nnz), vectors (B, k)), one value per problem, or shared ((nnz,), (k,)), one value for every problem. setup reads B from the batched arrays.

Results are Warp arrays. solver.result.x is a (B, n) warp.array on the GPU; use .numpy() for a host copy, or view it zero-copy from another framework (cupy.asarray(x), torch.from_dlpack(x), jax.dlpack.from_dlpack(x)).

cuPIQP is GPU-only and CSR-only and never converts formats behind your back: host arrays are rejected - move them to the GPU first, and convert other sparse layouts to CSR (e.g. with .tocsr()).

Parameters:

Name Type Description Default
dtype (float64, float32)

Floating-point precision used throughout the solve. wp.float32 is faster and uses less memory but converges to looser tolerances; the default convergence tolerances are chosen to match the dtype.

wp.float64
stream Stream

CUDA stream to run on. By default the solver creates and owns one; pass a warp.Stream to run on it instead (see the stream property).

None

Examples:

import numpy as np
import scipy.sparse as sp
import warp as wp
from cupiqp import SparseSolver, Status

B, n = 8, 4
P = sp.eye(n, format="csr")                 # pattern on the host
values = np.random.uniform(1.0, 2.0, (B, P.nnz))

solver = SparseSolver()
solver.setup(
    P=(P.indptr, P.indices, wp.array(values, dtype=wp.float64)),  # per-problem values on the GPU
    c=wp.zeros(n, dtype=wp.float64),        # shared by all B problems
)
solver.solve()

print(Status(solver.result.info.status.numpy()[0]).name)    # CUPIQP_SOLVED
See Also

DenseSolver: solver for general dense problems.

MultistageSolver: structure-exploiting solver for multistage optimization (e.g. optimal-control) problems.

Notes

setup can be called only once per instance; for a different structure (patterns, blocks, bound sides), create a new SparseSolver. update reuses all GPU allocations. Solver behaviour (tolerances, verbosity, iteration cap, ...) is configured through solver.settings.

Source code in cupiqp/solver.py
def __init__(self, dtype: Union[type[wp.float32], type[wp.float64]] = wp.float64, stream=None):
    wp.init()
    if dtype is not wp.float32 and dtype is not wp.float64:
        raise TypeError(
            f"Solver dtype must be wp.float32 or wp.float64; got {dtype!r}."
        )
    # One CUDA stream per solver instance: every public entry point makes
    # it Warp's current stream, so all kernels, copies, library calls and
    # CUDA graphs of this solver are issued and captured on it. A stream
    # given to the constructor is borrowed (never destroyed here).
    # Entering, the solver stream waits for the caller's current stream,
    # except while it is being captured: that wait would depend on
    # uncaptured work, and the caller already orders the captured stream.
    if stream is not None and not isinstance(stream, wp.Stream):
        raise TypeError(
            "stream must be a warp.Stream or None; got "
            f"{type(stream).__name__}. Order other frameworks' streams "
            "with the solver through cupiqp.torch, or with "
            "warp events on solver.stream."
        )
    self._stream = stream if stream is not None else wp.Stream("cuda")
    self._owns_stream = stream is None
    # All solver data lives on the device of the solver stream.
    self._device = self._stream.device
    self._dtype = dtype
    self._settings = Settings.for_dtype(dtype)
    self._data: Data = None
    self._result = Result()    # store the values of primal, dual and slack variables of current iteration, and other information
    self._step = Variables()   # used to store the step direction of primal and dual variables
    self._res_nr = Variables()  # used to store the non-regularized residuals
    self._res = Variables()  # used to store the regularized residuals
    self._prox_vars = Variables()  # used to store the proximal variables
    self._kkt_system = KKTSystem()
    self._preconditioner = None
    self._setup_done = False

    # Gradient smoothing (qpax-style relaxed-KKT differentiation) state.
    # The mode is frozen at setup(); the relaxation buffers (see setup) are
    # only allocated when it is on. The backward outer products read the
    # linearization point passed in by backward(): ``self._result`` (the true
    # optimum) in standard mode, or ``self._result_smoothed`` (the user-space
    # relaxed iterate) in smoothed mode.
    self._grad_smoothing = None
    self._result_scaled = Variables()
    self._result_smoothed = Variables()

setup

setup(
    P: CsrTriple,
    c: CudaArray,
    A: Optional[CsrTriple] = None,
    b: Optional[CudaArray] = None,
    G: Optional[CsrTriple] = None,
    h_u: Optional[CudaArray] = None,
    h_l: Optional[CudaArray] = None,
    x_u: Optional[CudaArray] = None,
    x_l: Optional[CudaArray] = None,
) -> None

Fix the problem structure, load the data of every problem of the batch, and allocate all GPU memory.

Each matrix is a CSR triple (indptr, indices, values): the pattern, shared by every problem, and its stored values. Every value array - the matrix values and the vectors - is either shared, in the shape of one problem, or batched, with a leading batch axis. The batch size B is read from the batched arrays, which must agree; if every array is shared, B = 1. The patterns and which constraint blocks and bound sides are present become the structure of the solver. Call :meth:solve right away, or change values with :meth:update first. Call setup() once per solver instance.

Parameters:

Name Type Description Default
P CSR triple

Quadratic cost, (n, n) with n = len(indptr) - 1; values (nnz,) or (B, nnz). Must be symmetric positive semidefinite; store the full matrix (both triangles). Required.

required
c GPU array

Linear cost, (n,) or (B, n). Required.

required
A CSR triple and GPU array

Equality constraints A x = b: A with p = len(indptr) - 1 rows and n columns, b of shape (p,) or (B, p). Provide both or neither.

None
b CSR triple and GPU array

Equality constraints A x = b: A with p = len(indptr) - 1 rows and n columns, b of shape (p,) or (B, p). Provide both or neither.

None
G CSR triple and GPU arrays

Inequalities h_l <= G x <= h_u: G with m rows and n columns, bounds (m,) or (B, m). At least one bound is required when G is given; an omitted side is absent for the lifetime of the solver. Use -inf / +inf entries for one-sided rows.

None
h_l CSR triple and GPU arrays

Inequalities h_l <= G x <= h_u: G with m rows and n columns, bounds (m,) or (B, m). At least one bound is required when G is given; an omitted side is absent for the lifetime of the solver. Use -inf / +inf entries for one-sided rows.

None
h_u CSR triple and GPU arrays

Inequalities h_l <= G x <= h_u: G with m rows and n columns, bounds (m,) or (B, m). At least one bound is required when G is given; an omitted side is absent for the lifetime of the solver. Use -inf / +inf entries for one-sided rows.

None
x_l GPU array

Box bounds x_l <= x <= x_u, (n,) or (B, n). An omitted side is absent for the lifetime of the solver.

None
x_u GPU array

Box bounds x_l <= x <= x_u, (n,) or (B, n). An omitted side is absent for the lifetime of the solver.

None

Raises:

Type Description
RuntimeError

If setup() has already been called on this instance.

TypeError

If a matrix is not a CSR triple, an index array is not integer, or a value array is not a GPU array of the solver dtype.

ValueError

If a pattern is invalid, a value array has the wrong shape, or the batched arrays disagree on the batch size.

Notes

indptr and indices are 1-D int32 or int64 arrays on the host (e.g. numpy) or the device; a host pattern is copied to the device once. The values must be device arrays of the solver dtype. Column indices must be sorted within each row, without duplicates. Every pattern fixes which entries are stored: an entry that may become nonzero in any problem must be stored (explicit zeros are fine). The number of stored entries nnz is the length of the values passed to :meth:update. For a scipy csr_matrix M pass (M.indptr, M.indices, values) with values on the device; for a cupyx csr_matrix M, (M.indptr, M.indices, M.data); for a CUDA torch CSR tensor T, (T.crow_indices(), T.col_indices(), T.values()). To set up B equal problems, give at least one value array its batch axis, e.g. cupy.broadcast_to(c, (B, n)).

Source code in cupiqp/sparse/sparse_solver.py
def setup(
    self,
    P: CsrTriple,
    c: CudaArray,
    A: Optional[CsrTriple] = None,
    b: Optional[CudaArray] = None,
    G: Optional[CsrTriple] = None,
    h_u: Optional[CudaArray] = None,
    h_l: Optional[CudaArray] = None,
    x_u: Optional[CudaArray] = None,
    x_l: Optional[CudaArray] = None,
) -> None:
    """Fix the problem structure, load the data of every problem of the
    batch, and allocate all GPU memory.

    Each matrix is a CSR triple ``(indptr, indices, values)``: the
    pattern, shared by every problem, and its stored values. Every value
    array - the matrix values and the vectors - is either **shared**, in
    the shape of one problem, or **batched**, with a leading batch axis.
    The batch size ``B`` is read from the batched arrays, which must
    agree; if every array is shared, ``B = 1``. The patterns and which
    constraint blocks and bound sides are present become the structure
    of the solver. Call :meth:`solve` right away, or change values with
    :meth:`update` first. Call ``setup()`` once per solver instance.

    Parameters
    ----------
    P : CSR triple
        Quadratic cost, ``(n, n)`` with ``n = len(indptr) - 1``; values
        ``(nnz,)`` or ``(B, nnz)``. Must be symmetric positive
        semidefinite; store the full matrix (both triangles). Required.
    c : GPU array
        Linear cost, ``(n,)`` or ``(B, n)``. Required.
    A, b : CSR triple and GPU array, optional
        Equality constraints ``A x = b``: ``A`` with ``p = len(indptr) - 1``
        rows and ``n`` columns, ``b`` of shape ``(p,)`` or ``(B, p)``.
        Provide both or neither.
    G, h_l, h_u : CSR triple and GPU arrays, optional
        Inequalities ``h_l <= G x <= h_u``: ``G`` with ``m`` rows and
        ``n`` columns, bounds ``(m,)`` or ``(B, m)``. At least one bound
        is required when ``G`` is given; an omitted side is absent for the
        lifetime of the solver. Use ``-inf`` / ``+inf`` entries for
        one-sided rows.
    x_l, x_u : GPU array, optional
        Box bounds ``x_l <= x <= x_u``, ``(n,)`` or ``(B, n)``. An omitted
        side is absent for the lifetime of the solver.

    Raises
    ------
    RuntimeError
        If ``setup()`` has already been called on this instance.
    TypeError
        If a matrix is not a CSR triple, an index array is not integer,
        or a value array is not a GPU array of the solver dtype.
    ValueError
        If a pattern is invalid, a value array has the wrong shape, or
        the batched arrays disagree on the batch size.

    Notes
    -----
    ``indptr`` and ``indices`` are 1-D ``int32`` or ``int64`` arrays on the
    host (e.g. numpy) or the device; a host pattern is copied to the device
    once. The values must be device arrays of the solver dtype. Column
    indices must be sorted within each row, without duplicates. Every pattern fixes
    which entries are stored: an entry that may become nonzero in *any*
    problem must be stored (explicit zeros are fine). The number of stored
    entries ``nnz`` is the length of the values passed to :meth:`update`.
    For a scipy ``csr_matrix`` ``M`` pass ``(M.indptr, M.indices, values)``
    with ``values`` on the device; for a cupyx ``csr_matrix`` ``M``,
    ``(M.indptr, M.indices, M.data)``; for a CUDA torch CSR tensor ``T``,
    ``(T.crow_indices(), T.col_indices(), T.values())``. To set up ``B``
    equal problems, give at least one value array its batch axis, e.g.
    ``cupy.broadcast_to(c, (B, n))``.
    """
    if P is None:
        raise TypeError("SparseSolver.setup requires P.")
    P = _check_csr("P", P, self._dtype)
    n = int(P[0].shape[0]) - 1
    A = _check_csr("A", A, self._dtype, cols=n)
    G = _check_csr("G", G, self._dtype, cols=n)
    c, b, h_u, h_l, x_u, x_l = (
        _check_dense_vector(name, v, self._dtype)
        for name, v in (("c", c), ("b", b), ("h_u", h_u), ("h_l", h_l),
                        ("x_u", x_u), ("x_l", x_l))
    )
    with wp.ScopedStream(self._stream, sync_enter=not self._stream.is_capturing):
        self._setup_impl(P, c, A, b, G, h_u, h_l, x_u, x_l)
        if self.settings.enable_grad:
            self._init_grad_data()

solve

solve() -> None

Solve the QP set up by setup().

Runs the proximal interior-point iterations on the GPU and writes the solution (primal x, dual, and slack variables) and per-problem diagnostics to solver.result. The call is asynchronous: it queues the work on solver.stream and returns without waiting for it.

Every result is a Warp array at a fixed address: solver.result.x and the other variables, and the per-problem diagnostics in solver.result.info (status holds one Status code per problem; CUPIQP_SOLVED means converged to tolerance). Work queued on solver.stream afterwards sees the finished solve without any synchronization; .numpy() waits for the solve and copies to the host::

solver.solve()
x = solver.result.x.numpy()
status = solver.result.info.status.numpy()
print(Status(status[0]).name)       # CUPIQP_SOLVED

After setup() you may solve() repeatedly, optionally calling update() in between to change the numerical data. Set solver.settings.verbose = True to print a per-iteration log (this makes the solve wait on the host every iteration).

CUDA graph capture: solve() and update() may be recorded into your own CUDA graph together with your own work, also inside your own wp.capture_while / wp.capture_if; every launch of the graph then runs a complete solve on the data the graph sets::

solver.solve()          # once before capturing: compiles every
                        # kernel and puts the settings on the GPU
with wp.ScopedStream(solver.stream), wp.ScopedCapture() as capture:
    wp.launch(build_qp, ..., outputs=[c_buf])
    solver.update(c=c_buf)
    solver.solve()
    wp.launch(use_solution, ..., inputs=[solver.result.x])
wp.capture_launch(capture.graph, stream=solver.stream)

Capture solver.stream and make it the current stream, as above, so that your work and the solver's are recorded on one stream. Pass the same input arrays to update() at every launch (the graph reads them by address), and keep the settings of the last solve before the capture. Verbose mode and the sparse backend are not supported inside a capture; iterative refinement (iterative_refinement_always_enabled) is, its loop is decided on the device like the interior-point loop.

Source code in cupiqp/solver.py
def solve(self) -> None:
    """Solve the QP set up by ``setup()``.

    Runs the proximal interior-point iterations on the GPU and writes the
    solution (primal ``x``, dual, and slack variables) and per-problem
    diagnostics to ``solver.result``. The call is asynchronous: it queues
    the work on ``solver.stream`` and returns without waiting for it.

    Every result is a Warp array at a fixed address:
    ``solver.result.x`` and the other variables, and the per-problem
    diagnostics in ``solver.result.info`` (``status`` holds one ``Status``
    code per problem; ``CUPIQP_SOLVED`` means converged to tolerance).
    Work queued on ``solver.stream`` afterwards sees the finished solve
    without any synchronization; ``.numpy()`` waits for the solve and
    copies to the host::

        solver.solve()
        x = solver.result.x.numpy()
        status = solver.result.info.status.numpy()
        print(Status(status[0]).name)       # CUPIQP_SOLVED

    After ``setup()`` you may ``solve()`` repeatedly, optionally calling
    ``update()`` in between to change the numerical data. Set
    ``solver.settings.verbose = True`` to print a per-iteration log (this
    makes the solve wait on the host every iteration).

    CUDA graph capture: ``solve()`` and ``update()`` may be recorded into
    your own CUDA graph together with your own work, also inside your own
    ``wp.capture_while`` / ``wp.capture_if``; every launch of the graph
    then runs a complete solve on the data the graph sets::

        solver.solve()          # once before capturing: compiles every
                                # kernel and puts the settings on the GPU
        with wp.ScopedStream(solver.stream), wp.ScopedCapture() as capture:
            wp.launch(build_qp, ..., outputs=[c_buf])
            solver.update(c=c_buf)
            solver.solve()
            wp.launch(use_solution, ..., inputs=[solver.result.x])
        wp.capture_launch(capture.graph, stream=solver.stream)

    Capture ``solver.stream`` and make it the current stream, as above,
    so that your work and the solver's are recorded on one stream. Pass
    the same input arrays to ``update()`` at every launch (the graph
    reads them by address), and keep the settings of the last solve
    before the capture. Verbose mode and the sparse backend are not
    supported inside a capture; iterative refinement
    (``iterative_refinement_always_enabled``) is, its loop is decided on
    the device like the interior-point loop.
    """
    with wp.ScopedStream(self._stream, sync_enter=not self._stream.is_capturing):
        self._solve_impl()

update

update(
    P: Optional[CudaArray] = None,
    c: Optional[CudaArray] = None,
    A: Optional[CudaArray] = None,
    b: Optional[CudaArray] = None,
    G: Optional[CudaArray] = None,
    h_u: Optional[CudaArray] = None,
    h_l: Optional[CudaArray] = None,
    x_u: Optional[CudaArray] = None,
    x_l: Optional[CudaArray] = None,
    check_validity: bool = False,
) -> None

Set new numerical data, per problem or shared, then solve().

The sparsity patterns of P, A and G are fixed by setup(), so this method takes only their nonzero values, not sparse matrices. Any argument left as None keeps its current value.

Parameters:

Name Type Description Default
P GPU array

Nonzero values of the matrix, as a dense GPU array of shape (B, nnz) (one row per problem) or (nnz,) (the same values for every problem). Entry [k, j] is the j-th stored entry of problem k, in the order of the indices given to setup(). solver.data.P.indices / solver.data.P.indptr (likewise for A / G) expose that pattern.

None
A GPU array

Nonzero values of the matrix, as a dense GPU array of shape (B, nnz) (one row per problem) or (nnz,) (the same values for every problem). Entry [k, j] is the j-th stored entry of problem k, in the order of the indices given to setup(). solver.data.P.indices / solver.data.P.indptr (likewise for A / G) expose that pattern.

None
G GPU array

Nonzero values of the matrix, as a dense GPU array of shape (B, nnz) (one row per problem) or (nnz,) (the same values for every problem). Entry [k, j] is the j-th stored entry of problem k, in the order of the indices given to setup(). solver.data.P.indices / solver.data.P.indptr (likewise for A / G) expose that pattern.

None
c GPU array

New vectors, shape (B, k) (one row per problem) or (k,) (shared). Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

None
b GPU array

New vectors, shape (B, k) (one row per problem) or (k,) (shared). Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

None
h_u GPU array

New vectors, shape (B, k) (one row per problem) or (k,) (shared). Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

None
h_l GPU array

New vectors, shape (B, k) (one row per problem) or (k,) (shared). Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

None
x_u GPU array

New vectors, shape (B, k) (one row per problem) or (k,) (shared). Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

None
x_l GPU array

New vectors, shape (B, k) (one row per problem) or (k,) (shared). Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

None
check_validity bool

If True, also validate the vector shapes. The shapes of the matrix values are always checked (this costs no GPU sync).

False

Examples:

solver.setup(P=(indptr, indices, P_values), c=c)   # pattern fixed here
solver.solve()
solver.update(P=P_values_new, c=c_new)            # (B, nnz), (B, n)
solver.solve()
Source code in cupiqp/sparse/sparse_solver.py
def update(
    self,
    P: Optional[CudaArray] = None,
    c: Optional[CudaArray] = None,
    A: Optional[CudaArray] = None,
    b: Optional[CudaArray] = None,
    G: Optional[CudaArray] = None,
    h_u: Optional[CudaArray] = None,
    h_l: Optional[CudaArray] = None,
    x_u: Optional[CudaArray] = None,
    x_l: Optional[CudaArray] = None,
    check_validity: bool = False,
) -> None:
    """Set new numerical data, per problem or shared, then ``solve()``.

    The sparsity patterns of ``P``, ``A`` and ``G`` are fixed by
    ``setup()``, so this method takes only their **nonzero values**, not
    sparse matrices. Any argument left as ``None`` keeps its current
    value.

    Parameters
    ----------
    P, A, G : GPU array, optional
        Nonzero values of the matrix, as a dense GPU array of shape
        ``(B, nnz)`` (one row per problem) or ``(nnz,)`` (the same values
        for every problem). Entry ``[k, j]`` is the ``j``-th stored entry
        of problem ``k``, in the order of the ``indices`` given to
        ``setup()``. ``solver.data.P.indices`` / ``solver.data.P.indptr``
        (likewise for ``A`` / ``G``) expose that pattern.
    c, b, h_u, h_l, x_u, x_l : GPU array, optional
        New vectors, shape ``(B, k)`` (one row per problem) or ``(k,)``
        (shared). Bound values may switch between finite and ``+/-inf``,
        but only bound sides given at ``setup()`` can be updated.
    check_validity : bool, default: False
        If ``True``, also validate the vector shapes. The shapes of the
        matrix values are always checked (this costs no GPU sync).

    Examples
    --------
    ```python
    solver.setup(P=(indptr, indices, P_values), c=c)   # pattern fixed here
    solver.solve()
    solver.update(P=P_values_new, c=c_new)            # (B, nnz), (B, n)
    solver.solve()
    ```
    """
    for name, v in (("P", P), ("A", A), ("G", G)):
        if v is not None and not is_cuda_array(v):
            raise TypeError(
                f"{name} must be a dense GPU array holding the nonzero values, "
                f"of shape (B, nnz) or (nnz,), in the CSR order of the matrix "
                f"passed to setup(); got {type(v).__name__}. The sparsity "
                "pattern is fixed at setup() and cannot be updated."
            )
    P, c, A, b, G, h_u, h_l, x_u, x_l = (
        as_warp_array(v, name, self._dtype)
        for name, v in (("P", P), ("c", c), ("A", A), ("b", b), ("G", G),
                        ("h_u", h_u), ("h_l", h_l), ("x_u", x_u), ("x_l", x_l))
    )
    with wp.ScopedStream(self._stream, sync_enter=not self._stream.is_capturing):
        self._update_impl(P, c, A, b, G, h_u, h_l, x_u, x_l, check_validity)

backward

backward(
    grad_x=None,
    grad_y=None,
    grad_z_u=None,
    grad_z_l=None,
    grad_z_bu=None,
    grad_z_bl=None,
    grad_s_u=None,
    grad_s_l=None,
    grad_s_bu=None,
    grad_s_bl=None,
)

Compute gradients of an outer scalar :math:L w.r.t. problem data, given upstream cotangents on the solution variables, by implicit differentiation of the latest solve() through its cached KKT factor.

Cotangents are GPU arrays of shape (B, k) in the solver dtype; missing ones are zero. Returns the backend's Data subclass holding the gradients in user (unscaled) space as Warp arrays, overwritten by the next backward.

Problems of the batch whose solve did not end with status CUPIQP_SOLVED follow settings.gradient_on_failure: "differentiate" (default) differentiates their final iterate like that of any other problem, "zero" returns zero gradients for them. A problem whose KKT factorization or adjoint solve failed always gets zero gradients. solver.grad_valid says which problems' gradients are valid; nothing is read on the host.

Raises:

Type Description
RuntimeError

If no solve() of the current data is cached: before the first solve, after an update() without a new solve, or after a :meth:backward_from_state, which replaces the cached factor.

Source code in cupiqp/solver.py
def backward(self,
         grad_x=None, grad_y=None,
         grad_z_u=None, grad_z_l=None, grad_z_bu=None, grad_z_bl=None,
         grad_s_u=None, grad_s_l=None, grad_s_bu=None, grad_s_bl=None):
    r"""Compute gradients of an outer scalar :math:`L` w.r.t. problem
    data, given upstream cotangents on the solution variables, by
    implicit differentiation of the latest ``solve()`` through its cached
    KKT factor.

    Cotangents are GPU arrays of shape ``(B, k)`` in the solver dtype;
    missing ones are zero. Returns the backend's ``Data`` subclass holding
    the gradients in user (unscaled) space as Warp arrays, overwritten by
    the next backward.

    Problems of the batch whose solve did not end with status
    ``CUPIQP_SOLVED`` follow ``settings.gradient_on_failure``:
    ``"differentiate"`` (default) differentiates their final iterate
    like that of any other problem, ``"zero"`` returns zero gradients for them. A problem
    whose KKT factorization or adjoint solve failed always gets zero
    gradients. ``solver.grad_valid`` says which problems' gradients are
    valid; nothing is read on the host.

    Raises
    ------
    RuntimeError
        If no ``solve()`` of the current data is cached: before the first
        solve, after an ``update()`` without a new solve, or after a
        :meth:`backward_from_state`, which replaces the cached factor.
    """
    with wp.ScopedStream(self._stream, sync_enter=not self._stream.is_capturing):
        self._check_backward_preconditions("backward()")
        if self._factor_revision != self._data_revision:
            raise RuntimeError(
                "backward() differentiates the latest solve() of the current data "
                "through its cached KKT factor, and there is none: call solve() "
                "after update(), and again after backward_from_state()."
            )
        if self._grad_smoothing:
            self._run_smoothing_steps(self._result_scaled, self._result.info.rho,
                                      self._result.info.delta)
            point = self._result_smoothed
        else:
            point = self._result
        return self._backward_impl(
            (grad_x, grad_y, grad_z_u, grad_z_l, grad_z_bu, grad_z_bl,
             grad_s_u, grad_s_l, grad_s_bu, grad_s_bl),
            point, self._result.info.status)

SparseSolver.setup takes P, A, G as CSR triples (indptr, indices, values), with the pattern on the host or the device and the values on the device; each value array and vector is shared or batched, and the batch size is read from the batched ones. update takes each matrix's values alone, (nnz,) or (B, nnz) (see setup and update above).

MultistageSolver

MultistageSolver

MultistageSolver(
    dtype: Union[type[float32], type[float64]] = wp.float64,
    stream=None,
)

Bases: SolverBase

GPU solver for multistage (block-structured) convex quadratic programs that solves a QP - or a whole batch of QPs - of the form

\[ \begin{aligned} \min_{x}\quad & \tfrac{1}{2}\, x^\top P x + c^\top x \\ \text{s.t.}\quad & A x = b, \\ & h_l \le G x \le h_u, \\ & x_l \le x \le x_u, \end{aligned} \]

using the proximal interior-point method with a block-Cholesky factorization that exploits block-tridiagonal / block-tridiagonal-arrow KKT structure, running entirely on the GPU. It is built for multistage problems such as optimal control (OCPs / MPC), where that structure arises from the stage-by-stage dynamics and costs.

Inputs - plain GPU arrays, block by block. A block-structured matrix is passed as a (diag, offdiag) tuple of GPU arrays (Warp, cupy, CUDA torch, JAX, ... - anything supporting DLPack) and vectors are GPU arrays; values are copied into the solver's Warp buffers and cast to the solver dtype (float64 by default). With N stages of size d (n = N*d):

  • P = (P_diag, P_offdiag): symmetric block-tridiagonal, with diagonal blocks (N, d, d) and lower off-diagonal blocks (N-1, d, d) (block (k+1, k); the upper ones are their transposes).
  • A = (A_diag, A_offdiag) and G = (G_diag, G_offdiag): block lower-bidiagonal with N + 1 block rows of r rows each. Block row k is [.. A_offdiag[k-1] A_diag[k] ..], i.e. both arrays have shape (N, r, d); the last block row holds only A_offdiag[N-1] (acting on the last stage).
  • c, x_l, x_u: (N, d) or flat (n,); b, h_l, h_u: (N + 1, r) or flat ((N + 1) * r,).

Per-stage problems. Any offdiag may be None at setup: the matrix then has no off-diagonal blocks for the lifetime of the solver, and the solver skips all work on them. P_offdiag = None gives a block diagonal P (per-stage costs). A_offdiag = None (or G_offdiag) gives per-stage constraints: A is block diagonal with only N block rows, and b (or h_l / h_u) is (N, r) or flat (N * r,). In MPC, typically only the dynamics (in A) couple consecutive stages, so P_offdiag and G_offdiag can be None.

Only P and c are required. As with the other backends, +/-inf entries in the bounds mark one-sided or free bounds.

Batching. Every array given to setup and update is either batched, with a leading batch axis (e.g. P_diag of shape (B, N, d, d), c of shape (B, N, d)), one value per problem, or shared, in the shapes above, one value for every problem. setup reads B from the batched arrays and fixes the structure shared by all problems. In update, either entry of a (diag, offdiag) tuple may be None to leave it unchanged. Every given array is copied once, in place, into the solver's Warp buffers - no allocation, no GPU-CPU sync. solver.result.x has shape (B, n).

Parameters:

Name Type Description Default
dtype (float64, float32)

Floating-point precision used throughout the solve. wp.float32 is faster and uses less memory but converges to looser tolerances; the default convergence tolerances are chosen to match the dtype.

wp.float64
stream Stream

CUDA stream to run on. By default the solver creates and owns one; pass a warp.Stream to run on it instead (see the stream property).

None

Examples:

import numpy as np
import warp as wp
from cupiqp import MultistageSolver

N, d = 10, 3
P_diag = wp.from_numpy(np.tile(2.0 * np.eye(d), (N, 1, 1)), dtype=wp.float64)
P_offdiag = wp.from_numpy(np.tile(-0.5 * np.eye(d), (N - 1, 1, 1)), dtype=wp.float64)
rng = np.random.default_rng(0)
c = wp.from_numpy(rng.standard_normal((8, N, d)), dtype=wp.float64)  # batched: B = 8
x_l = wp.from_numpy(-np.ones((N, d)), dtype=wp.float64)
x_u = wp.from_numpy(np.ones((N, d)), dtype=wp.float64)

s = MultistageSolver()
s.setup(P=(P_diag, P_offdiag), c=c, x_l=x_l, x_u=x_u)  # P and bounds shared by all 8
s.solve()
See Also

DenseSolver: solver for general dense problems.

SparseSolver: solver for general sparse problems.

Notes

Uses the socu block-solver package, which is installed with cuPIQP. setup can be called only once per instance; for a different structure (block sizes, which blocks and bound sides are present), create a new MultistageSolver. Solver behaviour (tolerances, verbosity, iteration cap, ...) is configured through solver.settings.

Source code in cupiqp/solver.py
def __init__(self, dtype: Union[type[wp.float32], type[wp.float64]] = wp.float64, stream=None):
    wp.init()
    if dtype is not wp.float32 and dtype is not wp.float64:
        raise TypeError(
            f"Solver dtype must be wp.float32 or wp.float64; got {dtype!r}."
        )
    # One CUDA stream per solver instance: every public entry point makes
    # it Warp's current stream, so all kernels, copies, library calls and
    # CUDA graphs of this solver are issued and captured on it. A stream
    # given to the constructor is borrowed (never destroyed here).
    # Entering, the solver stream waits for the caller's current stream,
    # except while it is being captured: that wait would depend on
    # uncaptured work, and the caller already orders the captured stream.
    if stream is not None and not isinstance(stream, wp.Stream):
        raise TypeError(
            "stream must be a warp.Stream or None; got "
            f"{type(stream).__name__}. Order other frameworks' streams "
            "with the solver through cupiqp.torch, or with "
            "warp events on solver.stream."
        )
    self._stream = stream if stream is not None else wp.Stream("cuda")
    self._owns_stream = stream is None
    # All solver data lives on the device of the solver stream.
    self._device = self._stream.device
    self._dtype = dtype
    self._settings = Settings.for_dtype(dtype)
    self._data: Data = None
    self._result = Result()    # store the values of primal, dual and slack variables of current iteration, and other information
    self._step = Variables()   # used to store the step direction of primal and dual variables
    self._res_nr = Variables()  # used to store the non-regularized residuals
    self._res = Variables()  # used to store the regularized residuals
    self._prox_vars = Variables()  # used to store the proximal variables
    self._kkt_system = KKTSystem()
    self._preconditioner = None
    self._setup_done = False

    # Gradient smoothing (qpax-style relaxed-KKT differentiation) state.
    # The mode is frozen at setup(); the relaxation buffers (see setup) are
    # only allocated when it is on. The backward outer products read the
    # linearization point passed in by backward(): ``self._result`` (the true
    # optimum) in standard mode, or ``self._result_smoothed`` (the user-space
    # relaxed iterate) in smoothed mode.
    self._grad_smoothing = None
    self._result_scaled = Variables()
    self._result_smoothed = Variables()

setup

setup(
    P: BlockPair,
    c: CudaArray,
    A: Optional[BlockPair] = None,
    b: Optional[CudaArray] = None,
    G: Optional[BlockPair] = None,
    h_u: Optional[CudaArray] = None,
    h_l: Optional[CudaArray] = None,
    x_u: Optional[CudaArray] = None,
    x_l: Optional[CudaArray] = None,
) -> None

Fix the problem structure, load the data of every problem of the batch, and allocate all GPU memory.

Each array below is given in the shape of one problem, shared by every problem, or with a leading batch axis, batched, one value per problem. The batch size B is read from the batched arrays, which must agree; if every array is shared, B = 1. The block sizes and which constraint blocks and bound sides are present become the structure of the solver. Call :meth:solve right away, or change values with :meth:update first. Call setup() once per solver instance. To set up B equal problems, give at least one array its batch axis, e.g. cupy.broadcast_to(c, (B, N, d)).

Parameters:

Name Type Description Default
P tuple of GPU arrays

(P_diag, P_offdiag): diagonal blocks (N, d, d) and lower off-diagonal blocks (N-1, d, d) of the symmetric block-tridiagonal cost matrix. P_offdiag may be None: P is then block diagonal. Required.

required
c GPU array

Linear cost, (N, d) or flat (N*d,). Required.

required
A optional

Equality constraints A x = b: A = (A_diag, A_offdiag), both (N, r, d), and b of shape (N+1, r) or flat. A_offdiag may be None (per-stage equalities): b is then (N, r) or flat. Provide both or neither.

None
b optional

Equality constraints A x = b: A = (A_diag, A_offdiag), both (N, r, d), and b of shape (N+1, r) or flat. A_offdiag may be None (per-stage equalities): b is then (N, r) or flat. Provide both or neither.

None
G optional

Inequalities h_l <= G x <= h_u: G = (G_diag, G_offdiag), both (N, r, d), and bounds (N+1, r) or flat. G_offdiag may be None (per-stage inequalities): the bounds are then (N, r) or flat. At least one bound is required when G is given; an omitted side is absent for the lifetime of the solver.

None
h_l optional

Inequalities h_l <= G x <= h_u: G = (G_diag, G_offdiag), both (N, r, d), and bounds (N+1, r) or flat. G_offdiag may be None (per-stage inequalities): the bounds are then (N, r) or flat. At least one bound is required when G is given; an omitted side is absent for the lifetime of the solver.

None
h_u optional

Inequalities h_l <= G x <= h_u: G = (G_diag, G_offdiag), both (N, r, d), and bounds (N+1, r) or flat. G_offdiag may be None (per-stage inequalities): the bounds are then (N, r) or flat. At least one bound is required when G is given; an omitted side is absent for the lifetime of the solver.

None
x_l GPU array

Box bounds, (N, d) or flat (N*d,). An omitted side is absent for the lifetime of the solver.

None
x_u GPU array

Box bounds, (N, d) or flat (N*d,). An omitted side is absent for the lifetime of the solver.

None

Raises:

Type Description
RuntimeError

If setup() has already been called on this instance.

TypeError

If a matrix is not a (diag, offdiag) tuple or an input is not a GPU array.

ValueError

If the block shapes are inconsistent or the batched arrays disagree on the batch size.

Source code in cupiqp/multistage/multistage_solver.py
def setup(
    self,
    P: BlockPair,
    c: CudaArray,
    A: Optional[BlockPair] = None,
    b: Optional[CudaArray] = None,
    G: Optional[BlockPair] = None,
    h_u: Optional[CudaArray] = None,
    h_l: Optional[CudaArray] = None,
    x_u: Optional[CudaArray] = None,
    x_l: Optional[CudaArray] = None
) -> None:
    """Fix the problem structure, load the data of every problem of the
    batch, and allocate all GPU memory.

    Each array below is given in the shape of one problem, **shared** by
    every problem, or with a leading batch axis, **batched**, one value
    per problem. The batch size ``B`` is read from the batched arrays,
    which must agree; if every array is shared, ``B = 1``. The block
    sizes and which constraint blocks and bound sides are present become
    the structure of the solver. Call :meth:`solve` right away, or change
    values with :meth:`update` first. Call ``setup()`` once per solver
    instance. To set up ``B`` equal problems, give at least one array its
    batch axis, e.g. ``cupy.broadcast_to(c, (B, N, d))``.

    Parameters
    ----------
    P : tuple of GPU arrays
        ``(P_diag, P_offdiag)``: diagonal blocks ``(N, d, d)`` and lower
        off-diagonal blocks ``(N-1, d, d)`` of the symmetric
        block-tridiagonal cost matrix. ``P_offdiag`` may be ``None``:
        P is then block diagonal. Required.
    c : GPU array
        Linear cost, ``(N, d)`` or flat ``(N*d,)``. Required.
    A, b : optional
        Equality constraints ``A x = b``: ``A = (A_diag, A_offdiag)``,
        both ``(N, r, d)``, and ``b`` of shape ``(N+1, r)`` or flat.
        ``A_offdiag`` may be ``None`` (per-stage equalities): ``b`` is
        then ``(N, r)`` or flat. Provide both or neither.
    G, h_l, h_u : optional
        Inequalities ``h_l <= G x <= h_u``: ``G = (G_diag, G_offdiag)``,
        both ``(N, r, d)``, and bounds ``(N+1, r)`` or flat.
        ``G_offdiag`` may be ``None`` (per-stage inequalities): the bounds
        are then ``(N, r)`` or flat. At least one bound is required when
        ``G`` is given; an omitted side is absent for the lifetime of the
        solver.
    x_l, x_u : GPU array, optional
        Box bounds, ``(N, d)`` or flat ``(N*d,)``. An omitted side is
        absent for the lifetime of the solver.

    Raises
    ------
    RuntimeError
        If ``setup()`` has already been called on this instance.
    TypeError
        If a matrix is not a ``(diag, offdiag)`` tuple or an input is not
        a GPU array.
    ValueError
        If the block shapes are inconsistent or the batched arrays
        disagree on the batch size.
    """
    P, c, A, b, G, h_u, h_l, x_u, x_l = self._inputs_to_warp(P, c, A, b, G, h_u, h_l, x_u, x_l)
    with wp.ScopedStream(self._stream, sync_enter=not self._stream.is_capturing):
        self._setup_impl(P, c, A, b, G, h_u, h_l, x_u, x_l)
        if self.settings.enable_grad:
            self._init_grad_data()

solve

solve() -> None

Solve the QP set up by setup().

Runs the proximal interior-point iterations on the GPU and writes the solution (primal x, dual, and slack variables) and per-problem diagnostics to solver.result. The call is asynchronous: it queues the work on solver.stream and returns without waiting for it.

Every result is a Warp array at a fixed address: solver.result.x and the other variables, and the per-problem diagnostics in solver.result.info (status holds one Status code per problem; CUPIQP_SOLVED means converged to tolerance). Work queued on solver.stream afterwards sees the finished solve without any synchronization; .numpy() waits for the solve and copies to the host::

solver.solve()
x = solver.result.x.numpy()
status = solver.result.info.status.numpy()
print(Status(status[0]).name)       # CUPIQP_SOLVED

After setup() you may solve() repeatedly, optionally calling update() in between to change the numerical data. Set solver.settings.verbose = True to print a per-iteration log (this makes the solve wait on the host every iteration).

CUDA graph capture: solve() and update() may be recorded into your own CUDA graph together with your own work, also inside your own wp.capture_while / wp.capture_if; every launch of the graph then runs a complete solve on the data the graph sets::

solver.solve()          # once before capturing: compiles every
                        # kernel and puts the settings on the GPU
with wp.ScopedStream(solver.stream), wp.ScopedCapture() as capture:
    wp.launch(build_qp, ..., outputs=[c_buf])
    solver.update(c=c_buf)
    solver.solve()
    wp.launch(use_solution, ..., inputs=[solver.result.x])
wp.capture_launch(capture.graph, stream=solver.stream)

Capture solver.stream and make it the current stream, as above, so that your work and the solver's are recorded on one stream. Pass the same input arrays to update() at every launch (the graph reads them by address), and keep the settings of the last solve before the capture. Verbose mode and the sparse backend are not supported inside a capture; iterative refinement (iterative_refinement_always_enabled) is, its loop is decided on the device like the interior-point loop.

Source code in cupiqp/solver.py
def solve(self) -> None:
    """Solve the QP set up by ``setup()``.

    Runs the proximal interior-point iterations on the GPU and writes the
    solution (primal ``x``, dual, and slack variables) and per-problem
    diagnostics to ``solver.result``. The call is asynchronous: it queues
    the work on ``solver.stream`` and returns without waiting for it.

    Every result is a Warp array at a fixed address:
    ``solver.result.x`` and the other variables, and the per-problem
    diagnostics in ``solver.result.info`` (``status`` holds one ``Status``
    code per problem; ``CUPIQP_SOLVED`` means converged to tolerance).
    Work queued on ``solver.stream`` afterwards sees the finished solve
    without any synchronization; ``.numpy()`` waits for the solve and
    copies to the host::

        solver.solve()
        x = solver.result.x.numpy()
        status = solver.result.info.status.numpy()
        print(Status(status[0]).name)       # CUPIQP_SOLVED

    After ``setup()`` you may ``solve()`` repeatedly, optionally calling
    ``update()`` in between to change the numerical data. Set
    ``solver.settings.verbose = True`` to print a per-iteration log (this
    makes the solve wait on the host every iteration).

    CUDA graph capture: ``solve()`` and ``update()`` may be recorded into
    your own CUDA graph together with your own work, also inside your own
    ``wp.capture_while`` / ``wp.capture_if``; every launch of the graph
    then runs a complete solve on the data the graph sets::

        solver.solve()          # once before capturing: compiles every
                                # kernel and puts the settings on the GPU
        with wp.ScopedStream(solver.stream), wp.ScopedCapture() as capture:
            wp.launch(build_qp, ..., outputs=[c_buf])
            solver.update(c=c_buf)
            solver.solve()
            wp.launch(use_solution, ..., inputs=[solver.result.x])
        wp.capture_launch(capture.graph, stream=solver.stream)

    Capture ``solver.stream`` and make it the current stream, as above,
    so that your work and the solver's are recorded on one stream. Pass
    the same input arrays to ``update()`` at every launch (the graph
    reads them by address), and keep the settings of the last solve
    before the capture. Verbose mode and the sparse backend are not
    supported inside a capture; iterative refinement
    (``iterative_refinement_always_enabled``) is, its loop is decided on
    the device like the interior-point loop.
    """
    with wp.ScopedStream(self._stream, sync_enter=not self._stream.is_capturing):
        self._solve_impl()

update

update(
    P: Optional[BlockPair] = None,
    c: Optional[CudaArray] = None,
    A: Optional[BlockPair] = None,
    b: Optional[CudaArray] = None,
    G: Optional[BlockPair] = None,
    h_u: Optional[CudaArray] = None,
    h_l: Optional[CudaArray] = None,
    x_u: Optional[CudaArray] = None,
    x_l: Optional[CudaArray] = None,
    check_validity: bool = False,
) -> None

Set new numerical data, per problem or shared, then solve().

Any argument left as None keeps its current value; so does a None entry of a (diag, offdiag) tuple, e.g. update(P=(None, P_offdiag)) changes only the coupling blocks. Each array is either batched - a leading batch axis, e.g. P_diag of shape (B, N, d, d) or c of shape (B, N, d) / (B, N*d) - or unbatched (the setup() shape), in which case the same value is used for every problem. Each given array is copied once, in place, into the solver's buffers.

Parameters:

Name Type Description Default
P tuple of GPU arrays

New (diag, offdiag) blocks; see :meth:setup for the layout. Off-diagonal blocks absent at setup() cannot be added.

None
A tuple of GPU arrays

New (diag, offdiag) blocks; see :meth:setup for the layout. Off-diagonal blocks absent at setup() cannot be added.

None
G tuple of GPU arrays

New (diag, offdiag) blocks; see :meth:setup for the layout. Off-diagonal blocks absent at setup() cannot be added.

None
c GPU array

New vectors, block or flat layout. Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

None
b GPU array

New vectors, block or flat layout. Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

None
h_u GPU array

New vectors, block or flat layout. Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

None
h_l GPU array

New vectors, block or flat layout. Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

None
x_u GPU array

New vectors, block or flat layout. Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

None
x_l GPU array

New vectors, block or flat layout. Bound values may switch between finite and +/-inf, but only bound sides given at setup() can be updated.

None
check_validity bool

Unused by this backend: shapes are always checked (metadata only, no GPU sync).

False
Source code in cupiqp/multistage/multistage_solver.py
def update(
    self,
    P: Optional[BlockPair] = None,
    c: Optional[CudaArray] = None,
    A: Optional[BlockPair] = None,
    b: Optional[CudaArray] = None,
    G: Optional[BlockPair] = None,
    h_u: Optional[CudaArray] = None,
    h_l: Optional[CudaArray] = None,
    x_u: Optional[CudaArray] = None,
    x_l: Optional[CudaArray] = None,
    check_validity: bool = False
) -> None:
    """Set new numerical data, per problem or shared, then ``solve()``.

    Any argument left as ``None`` keeps its current value; so does a
    ``None`` entry of a ``(diag, offdiag)`` tuple, e.g.
    ``update(P=(None, P_offdiag))`` changes only the coupling blocks.
    Each array is either **batched** - a leading batch axis, e.g.
    ``P_diag`` of shape ``(B, N, d, d)`` or ``c`` of shape ``(B, N, d)`` /
    ``(B, N*d)`` - or **unbatched** (the ``setup()`` shape), in which case
    the same value is used for every problem. Each given array is copied
    once, in place, into the solver's buffers.

    Parameters
    ----------
    P, A, G : tuple of GPU arrays, optional
        New ``(diag, offdiag)`` blocks; see :meth:`setup` for the layout.
        Off-diagonal blocks absent at ``setup()`` cannot be added.
    c, b, h_u, h_l, x_u, x_l : GPU array, optional
        New vectors, block or flat layout. Bound values may switch between
        finite and ``+/-inf``, but only bound sides given at ``setup()``
        can be updated.
    check_validity : bool, default: False
        Unused by this backend: shapes are always checked (metadata only,
        no GPU sync).
    """
    P, c, A, b, G, h_u, h_l, x_u, x_l = self._inputs_to_warp(P, c, A, b, G, h_u, h_l, x_u, x_l)
    with wp.ScopedStream(self._stream, sync_enter=not self._stream.is_capturing):
        self._update_impl(P, c, A, b, G, h_u, h_l, x_u, x_l, check_validity)

backward

backward(
    grad_x=None,
    grad_y=None,
    grad_z_u=None,
    grad_z_l=None,
    grad_z_bu=None,
    grad_z_bl=None,
    grad_s_u=None,
    grad_s_l=None,
    grad_s_bu=None,
    grad_s_bl=None,
)

Compute gradients of an outer scalar :math:L w.r.t. problem data, given upstream cotangents on the solution variables, by implicit differentiation of the latest solve() through its cached KKT factor.

Cotangents are GPU arrays of shape (B, k) in the solver dtype; missing ones are zero. Returns the backend's Data subclass holding the gradients in user (unscaled) space as Warp arrays, overwritten by the next backward.

Problems of the batch whose solve did not end with status CUPIQP_SOLVED follow settings.gradient_on_failure: "differentiate" (default) differentiates their final iterate like that of any other problem, "zero" returns zero gradients for them. A problem whose KKT factorization or adjoint solve failed always gets zero gradients. solver.grad_valid says which problems' gradients are valid; nothing is read on the host.

Raises:

Type Description
RuntimeError

If no solve() of the current data is cached: before the first solve, after an update() without a new solve, or after a :meth:backward_from_state, which replaces the cached factor.

Source code in cupiqp/solver.py
def backward(self,
         grad_x=None, grad_y=None,
         grad_z_u=None, grad_z_l=None, grad_z_bu=None, grad_z_bl=None,
         grad_s_u=None, grad_s_l=None, grad_s_bu=None, grad_s_bl=None):
    r"""Compute gradients of an outer scalar :math:`L` w.r.t. problem
    data, given upstream cotangents on the solution variables, by
    implicit differentiation of the latest ``solve()`` through its cached
    KKT factor.

    Cotangents are GPU arrays of shape ``(B, k)`` in the solver dtype;
    missing ones are zero. Returns the backend's ``Data`` subclass holding
    the gradients in user (unscaled) space as Warp arrays, overwritten by
    the next backward.

    Problems of the batch whose solve did not end with status
    ``CUPIQP_SOLVED`` follow ``settings.gradient_on_failure``:
    ``"differentiate"`` (default) differentiates their final iterate
    like that of any other problem, ``"zero"`` returns zero gradients for them. A problem
    whose KKT factorization or adjoint solve failed always gets zero
    gradients. ``solver.grad_valid`` says which problems' gradients are
    valid; nothing is read on the host.

    Raises
    ------
    RuntimeError
        If no ``solve()`` of the current data is cached: before the first
        solve, after an ``update()`` without a new solve, or after a
        :meth:`backward_from_state`, which replaces the cached factor.
    """
    with wp.ScopedStream(self._stream, sync_enter=not self._stream.is_capturing):
        self._check_backward_preconditions("backward()")
        if self._factor_revision != self._data_revision:
            raise RuntimeError(
                "backward() differentiates the latest solve() of the current data "
                "through its cached KKT factor, and there is none: call solve() "
                "after update(), and again after backward_from_state()."
            )
        if self._grad_smoothing:
            self._run_smoothing_steps(self._result_scaled, self._result.info.rho,
                                      self._result.info.delta)
            point = self._result_smoothed
        else:
            point = self._result
        return self._backward_impl(
            (grad_x, grad_y, grad_z_u, grad_z_l, grad_z_bu, grad_z_bl,
             grad_s_u, grad_s_l, grad_s_bu, grad_s_bl),
            point, self._result.info.status)

MultistageSolver takes plain GPU arrays: each block-structured matrix is a (diag, offdiag) tuple and each vector an array, shared by every problem or with a leading batch axis, at setup and at update (see setup and update above for the block layout).