CLP Implementation

Status: Not Implemented. This spec documents the planned CLP integration. No CLP code currently exists in the Cobre codebase. HiGHS is the sole solver backend.

Purpose

This spec provides implementation guidance specific to CLP (Coin-OR Linear Programming) integration as the second open-source LP solver reference implementation for Cobre. It complements the Solver Abstraction Layer with CLP-specific patterns for the C API baseline, mutable pointer optimization for bound updates, the C++ wrapper strategy for LP template cloning, retry strategy, basis management, memory footprint, and optimization-tuned configuration. For thread-local workspace management, see Solver Workspaces.

CLP and HiGHS are both first-class reference implementations — the solver abstraction interface is designed and validated against both (see Solver Abstraction, Decision 5).

1. Architecture Alignment

1.1 CLP API Layers

CLP exposes two API layers relevant to Cobre:

The C API is sufficient for the StageLpCache baseline (Clp_loadProblem). The C++ API was anticipated for a cloning optimization (SS5) that is deferred pending profiling (StageLpCache eliminates the rebuild that cloning was designed to accelerate). The bridge between them is Clp_getClpSimplex(model), which returns the underlying ClpSimplex*.

2. Solver Interface Mapping

This section maps each operation from the Solver Abstraction SS4 to the specific CLP API calls.

2.1 Load Model

Clp_loadProblem loads a complete LP in column-major sparse format:

Clp_loadProblem(model, numcols, numrows,
                start, index, value,       // CSR/CSC matrix
                collb, colub, obj,         // column bounds + objective
                rowlb, rowub)              // row bounds

Format note: Clp_loadProblem expects column-major (CSC) format (column starts, row indices, values). The stage LP templates (see Solver Abstraction SS11) store the structural matrix in CSC form — this is CLP’s native internal format (CLP stores the LP matrix as a ClpPackedMatrix in column-ordered form). The CLP implementation passes the template CSC arrays directly to Clp_loadProblem with no format transposition needed.

2.2 Add Rows

Clp_addRows adds multiple rows in a single batch call:

Clp_addRows(model, number, rowLower, rowUpper,
            rowStarts, columns, elements)

The rowStarts/columns/elements arrays follow the standard CSR (row-major) format. This is the natural format for cut addition since cut pool storage is already CSR-friendly (see Binary Formats SS3.4).

2.3 Set Row Bounds

CLP’s mutable pointer access is a key differentiator from HiGHS. The C API functions Clp_rowLower(), Clp_rowUpper(), Clp_columnLower(), Clp_columnUpper(), and Clp_objective() return writable double* pointers directly into the solver’s internal arrays.

Performance implication: Row bound updates can be done as direct memory writes with no function call overhead per element. In SDDP, the ~2,240 RHS updates per solve (incoming storage, AR lag fixing, noise fixing) are the primary use case. The LP layout convention (Solver Abstraction SS2) places state-linking constraints at the top, so these updates target a contiguous prefix of the row arrays — cache-friendly sequential writes.

Alternatively, CLP provides bulk array replacement via Clp_chgRowLower(model, rowLower) and Clp_chgRowUpper(model, rowUpper), which copy entire arrays. This is only useful if the majority of values change — for use cases where only a subset (e.g., ~2,240 of potentially 10,000+ rows in SDDP) changes, direct pointer writes to specific indices are more efficient.

Safety note: These mutable pointers are valid only while the solver model is unchanged (no Clp_addRows, Clp_loadProblem, etc.). After any structural modification, the pointers must be re-obtained. Within the SDDP hot path, structural modifications only occur at stage transitions (load template + add dynamic constraints), after which new pointers are obtained for the bound-update phase.

2.4 Solve

Status interpretation after solve:

Use Clp_secondaryStatus(model) for further disambiguation when Clp_status returns 3 or 4. Use Clp_isAbandoned(model) to distinguish numerical difficulties from clean iteration/time limits.

2.5 Solution Extraction

CLP returns mutable double* pointers to its internal solution arrays:

Since these are pointers into solver-owned memory, the values must be copied out before any subsequent solver operation that could invalidate them (e.g., the next solve).

Dual normalization: CLP uses the convention where the dual row solution Clp_dualRowSolution(model) returns row duals that follow the same sign convention as the solver abstraction’s canonical form (Solver Abstraction SS8). This must be verified empirically for $\le$ vs $\ge$ constraint forms and documented in the implementation.

2.6 Basis Management

CLP uses a compact combined status array for both rows and columns:

unsigned char* status = Clp_statusArray(model);
// Layout: status[0..numcols-1] = column status, status[numcols..numcols+numrows-1] = row status

Status codes (2 bits used from each byte):

Setting basis: Clp_copyinStatus(model, statusArray) copies a status array into the solver. The array must be sized numcols + numrows.

Getting basis: Clp_statusArray(model) returns a mutable pointer to the internal status array. Copy the values before any structural change.

Interaction with LP layout convention: Per Solver Abstraction SS2.3, the structural portion of the basis (rows [0, n_structural)) is position-stable across iterations. When warm-starting after adding cuts, the implementation:

1. Copies the cached basis (truncated or extended to match the new row count)
2. Sets new dynamic constraint rows to status 1 (Basic — slack in basis)
3. Calls Clp_copyinStatus with the assembled array
4. Calls Clp_dual(model, 0) for warm-start solve

2.7 Reset

CLP does not have a direct equivalent of HiGHS’s clearSolver. Two approaches:

• Reconstruct: Clp_deleteModel(model) + Clp_newModel(). Clean but discards all state.
• Re-solve cold: Call Clp_initialDualSolve(model) which performs a full re-factorization from scratch, effectively resetting the solver’s internal state while keeping the LP loaded.

For the retry strategy (SS3), re-solving cold is preferred since it avoids reloading the LP.

2.8 Infeasibility Diagnostics

3. Retry Strategy

The retry strategy follows the behavioral contract defined in Solver Abstraction SS7. CLP-specific retry escalation:

After each successful retry, the implementation restores default settings for the next solve. After all retries are exhausted, return a terminal error with the best partial solution if available (check Clp_primalColumnSolution and Clp_dualRowSolution — they may contain useful values even after a failed solve).

4. Configuration

4.1 SDDP-Tuned Settings

4.2 Scaling Note

CLP supports four scaling modes via Clp_scaling(model, mode):

The scaling strategy interacts with the open point in Solver Abstraction SS3 (single-phase vs two-phase scaling). If Cobre manages its own scaling (applied at the template level), CLP internal scaling should remain off to avoid double-scaling. If Cobre delegates scaling to the solver, CLP’s auto mode (3) is a reasonable default.

5. C++ Wrapper Strategy (Deferred)

CLP’s C++ API offers makeBaseModel()/setToBaseModel() cloning and a setPersistenceFlag micro-optimization. Under StageLpCache (Solver Abstraction SS11.4), the per-stage LP rebuild that cloning was designed to accelerate no longer exists — each stage transition is a single Clp_loadProblem call loading the pre-assembled StageLpCache CSC. The C++ wrapper is deferred until profiling identifies a concrete benefit from setPersistenceFlag (which keeps solver factorization data across loadProblem calls, potentially saving re-factorization cost).

6. Memory Footprint

This is within acceptable bounds for production HPC nodes (384 GB RAM). The cloning overhead (~7 MB per instance) is modest relative to the cut pool memory (~14.3 GB per rank for 60 stages).

7. CLP-Specific Considerations

7.1 Column-Major Format and StageLpCache

Clp_loadProblem expects column-major (CSC) format, which matches the StageLpCache format directly — the StageLpCache stores the complete LP (structural template + active cuts) in CSC form (see Solver Abstraction SS11.4). No format transposition is needed at stage transitions.

Clp_addRows for dynamic constraint addition accepts row-major (CSR) format (row starts, column indices, elements), which matches the cut pool’s CSR-friendly storage layout. Under the StageLpCache strategy, Clp_addRows is used during StageLpCache assembly between iterations but is no longer invoked on the hot-path stage transition.

7.2 Dual Sign Convention

CLP’s dual sign convention must be verified against the canonical sign convention defined in Solver Abstraction SS8. If CLP reports duals with a different sign for $\ge$ constraints, the CLP implementation must negate the appropriate dual values before returning them to the SDDP algorithm. This is a critical correctness requirement — sign errors in duals produce divergent cuts.

7.3 Thread Safety

Each Clp_Simplex* instance is not thread-safe — it must be exclusively owned by one thread. This aligns with the solver abstraction’s thread-safety requirement (Solver Abstraction SS4.2). Each Rayon thread creates its own Clp_Simplex* via Clp_newModel() at initialization and destroys it at shutdown via Clp_deleteModel.

7.4 Persistence Flag

CLP’s setPersistenceFlag(int value) controls memory reuse behavior:

For SDDP where the LP is rebuilt many times with similar (but not identical) sizes, setPersistenceFlag(2) reduces allocation overhead by keeping internal arrays sized for the largest LP seen so far. This is a CLP-specific micro-optimization accessible via the C++ wrapper (SS5).

Cross-References

• Solver Abstraction — Interface contract (SS4), LP layout convention (SS2), cut pool design (SS5), error categories (SS6), retry contract (SS7), dual normalization (SS8), basis storage (SS9), stage templates (SS11)
• Solver Abstraction — Decision 5 (dual-solver validation) that motivated this spec
• HiGHS Implementation — Companion solver implementation for cross-validation
• Solver Workspaces & LP Scaling — Thread-local workspace infrastructure that owns the Clp_Simplex* instance
• LP Formulation — Constraint structure that CLP operates on
• Cut Management — How cuts are generated; this spec handles how they are loaded via Clp_addRows
• Training Loop — Forward/backward pass orchestration driving solver invocations
• Binary Formats — Cut pool CSR layout (SS3.4)
• Hybrid Parallelism — Rayon threading model requiring one CLP instance per thread
• Memory Architecture — NUMA-aware allocation for solver workspaces
• Configuration Reference — Solver configuration parameters