LP Formulation
Purpose
Section titled “Purpose”This spec presents the complete stage subproblem LP for the Cobre SDDP solver: the objective function with its cost taxonomy, all constraint families, slack/penalty variables, and the Benders cut interface to the future cost function. It uses the parallel blocks formulation by default.
Reading order: SDDP algorithm → system elements → this spec → equipment formulations
For what each physical element represents and its decision variables, see system elements. For variable naming conventions and index sets, see notation conventions.
1. Cost and Penalty Taxonomy
Section titled “1. Cost and Penalty Taxonomy”The objective function includes three categories of penalty/cost terms plus resource costs. This taxonomy aligns with Penalty System. Understanding these categories is essential for setting appropriate parameter values and interpreting solution reports.
1.1 Resource Costs (Actual Operating Expenses)
Section titled “1.1 Resource Costs (Actual Operating Expenses)”Resource costs represent actual generation or contractual expenditures:
| Cost | Symbol | Units | Objective Term |
|---|---|---|---|
| Thermal generation | $/MWh | ||
| Contract dispatch | $/MWh |
Contract prices are positive for imports (cost) and negative for exports (revenue), so a single summation naturally handles both directions. See system elements §8 for the unidirectional contract model.
1.2 Category 1: Recourse Slacks (LP Feasibility)
Section titled “1.2 Category 1: Recourse Slacks (LP Feasibility)”These ensure the SDDP algorithm has relatively complete recourse — every subproblem must be feasible regardless of scenario realization:
| Penalty | Symbol | Units | Purpose |
|---|---|---|---|
| Deficit | $/MWh | Value of unserved energy (piecewise) | |
| Excess generation | $/MWh | Absorb uncontrollable surplus |
1.3 Category 2: Constraint Violation Penalties (Policy Shaping)
Section titled “1.3 Category 2: Constraint Violation Penalties (Policy Shaping)”These provide slack for physical or operational constraints that may be impossible to satisfy under extreme conditions. Their cost must be high enough to affect the value function in earlier stages:
| Penalty | Symbol | Units | Violated Constraint |
|---|---|---|---|
| Storage below minimum | $/hm³ | ||
| Filling target shortfall | $/hm³ | (per-stage filling floor) | |
| Turbined flow minimum | $/(m³/s·h) | ||
| Outflow minimum | $/(m³/s·h) | ||
| Outflow maximum | $/(m³/s·h) | ||
| Generation minimum | $/MWh | ||
| Evaporation violation | $/(m³/s·h) | Evaporation within physical limits | |
| Withdrawal violation | $/(m³/s·h) | Water withdrawal commitment (bidirectional: under/over) |
1.4 Category 3: Regularization Costs (Solution Guidance)
Section titled “1.4 Category 3: Regularization Costs (Solution Guidance)”Small costs that guide the solver toward physically preferred solutions when the LP would otherwise be indifferent. Must be orders of magnitude smaller than any economic cost:
| Cost | Symbol | Units | Purpose |
|---|---|---|---|
| Spillage | $/(m³/s·h) | Prefer turbining over spilling when indifferent | |
| FPHA turbined flow | $/(m³/s·h) | Prevent interior FPHA solutions (FPHA-only) | |
| Diversion | $/(m³/s·h) | Prefer main channel flow | |
| Curtailment | $/MWh | Prioritize using available NCS generation | |
| Exchange | $/MWh | Prevent unnecessary power flows |
1.5 Penalty Priority Ordering
Section titled “1.5 Penalty Priority Ordering”The following ordering must be maintained (from highest to lowest):
with the filling-target penalty pinned below deficit on a separate rung: .
- Storage violation (): Highest penalty — reservoir below dead volume risks dam safety, so it must exceed deficit
- Deficit (): Value of lost load; exceeds any generation cost
- Constraint violations (, , , , ): Exceed typical marginal cost but allow violation when physically necessary
- Resource costs (, ): Market-based or fuel-based
- Regularization (, , , , ): Near-zero
- Filling target (): Pinned below deficit — a commissioning fill schedule is not defended as hard as load serving. Its position relative to the operational-constraint tier (item 3) is left to study calibration.
For the full penalty specification, cascade resolution, and stage-varying overrides, see Penalty System.
1.6 Objective Function Structure
Section titled “1.6 Objective Function Structure”The complete stage objective is:
where each component is summed over blocks with appropriate time weighting:
2. Objective Function
Section titled “2. Objective Function”3. Load Balance Constraint
Section titled “3. Load Balance Constraint”For each bus and block :
Dual variable: (marginal cost of energy at bus , block , in $/MW; divide by for $/MWh — see Variable Units Convention)
For the physical meaning of each element in the balance, see system elements.
4. Hydro Water Balance
Section titled “4. Hydro Water Balance”For each hydro (parallel blocks formulation):
where:
- = incoming storage LP variable, pinned to the previous stage’s value via column bounds (§4a)
- = incremental inflow (from AR model, see PAR(p) inflow model); equivalently from the z-inflow constraint (§5b)
- = water withdrawal target (m³/s), a signed fixed RHS parameter from
water_withdrawal_m3s(not a per-block LP decision variable). schedules a consumptive removal; schedules an inter-basin return/addition that adds water to the reservoir. The realized withdrawal removed from the reservoir is ; see §9 for the slack bounds that prevent the realized flow from flipping sign. Withdrawal is applied at the stage level, outside the per-block summation - = signed net evaporation flow (m³/s): positive values represent net evaporative loss subtracted from storage; negative values represent net rainfall input on the lake surface that adds to storage through the same coefficient (the leading sign on flips the contribution automatically — see penalty system §5)
- = block weight
- = time conversion factor
Dual variable: (water value — captures the marginal value of incoming storage as seen through the hydro balance, but is not used directly as a cut coefficient; the cut coefficient comes from the reduced cost of the pinned incoming-storage column, see §4a and cut management)
4a. Incoming-Storage Pinning
Section titled “4a. Incoming-Storage Pinning”The water balance (§4), FPHA hyperplanes (§6), and generic constraints (§10) all involve the incoming storage value . Rather than embedding as a constant in the RHS of each of these constraints (which would require collecting duals from all of them to compute cut coefficients), Cobre introduces an explicit incoming storage LP variable that every such constraint references, and pins it to the trial value.
For each hydro , the incoming-storage column (storage_in, §4b) is pinned by setting equal lower and upper column bounds:
where:
- = LP variable representing the incoming storage for hydro
- = incoming state value (end-of-stage storage from the previous stage), written into both bounds per scenario via bound patching
The variable then appears as an LP variable (not a constant) in all constraints that depend on incoming storage: the water balance (§4), the FPHA average storage computation (§6), and any generic constraints (§10) that reference incoming storage.
Cut coefficient: the storage cut coefficient is the reduced cost of the pinned storage_in column (unscaled by its prescaler column factor — see §12 and cut management). No fixing-constraint dual is involved.
Why this design: By LP duality, when a column is pinned at its reduced cost equals the sensitivity of the optimal value to the pinned bound — exactly the multiplier the equivalent equality row would have carried (KKT parity). This sensitivity automatically accounts for all downstream effects through water balance, FPHA, and generic constraints, so a single reduced-cost value suffices — no combination of duals from multiple constraint types is needed. This is the same “fishing” technique used by SDDP.jl, realised through column bounds rather than a fixing row, and is analogous to how the AR lags (section 5a) are pinned for inflow history.
Column count: pinned incoming-storage columns, where is the number of operating hydros. There is no corresponding fixing-row block.
4b. LP Column and Row Layout
Section titled “4b. LP Column and Row Layout”LP column layout — state variables (storage, AR lags) first for contiguous
reduced-cost extraction, dispatch variables per block, and θ (future cost) last
for the Benders cuts, assembled with the constraint-row families below.
The stage LP uses a fixed column and row layout that places state variables first, followed by auxiliary and equipment columns. State is pinned by column bounds on the incoming-state columns (§4a, §5a, §5c), and cut coefficients are read as the reduced costs of those columns — so the fixed column order, not a fixed row order, is what enables contiguous coefficient extraction. With hydros, = maximum AR order, = number of anticipated thermals, and :
Column layout:
| Region | Count | Description |
|---|---|---|
storage | Outgoing storage volumes (state) — first | |
inflow_lags | AR lag variables (state) — after storage | |
anticipated_state | Ring-buffer slots for anticipated thermals (state, slot-major plant-minor) | |
z_inflow | Realized inflow (auxiliary, not state) — after the state block | |
storage_in | Incoming storage volumes (auxiliary, for §4a) — after z-inflow | |
theta | Future cost variable — last of the state prefix |
Equipment columns (turbine, spillage, diversion, thermal, anticipated-decision and anticipated-state-out, line flows, deficit, excess, slacks) follow immediately after theta. Two auxiliary blocks are reserved for anticipated thermals: anticipated-decision columns carrying the commitment placed at this stage for delivery stages later, and anticipated-state-out columns used to decouple the post-shift state from the decision-write coefficient — see §5c.
The z_inflow region holds one free column per hydro representing the total realized inflow (m³/s) for each hydro at the current stage. These are auxiliary columns (zero objective cost, unbounded) whose primal values after solving give the realized inflow. They participate in the water balance (§4) and are defined by the z-inflow constraints (§5b).
Row layout (equality-constraint prefix):
Because state is pinned by column bounds (§4a, §5a, §5c) rather than by equality rows, there are no state-fixing rows: the former storage_fixing (), lag_fixing (), and anticipated_state_fixing () row blocks are permanent empty sentinels (zero rows). The equality-constraint prefix therefore begins directly with the z-inflow definitions:
| Region | Count | Description |
|---|---|---|
z_inflow | Realized-inflow definition constraints (§5b) — first equality block |
Equipment rows (water balance, load balance, FPHA, evaporation, outflow bounds, anticipated-fishing and anticipated-state-out equalities, generic constraints, etc.) follow after the z-inflow rows.
Cut coefficients are not read from a contiguous dual slice over a fixing-row prefix. Instead, each incoming-state coordinate is pinned on its own LP column — storage_in for storage, inflow_lags for AR lags, anticipated_state for anticipated-thermal slots — and its cut coefficient is the reduced cost of that column (§4a, cut management). The map from a state coordinate to its pinned column is fixed (state_to_lp_incoming_column), and each of these incoming-state column regions is contiguous, so all storage, inflow-lag, and anticipated-state coefficients are still gathered by reading a few contiguous slices — of the reduced-cost vector rather than the dual vector.
Worked example (, , ): the storage region holds 3 columns, the AR lag region holds 6 (3 hydros × 2 lags), the z-inflow region holds 3, and the incoming-storage region holds 3, so theta is the 16th column. The state count (outgoing storage + AR lags) is .
5. AR Inflow Dynamics
Section titled “5. AR Inflow Dynamics”The incremental inflow is determined by the PAR(p) autoregressive model:
To maintain the Markov property, lagged inflows are promoted to state variables pinned by column bounds — see section 5a below.
See PAR(p) inflow model for the complete PAR(p) model specification.
5a. AR Lag Pinning
Section titled “5a. AR Lag Pinning”The AR dynamics equation (section 5) uses lagged inflows as LP variables. To maintain the Markov property in the SDDP decomposition, each lag variable is pinned to its incoming state value via equal lower and upper column bounds on the inflow_lags column. This binds the lag variables to the known incoming state, and the reduced cost of each pinned column provides the cut coefficient for the corresponding inflow-lag dimension of the Benders cuts (section 11).
For each hydro and each lag :
where:
- = LP variable representing the inflow at lag for hydro
- = incoming state value (inflow observation from stages ago), written into both bounds via bound patching
- = maximum AR order across all hydros (uniform lag storage convention)
Column count: pinned lag columns, where is the number of operating hydros and is the system-wide maximum lag. All hydros store lags regardless of their individual AR order ; hydros with have zero-valued AR coefficients ( for ) in the dynamics equation, but their lag columns are still present and pinned. This uniform layout keeps the inflow_lags columns contiguous, so all lag cut coefficients are read in a single slice of the reduced-cost vector (section 4b). There is no corresponding lag-fixing row block.
Cut coefficient: (marginal value of inflow history at lag for hydro ) is the reduced cost of the pinned inflow_lags column, unscaled by its prescaler column factor (§12) — see cut management.
5b. Realized-Inflow Definition Constraints (z-inflow)
Section titled “5b. Realized-Inflow Definition Constraints (z-inflow)”For each hydro , the LP includes an auxiliary variable representing the total realized inflow (m³/s) at the current stage. These variables are defined by equality constraints that combine the deterministic base, lag contributions, and stochastic noise:
where:
- = LP variable representing the realized inflow for hydro (free column, zero cost)
- = deterministic base (precomputed from seasonal means and AR coefficients — see PAR(p) model §7.4)
- = original-unit AR coefficients (constraint matrix entries, set once at LP construction)
- = LP variables for lagged inflows (state variables, fixed by §5a)
- = noise innovation (patched into the constraint RHS per scenario)
The z-inflow variable then enters the water balance constraint (§4) in place of the raw inflow term , and its primal value after solving gives the realized inflow for reporting and simulation extraction.
The z-inflow columns sit between the AR lag columns and the incoming storage columns in the column layout (section 4b). Because state is pinned by column bounds rather than fixing rows, their constraint rows form the first equality block (section 4b). The RHS is patched per scenario with , where is the effective noise (possibly clamped for inflow non-negativity — see Inflow Non-Negativity). These are not state variables and do not contribute to cut coefficients.
Constraint count: total constraints, where is the number of operating hydros. See section 4b for the row layout.
5c. Anticipated Thermal Dispatch
Section titled “5c. Anticipated Thermal Dispatch”Anticipated thermals (see System Elements §4) introduce a per-plant ring buffer of pending commitments and a per-stage commitment column. The incoming ring-buffer state is pinned by column bounds (like all other state, §4a); two constraint blocks then couple the remaining variables. The layout is engineered so that the reduced cost on slot 0 of the pinned anticipated-state column at stage propagates back to the predecessor’s commitment column via the standard SDDP cut machinery without any decision-side coefficient corrupting the routing.
State pinning (column bounds, one per (slot, plant))
Section titled “State pinning (column bounds, one per (slot, plant))”For each plant and slot , the anticipated-state slot column is pinned by equal column bounds:
The value is the incoming state from the previous stage’s ring-buffer shift (or, at , the seed past_anticipated_commitments[i].values_mw[s]). The slot is pinned by its column bounds alone; no decision-write coefficient appears anywhere on the slot column. The cut subgradient with respect to the incoming-state coordinate, , is the reduced cost of the pinned slot column (§4a). Padding slots are pinned to zero by the same bounds; their reduced cost is zero because the slot carries no information.
Fishing equality (one row per anticipated plant, every stage)
Section titled “Fishing equality (one row per anticipated plant, every stage)”For each plant and every stage , the per-block generation of plant is bound to the matured commitment in slot 0:
where is the block- duration and . The row is active at every study stage; at the slot-0 value comes from the seed values_mw[t] and the LP cannot freely choose the per-block generation. From onward, slot 0 carries a past LP decision delivered via the ring buffer.
State-out equality (one row per active plant)
Section titled “State-out equality (one row per active plant)”For each plant active at stage (i.e., ), one auxiliary row pins the anticipated-state-out column to the decision :
The ring-buffer shift between stages uses — not — as the value written into slot of the next stage’s incoming state. The auxiliary column carries zero objective cost and serves only as the “carrier” that decouples the post-shift state from the decision column. Without this decoupling, the decision column would feed directly into the slot whose pinned reduced cost the next stage’s cut reads back, corrupting the subgradient routing at (slot 0 = slot collision).
Objective contributions
Section titled “Objective contributions”Two terms enter the objective for each anticipated plant:
The first sum is the commitment cost discounted to the delivery stage . The second sum subtracts the standard per-block thermal cost at every delivery stage so the same MWh is not charged twice — once through the matured commitment and once through the per-block dispatch. Anticipated-state columns and anticipated-state-out columns carry zero objective cost; they are pure carriers of state. In run cost output this commitment term is reported as its own anticipated_thermal_cost category (zero when no anticipated plants are present), distinct from the per-block thermal_cost, so the named cost categories sum to the stage’s immediate cost.
Cut subgradient remapping
Section titled “Cut subgradient remapping”When the backward pass returns a subgradient on slot of stage — read as the reduced cost of that pinned slot column — the cut-row builder maps it to a column in the predecessor’s stage problem as follows:
- (slot 0 viewed from the next stage equals slot viewed from this stage): the coefficient targets the predecessor’s commitment column directly. This is the only branch that fires for .
- : the coefficient targets the predecessor’s outgoing-state slot , which holds the same commitment one stage earlier in its journey through the ring buffer.
- (padding): identity remap; the reduced cost is structurally zero so the cut coefficient on the padded slot does not propagate any sensitivity.
The recursion guarantees that, no matter how many stages elapse between commitment and delivery, the marginal cost of a future obligation reaches the original column it should price.
6. Hydro Generation Constraints
Section titled “6. Hydro Generation Constraints”Cobre supports two production models during training, in increasing order of complexity. A third model (linearized head) is available during simulation only — see hydro production models §3. The model can vary by stage or season per hydro.
Constant Productivity Model (for each hydro , block ):
FPHA Model (for each plane , hydro , block ):
where is the average storage during the stage, with being the incoming storage LP variable (§4a) and the end-of-stage storage.
Generation Bounds (per hydro , block ):
Generation bounds are user-defined (not derived from turbined flow). The lower bound is soft (with slack ); the upper bound is hard. See system elements §5.
For details on the FPHA construction and production function model variants, see hydro production models.
7. Outflow Constraints
Section titled “7. Outflow Constraints”Outflow Definition (per hydro , block ):
Outflow Bounds (with slacks for soft enforcement):
8. Variable Bounds and Minimum Constraints
Section titled “8. Variable Bounds and Minimum Constraints”Storage Bounds (per hydro )
Section titled “Storage Bounds (per hydro hhh)”The lower bound (dead volume) is soft — the slack has a very high penalty above deficit cost. The upper bound (reservoir capacity) is hard; excess water is handled by emergency spillage. During the filling period, this lower bound is inactive (storage can be anywhere in ); the per-stage filling floor below takes its place.
Filling floors (for filling hydros, at every stage ):
The minimum end-of-stage storage ramps up at the configured accumulation rate filling_min_rate_m3s (= ) and reaches the dead volume at the last filling stage ; converts the rate over the stage duration into hm³. The slack is priced at , which is pinned below deficit (not the system maximum). This replaces the earlier single terminal constraint at entry_stage_id - 1. See Penalty System §6.
Turbined Flow Bounds (per hydro , block )
Section titled “Turbined Flow Bounds (per hydro hhh, block kkk)”Lower bound is soft; upper bound is hard.
Diversion Flow Bounds (per hydro , block )
Section titled “Diversion Flow Bounds (per hydro hhh, block kkk)”Both bounds are hard. Diversion cost is a regularization term (see §1.4), not a violation penalty.
Pumping Flow Bounds (per station , block )
Section titled “Pumping Flow Bounds (per station jjj, block kkk)”Both bounds are hard.
9. Constraint Violation Penalty Terms
Section titled “9. Constraint Violation Penalty Terms”The per-block constraint violation penalties in the objective (referenced from §2) are:
where is the total stage duration in hours. Withdrawal violation slacks (, ) are stage-level (not per-block) and bidirectional: penalizes under-delivery (the realized withdrawal falls short of the target), and penalizes over-delivery. The withdrawal target is signed (§4), and the slack bounds ensure the realized withdrawal cannot flip sign relative to the target:
- (scheduled removal): (under-delivery slack capped at the target magnitude; floors ), unbounded.
- (scheduled inter-basin return/addition): (over-delivery slack capped at ; caps ), unbounded.
- : both slacks are pinned to zero (presolve-eliminated).
The cap is what keeps the under-delivery slack bounded: were it unbounded, in degenerate cases a run-of-river plant could “un-withdraw” past its target and inject phantom water into the reservoir.
Storage violation penalties ( and ) appear outside the sum because they apply to end-of-stage storage — see §2.
Penalty Resolution
Section titled “Penalty Resolution”The effective penalty for any (entity, stage, penalty_type) tuple follows a three-level cascade:
- Stage-specific override (from Parquet files in
constraints/) - Entity-specific override (from entity registry JSON)
- Global default (from
penalties.json)
For the full resolution semantics and all penalty value definitions, see Penalty System.
10. Generic Constraints
Section titled “10. Generic Constraints”User-defined linear constraints (per constraint ):
where can reference any LP variable using expression syntax:
hydro_storage(id),hydro_turbined(id),hydro_spillage(id)thermal_generation(id),bus_deficit(id), etc.
The coefficients and the RHS may be either literal numeric values or named scalar parameters. A scalar parameter resolves to a single number per stage and can carry one of four kinds:
| Kind | Value semantics |
|---|---|
constant | One value for every stage |
per_stage | Explicit value per stage |
seasonal | One value per season; stages inherit the value from their season |
computed | Value derived from a small expression evaluated at LP-build time (e.g., a fraction of installed capacity) |
Resolution happens once at LP-build time, so the LP coefficients are still numeric at solve time — the parameter mechanism does not introduce LP-variable coupling between constraints. Methodology relevance: it lets the corpus express ramping limits, capacity caps, and operator-imposed quotas that vary by stage or season without authoring a separate constraint per stage. Coefficient and RHS values can therefore be stage-varying constants, not just literal numbers.
Generic constraints can have optional slack variables with configurable penalties.
Row materialization: a constraint bound declared with block_id = None over a block-independent expression — one whose every term references a stock variable (incoming storage , outgoing storage , evaporation outflow collapsed to its stage average, or an anticipated-thermal commitment) — is materialized as a single stage-level row priced by the total stage hours , since per-block rows would be identical. A block_id = None bound on a block-level expression, or any block_id = Some(k) bound, still produces one row per relevant block. This is an LP row-count optimization that is cost- and parity-neutral.
11. Benders Cuts
Section titled “11. Benders Cuts”For each active cut from previous iterations:
where:
- = cut intercept (RHS)
- = coefficient for storage state variable
- = coefficient for AR lag state variable
When anticipated thermals are present, the cut carries one additional coefficient per anticipated-state slot (§5c), read from the same reduced-cost mechanism.
Cuts live in an append-only pool at stable slot indices: every cut ever generated is retained for the lifetime of the run, and only the active subset is baked into each iteration’s stage template. Deactivation toggles a cut row’s bound to a trivially-satisfied sentinel rather than removing the row, so slot indices stay stable and reactivation is exact. See cut management.
For cut coefficient derivation, aggregation, and selection strategies, see cut management.
12. LP Scaling
Section titled “12. LP Scaling”The stage LP is numerically conditioned via a three-step scaling procedure applied once at template construction time. Scaling improves solver convergence by reducing the condition number of the constraint matrix without changing the optimization argmin.
12.1 Cost Scaling (COST_SCALE_FACTOR)
Section titled “12.1 Cost Scaling (COST_SCALE_FACTOR)”All objective coefficients (except the future cost variable ) are divided by a constant factor :
The variable retains its coefficient of 1.0 because the Benders cuts enforce where , so already operates in scaled cost space. The LP objective is , and the total scaled objective equals . All cost-domain outputs (objective values, duals, cost breakdowns) are multiplied by at the reporting boundary to recover original units.
12.2 Column Scaling (Geometric Mean)
Section titled “12.2 Column Scaling (Geometric Mean)”After cost scaling, each column is assigned a scale factor:
where the max and min are taken over nonzero entries in column . Columns with no nonzero entries receive . The transformation replaces:
- Matrix entries:
- Objective coefficients:
- Column bounds: ,
12.3 Row Scaling (Geometric Mean)
Section titled “12.3 Row Scaling (Geometric Mean)”After column scaling, each row is assigned a scale factor using the same geometric-mean formula applied to the already column-scaled matrix:
The transformation replaces:
- Matrix entries:
- Row bounds: ,
Column bounds and objective coefficients are not modified by row scaling.
The combined scaling produces the standard form where and are diagonal scaling matrices.
Cross-References
Section titled “Cross-References”- Notation conventions — index sets, parameters, decision variable naming
- System elements — physical meaning of each element, decision variables, Variable Units Convention
- Penalty System — three-category taxonomy, penalty names, priority ordering, cascade resolution
- SDDP algorithm — iterative structure that solves this LP at each stage
- PAR(p) inflow model — complete AR inflow model specification
- Hydro production models — constant, linearized head, and FPHA model details
- Cut management — dual extraction, cut coefficients, aggregation, and selection
- Equipment formulations — per-equipment constraint derivations, pumping details