Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

LP Formulation

Purpose

This spec presents the complete stage subproblem LP for the Cobre SDDP solver: the objective function with its cost taxonomy, all constraint families, slack/penalty variables, and the Benders cut interface to the future cost function. It uses the parallel blocks formulation by default.

Reading order: SDDP algorithmsystem elementsthis specequipment formulations

For what each physical element represents and its decision variables, see system elements. For variable naming conventions and index sets, see notation conventions.

1. Cost and Penalty Taxonomy

The objective function includes three categories of penalty/cost terms plus resource costs. This taxonomy aligns with Penalty System §2. Understanding these categories is essential for setting appropriate parameter values and interpreting solution reports.

1.1 Resource Costs (Actual Operating Expenses)

Resource costs represent actual generation or contractual expenditures:

CostSymbolUnitsTypical ValuesObjective Term
Thermal generation$/MWh50-500
Contract dispatch$/MWh50-300

Contract prices are positive for imports (cost) and negative for exports (revenue), so a single summation naturally handles both directions. See system elements §8 for the unidirectional contract model.

Note on Pumping: Pumping stations do not have an explicit cost parameter. The cost of pumping is implicitly determined by the marginal cost of energy at the bus where the pump is connected — see equipment formulations for details.

1.2 Category 1: Recourse Slacks (LP Feasibility)

These ensure the SDDP algorithm has relatively complete recourse — every subproblem must be feasible regardless of scenario realization:

PenaltySymbolUnitsTypical ValuesPurpose
Deficit$/MWh1,000-10,000Value of unserved energy (piecewise)
Excess generation$/MWh0.001-0.1Absorb uncontrollable surplus

1.3 Category 2: Constraint Violation Penalties (Policy Shaping)

These provide slack for physical or operational constraints that may be impossible to satisfy under extreme conditions. Their cost must be high enough to affect the value function in earlier stages:

PenaltySymbolUnitsTypical ValuesViolated Constraint
Storage below minimum$/hm³10,000+
Filling target shortfall$/hm³50,000+ (terminal)
Turbined flow minimum$/(m³/s·h)500-1,000
Outflow minimum$/(m³/s·h)500-1,000
Outflow maximum$/(m³/s·h)500-1,000
Generation minimum$/MWh1,000-2,000
Evaporation violation$/(m³/s·h)5,000+Evaporation within physical limits
Withdrawal violation$/(m³/s·h)1,000-5,000Water withdrawal commitment (bidirectional: under/over)

1.4 Category 3: Regularization Costs (Solution Guidance)

Small costs that guide the solver toward physically preferred solutions when the LP would otherwise be indifferent. Must be orders of magnitude smaller than any economic cost:

CostSymbolUnitsTypical ValuesPurpose
Spillage$/(m³/s·h)0.001-0.01Prefer turbining over spilling when indifferent
FPHA turbined flow$/(m³/s·h)0.01-0.1Prevent interior FPHA solutions (FPHA-only)
Diversion$/(m³/s·h)0.01-0.1Prefer main channel flow
Curtailment$/MWh0.001-0.01Prioritize using available NCS generation
Exchange$/MWh0.01-1.0Prevent unnecessary power flows

Note: Regularization costs should be at least 2-3 orders of magnitude smaller than economic costs to avoid distorting the optimal solution.

1.5 Penalty Priority Ordering

The following ordering must be maintained (from highest to lowest):

  1. Filling target (): Highest penalty — filling dead volume is prioritized above all other objectives
  2. Storage violation (): Above deficit — reservoir below dead volume risks dam safety
  3. Deficit (): Value of lost load; exceeds any generation cost
  4. Constraint violations (, , , , ): Exceed typical marginal cost but allow violation when physically necessary
  5. Resource costs (, ): Market-based or fuel-based
  6. Regularization (, , , , ): Near-zero

For the full penalty specification, cascade resolution, and stage-varying overrides, see Penalty System.

Note on Thermal Plants: Thermal bounds (, ) are hard constraints with no slack variables. Thermal dispatch is directly controllable, unlike hydro constraints that may be violated due to exogenous inflow uncertainty.

FPHA validation rule: For each hydro using the fpha production model, must hold. See Penalty System §2.

1.6 Objective Function Structure

The complete stage objective is:

where each component is summed over blocks with appropriate time weighting:

2. Objective Function

Note: Storage violation penalties (, ) are not multiplied by because they apply to end-of-stage storage (hm³), not to per-block flow rates. All other penalty terms are per-block and carry the weighting. Contract prices are positive for imports and negative for exports, so a single sum handles both.

3. Load Balance Constraint

For each bus and block :

Dual variable: (marginal cost of energy at bus , block , in $/MW; divide by for $/MWh — see Variable Units Convention)

For the physical meaning of each element in the balance, see system elements.

4. Hydro Water Balance

For each hydro (parallel blocks formulation):

where:

  • = incoming storage LP variable, fixed to the previous stage’s value via the storage fixing constraint (§4a)
  • = incremental inflow (from AR model, see PAR(p) inflow model); equivalently from the z-inflow constraint (§5b)
  • = water withdrawal rate (m³/s), a fixed RHS parameter from water_withdrawal_m3s (not a per-block LP decision variable). Withdrawal is subtracted from the available water at the stage level, outside the per-block summation
  • = block weight
  • = time conversion factor

Dimensional Consistency (see Variable Units Convention):

  • LHS: [hm³]
  • RHS: [hm³] + [hm³/(m³/s)] × (flow terms [m³/s])
  • The factor converts all flow rates (m³/s) to volumes (hm³) accumulated over the stage
  • Block weights are dimensionless and sum to 1
  • The AR inflow is in m³/s (average rate over the stage)
  • All flow decision variables use rate units (m³/s) — the conversion to volume happens only through in this constraint

Dual variable: (water value — captures the marginal value of incoming storage as seen through the hydro balance, but is not used directly as a cut coefficient; the cut coefficient comes from the storage fixing constraint dual, see §4a and cut management)

4a. Storage Fixing Constraints

The water balance (§4), FPHA hyperplanes (§6), and generic constraints (§10) all involve the incoming storage value . Rather than embedding as a constant in the RHS of each of these constraints (which would require collecting duals from all of them to compute cut coefficients), Cobre introduces an explicit incoming storage LP variable with a fixing constraint:

For each hydro :

where:

  • = LP variable representing the incoming storage for hydro (unbounded)
  • = incoming state value (end-of-stage storage from the previous stage, fixed via RHS patching)

The variable then appears as an LP variable (not a constant) in all constraints that depend on incoming storage: the water balance (§4), the FPHA average storage computation (§6), and any generic constraints (§10) that reference incoming storage.

Dual variable: (marginal value of incoming storage for hydro , used directly as the storage cut coefficient — see cut management)

Why this design: By LP duality, the dual of the fixing constraint captures the total sensitivity , automatically accounting for all downstream effects through water balance, FPHA, and generic constraints. This eliminates the need to combine duals from multiple constraint types to compute the storage cut coefficient — a single dual value suffices. This is the same “fishing constraint” technique used by SDDP.jl and is analogous to how the AR lag fixing constraints (§5a) work for inflow lags.

Constraint count: total constraints, where is the number of operating hydros.

Implementation notes:

  • The incoming storage variable has no bounds — its value is entirely determined by the fixing constraint. The outgoing storage retains its original bounds (§8).
  • The RHS value is patched per scenario during the forward pass (Training Loop SS4.2a) and backward pass, at row (Solver Abstraction SS2.2).
  • The dual extraction for cut coefficients reads from the contiguous top region of the dual vector, where the row-column index symmetry enables a single slice read for all storage cut coefficients (Training Loop SS7.2).
  • The column for is placed after the state prefix (after outgoing storage, lag variables, and realized-inflow variables) — see Solver Abstraction SS2.1 and §4b below. The incoming storage variables are not part of the state vector; they are auxiliary variables whose only purpose is to provide a clean dual for cut coefficient extraction.

4b. LP Column and Row Layout

LP column layout — state variables (storage, AR lags) first for contiguous dual extraction, dispatch variables per block, theta (future cost) last for Benders cuts

The stage LP uses a fixed column and row layout that places state variables first (for contiguous dual extraction), followed by auxiliary and equipment columns. With hydros and = maximum AR order:

Column layout:

RegionIndex rangeCountDescription
storageOutgoing storage volumes (state)
inflow_lagsAR lag variables (state)
z_inflowRealized inflow (auxiliary, not state)
storage_inIncoming storage volumes (auxiliary, for §4a)
thetaFuture cost variable

Equipment columns (turbine, spillage, diversion, thermal, line flows, deficit, excess, slacks) follow immediately after theta.

The z_inflow region holds one free column per hydro representing the total realized inflow (m³/s) for each hydro at the current stage. These are auxiliary columns (zero objective cost, unbounded) whose primal values after solving give the realized inflow. They participate in the water balance (§4) and are defined by the z-inflow constraints (§5b).

Row layout (dual-relevant prefix):

RegionIndex rangeCountDescription
storage_fixingStorage fixing constraints (§4a)
lag_fixingAR lag fixing constraints (§5a)
z_inflowRealized-inflow definition constraints (§5b)

Equipment rows (water balance, load balance, FPHA, evaporation, outflow bounds, etc.) follow after the z-inflow rows.

Worked example (, ):

RegionRangeFormula
storage0..3
inflow_lags3..9
z_inflow9..12
storage_in12..15
theta15
n_state9

5. AR Inflow Dynamics

The incremental inflow is determined by the PAR(p) autoregressive model:

To maintain the Markov property, lagged inflows are promoted to state variables with explicit fixing constraints — see SS5a below.

See PAR(p) inflow model for the complete PAR(p) model specification.

5a. AR Lag Fixing Constraints

The AR dynamics equation (SS5) uses lagged inflows as LP variables. To maintain the Markov property in the SDDP decomposition, each lag variable must be fixed to its incoming state value via an explicit equality constraint. These constraints serve a dual purpose: they bind the lag variables to the known incoming state, and their dual multipliers provide the cut coefficients for the inflow lag dimensions of the Benders cuts (SS11).

For each hydro and each lag (0-based, matching the implementation index convention in Solver Abstraction SS2.2):

where:

  • = LP variable representing the inflow at lag for hydro
  • = incoming state value (inflow observation from stages ago, fixed via RHS patching)
  • = maximum AR order across all hydros (uniform lag storage convention)

Constraint count: total constraints, where is the number of operating hydros and is the system-wide maximum lag. All hydros store lags regardless of their individual AR order ; hydros with have zero-valued AR coefficients ( for ) in the dynamics equation, but their lag fixing constraints are still present. This uniform layout enables contiguous memory access and SIMD-friendly cut coefficient extraction — see Solver Abstraction SS2.2 for the row layout.

Dual variable: (marginal value of inflow history at lag for hydro , used as the cut coefficient for the corresponding inflow lag state variable — see cut management)

Implementation notes:

  • The RHS value is patched per scenario during the forward pass (Training Loop SS4.2a) and backward pass, following the same index formula as the column layout: row for hydro , lag (Solver Abstraction SS2.2).
  • The dual extraction for cut coefficients reads from the contiguous top region of the dual vector, where the row-column index symmetry enables a single slice read (Training Loop SS7.2).

5b. Realized-Inflow Definition Constraints (z-inflow)

For each hydro , the LP includes an auxiliary variable representing the total realized inflow (m³/s) at the current stage. These variables are defined by equality constraints that combine the deterministic base, lag contributions, and stochastic noise:

where:

  • = LP variable representing the realized inflow for hydro (free column, zero cost)
  • = deterministic base (precomputed from seasonal means and AR coefficients — see PAR(p) model §7.4)
  • = original-unit AR coefficients (constraint matrix entries, set once at LP construction)
  • = LP variables for lagged inflows (state variables, fixed by §5a)
  • = noise innovation (patched into the constraint RHS per scenario)

The z-inflow variable then enters the water balance constraint (§4) in place of the raw inflow term , and its primal value after solving gives the realized inflow for reporting and simulation extraction.

Constraint count: total constraints, where is the number of operating hydros. See §4b for the row layout.

Implementation notes:

  • The z-inflow columns are placed at in the column layout — between the AR lag columns and the incoming storage columns (§4b).
  • The z-inflow constraint rows are placed at in the row layout — after the lag fixing rows and before the water balance rows.
  • The RHS is patched per scenario with , where is the effective noise (possibly clamped for inflow non-negativity — see Inflow Non-Negativity).
  • These are not state variables and do not contribute to cut coefficients. Their purpose is twofold: (1) provide the realized inflow for the water balance, and (2) enable direct extraction of per-hydro inflow values from the primal solution.

6. Hydro Generation Constraints

Cobre supports two production models during training, in increasing order of complexity. A third model (linearized head) is available during simulation only — see hydro production models §3. The model can vary by stage or season per hydro — see Input Hydro Extensions §2.

Constant Productivity Model (for each hydro , block ):

FPHA Model (for each plane , hydro , block ):

where is the average storage during the stage, with being the incoming storage LP variable (§4a) and the end-of-stage storage.

Generation Bounds (per hydro , block ):

Generation bounds are user-defined (not derived from turbined flow). The lower bound is soft (with slack ); the upper bound is hard. See system elements §5.

For details on the FPHA construction and production function model variants, see hydro production models.

7. Outflow Constraints

Outflow Definition (per hydro , block ):

Clarification: Outflow represents water released to the downstream channel (affecting tailrace level). It does NOT include:

  • Withdrawal : Consumptive use removed from the system (irrigation, water supply). This is a fixed parameter (not a decision variable) — see §4
  • Diversion : Water bypassed to a separate channel (not affecting main tailrace)

The water balance (§4) accounts for all flows: inflow evaporation withdrawal = storage change. Withdrawal enters as a fixed RHS parameter; bidirectional violation slacks (, ) allow the LP to relax the withdrawal commitment when necessary (see §9).

Outflow Bounds (with slacks for soft enforcement):

8. Variable Bounds and Minimum Constraints

Storage Bounds (per hydro )

The lower bound (dead volume) is soft — the slack has a very high penalty above deficit cost. The upper bound (reservoir capacity) is hard; excess water is handled by emergency spillage. During the filling period, the lower bound is inactive (storage can be anywhere in ).

Filling terminal constraint (at stage entry_stage_id - 1, for filling hydros only):

with the highest penalty in the system (). See Penalty System §7.

Turbined Flow Bounds (per hydro , block )

Lower bound is soft; upper bound is hard.

Diversion Flow Bounds (per hydro , block )

Both bounds are hard. Diversion cost is a regularization term (see §1.4), not a violation penalty.

Pumping Flow Bounds (per station , block )

Both bounds are hard.

9. Constraint Violation Penalty Terms

The per-block constraint violation penalties in the objective (referenced from §2) are:

where is the total stage duration in hours. Withdrawal violation slacks (, ) are stage-level (not per-block) and bidirectional: penalizes under-withdrawal and penalizes over-withdrawal relative to the fixed water_withdrawal_m3s target. Both slacks are active only when the withdrawal target is positive; otherwise they are pinned to zero.

Storage violation penalties ( and ) appear outside the sum because they apply to end-of-stage storage — see §2.

Penalty Resolution

The effective penalty for any (entity, stage, penalty_type) tuple follows a three-level cascade:

  1. Stage-specific override (from Parquet files in constraints/)
  2. Entity-specific override (from entity registry JSON)
  3. Global default (from penalties.json)

For the full resolution semantics and all penalty value definitions, see Penalty System.

10. Generic Constraints

User-defined linear constraints (per constraint ):

where can reference any LP variable using expression syntax:

  • hydro_storage(id), hydro_turbined(id), hydro_spillage(id)
  • thermal_generation(id), bus_deficit(id), etc.

Generic constraints can have optional slack variables with configurable penalties.

11. Benders Cuts

For each active cut from previous iterations:

where:

  • = cut intercept (RHS)
  • = coefficient for storage state variable
  • = coefficient for AR lag state variable

Cuts are pre-allocated and toggled active/inactive via bound changes for warm-starting efficiency.

For cut coefficient derivation, aggregation, and selection strategies, see cut management.

12. LP Scaling

The stage LP is numerically conditioned via a three-step scaling procedure applied once at template construction time. Scaling improves solver convergence by reducing the condition number of the constraint matrix without changing the optimization argmin.

12.1 Cost Scaling (COST_SCALE_FACTOR)

All objective coefficients (except the future cost variable ) are divided by a constant factor :

The variable retains its coefficient of 1.0 because the Benders cuts enforce where , so already operates in scaled cost space. The LP objective is , and the total scaled objective equals . All cost-domain outputs (objective values, duals, cost breakdowns) are multiplied by at the reporting boundary to recover original units.

Impact on cut coefficients: Cut intercepts and coefficients are stored in scaled cost space (divided by ). When evaluating or reporting cut values, the factor must be applied. Duals extracted from the LP are already in scaled cost space and must be multiplied by to obtain original-unit values.

12.2 Column Scaling (Geometric Mean)

After cost scaling, each column is assigned a scale factor:

where the max and min are taken over nonzero entries in column . Columns with no nonzero entries receive . The transformation replaces:

  • Matrix entries:
  • Objective coefficients:
  • Column bounds:

12.3 Row Scaling (Geometric Mean)

After column scaling, each row is assigned a scale factor using the same geometric-mean formula applied to the already column-scaled matrix:

The transformation replaces:

  • Matrix entries:
  • Row bounds: ,

Column bounds and objective coefficients are not modified by row scaling.

The combined scaling produces the standard form where and are diagonal scaling matrices.

Dual unscaling: LP duals are in the scaled problem’s space. To recover original-unit duals: . The per-column and per-row scale factors are stored in the StageTemplate for use during dual extraction and cut coefficient computation.

Cross-References

  • Notation conventions — index sets, parameters, decision variable naming
  • System elements — physical meaning of each element, decision variables, Variable Units Convention
  • Penalty System — three-category taxonomy, penalty names, priority ordering, cascade resolution
  • Input System Entities — entity registries (contracts §6, NCS §7, GNL §4, pumping §5)
  • SDDP algorithm — iterative structure that solves this LP at each stage
  • PAR(p) inflow model — complete AR inflow model specification
  • Hydro production models — constant, linearized head, and FPHA model details
  • Cut management — dual extraction, cut coefficients, aggregation, and selection
  • Internal Structures — Pre-resolved in-memory data model that provides entity data and bounds to the LP builder
  • Solver Abstraction — LP layout convention (SS2) mapping this mathematical formulation to solver column/row indices, with exact index formulas and a worked example
  • Equipment formulations — per-equipment constraint derivations, pumping details