Configuration Reference

Purpose

This spec provides a comprehensive mapping between Cobre configuration options and their effects on LP subproblem construction and solver behavior. Configuration is split across two files: config.json (solver-side settings) and stages.json (temporal structure, per-stage variants, scenario configuration). This spec documents all options, their valid values, LP effects, and links to the defining math or data model spec.

Decision DEC-018 (active): MPI/HPC parameters removed from config.json — all are auto-detected implementation details or contradicted by approved architecture.

For the file layout and config.json schema overview, see Input Directory Structure §2. For the stages.json schema, see Input Scenarios.

1. Configuration File Split

Design rationale: Settings that are inherently per-stage (block mode, risk measure) live in stages.json alongside the stage definitions. Settings that are global solver parameters (training iteration count, cut selection, inflow non-negativity method, upper bound evaluation, scenario source) live in config.json.

1.1 CLI Presentation Settings

The output format (--output-format human|json|json-lines) is a CLI flag, not a configuration parameter. It is intentionally excluded from config.json because:

1. Output format is a presentation concern, not a computation concern. It does not affect solver behavior, random seeds, convergence criteria, or output files on disk. Mixing presentation settings with algorithm configuration would violate separation of concerns.

2. The same case may be run with different output formats depending on the consumer: human for interactive HPC sessions, json for CI/CD pipelines, json-lines for agent monitoring. These are per-invocation choices, not per-case choices.

3. Reproducibility: Two runs with the same config.json and inputs must produce identical results regardless of output format. If output format were in config.json, it would appear in the configuration hash (data_integrity.config_hash in Output Infrastructure §2), creating spurious hash differences.

Similarly, the --quiet and --no-progress flags are CLI-only. They suppress output but do not change what is computed or written to disk. See CLI and Lifecycle §3.1 for the global CLI flags and Structured Output §5 for the output format negotiation.

2. Modeling Options (config.json → modeling)

2.1 Inflow Non-Negativity Treatment

2.2 Penalty Coefficients

Penalty defaults are defined in penalties.json, not config.json. The penalty system uses a three-tier cascade: global defaults (penalties.json) → entity overrides (entity registry files) → stage overrides (constraints/penalty_overrides_*.parquet). See Penalty System for the full specification.

The penalty categories are:

Priority ordering: Filling target > Storage violation > Deficit > Constraint violations > Resource costs > Regularization.

3. Training Options (config.json → training)

3.1 Training Enabled

3.1a Seed and Cut Formulation

3.2 Forward Pass Count

The loader must reject any configuration that omits training.forward_passes. This field has no default because the optimal value depends on the problem structure and the available MPI ranks; an absent value would silently produce incorrect cut pool sizing (see Cut Management Implementation SS1.3).

3.3 Cut Selection

3.4 Stopping Rules

Available rule types:

{
  "training": {
    "stopping_rules": [
      { "type": "iteration_limit", "limit": 100 },
      { "type": "bound_stalling", "iterations": 10, "tolerance": 0.0001 }
    ],
    "stopping_mode": "any"
  }
}

3.5 Solver Retry Configuration

The retry escalation strategy — which solver parameters are varied across attempts — is encapsulated within each solver implementation and is not user-configurable. The external config parameters above control only the stopping conditions for the retry loop. See Solver Interface Trait SS6 and Solver Abstraction SS7 for the encapsulation contract.

4. Upper Bound Evaluation (config.json → upper_bound_evaluation)

{
  "upper_bound_evaluation": {
    "enabled": true,
    "initial_iteration": 10,
    "interval_iterations": 5,
    "lipschitz": {
      "mode": "auto",
      "fallback_value": 10000.0,
      "scale_factor": 1.1
    }
  }
}

5. Horizon and Discount (stages.json → policy_graph)

The horizon mode and discount rate are derived from the policy_graph structure in stages.json. See Input Scenarios §1.2 for the full schema and Extension Points §4 for variant selection.

Discount rate conversion: The annual_discount_rate $r$ is converted to a per-transition discount factor:

$$d_{t\to t+1}=\frac{1}{(1+r{)}^{\Delta t}}$$

where $\Delta t$ is the source stage duration in years. A rate of 0.0 means no discounting ($d=1.0$).

Finite horizon example:

{
  "policy_graph": {
    "type": "finite_horizon",
    "annual_discount_rate": 0.06,
    "transitions": [
      { "source_id": 0, "target_id": 1, "probability": 1.0 },
      { "source_id": 1, "target_id": 2, "probability": 1.0 }
    ]
  }
}

Cyclic (infinite periodic) horizon:

{
  "policy_graph": {
    "type": "cyclic",
    "annual_discount_rate": 0.06,
    "transitions": [
      { "source_id": 0, "target_id": 1, "probability": 1.0 },
      { "source_id": 59, "target_id": 48, "probability": 1.0 }
    ]
  }
}

Validation: For cyclic graphs, the cumulative cycle discount must be $<1$ (see Extension Points §4.3).

6. Per-Stage Options (stages.json → stages[])

These settings vary by stage. Each is configured in the stage object within stages.json. See Input Scenarios §1.4 for the full stage field table.

6.1 Block Mode

Default: "parallel". Can vary by stage (e.g., chronological for near-term, parallel for distant stages).

6.2 Risk Measure

Default: "expectation". Can vary by stage (e.g., higher risk aversion for near-term stages). See Extension Points §2 for variant selection and validation rules.

6.3 Scenario Source and Sampling Scheme

The scenario_source object configures how scenarios are selected during the SDDP forward pass. It lives in config.json under training.scenario_source (for training) and simulation.scenario_source (for simulation). When simulation.scenario_source is absent, it falls back to training.scenario_source. See Scenario Generation §3 for the full abstraction and Extension Points §5 for variant selection.

Each stochastic class (inflow, load, NCS) has its own sampling scheme, configured via per-class sub-objects. The ScenarioSource struct groups the three per-class schemes with a shared seed and optional historical year selection:

6.4 Opening Tree Size

The opening tree is generated once before training and remains fixed throughout. Larger values improve cut quality but increase backward pass cost linearly. Each stage can have a different branching factor — this is required for complete tree mode where near-term stages use num_scenarios: 1 and the final stage uses the full branching count.

Deferred: Monte Carlo backward sampling — sample $n<\text{num\_scenarios}$ noise terms per backward step. See Deferred Features §C.14.

6.5 Production Function

The hydro production model is configured per-hydro (not globally) in hydros.json, with optional per-stage selection in hydro_production_models.json. See Input Hydro Extensions §2.

7. Simulation Options (config.json → simulation)

The simulation section controls the optional post-training simulation phase and its I/O behavior.

7.1 Core Simulation Settings

When simulation.enabled is false or num_scenarios is 0, the simulation phase is skipped entirely.

7.2 Scenario Source (Simulation)

The simulation phase uses simulation.scenario_source when present, otherwise falls back to training.scenario_source. The format is identical to the training scenario source (see §6.3) – a per-class object with inflow, load, and ncs sub-objects, plus optional seed and historical_years.

7.3 I/O Options

{
  "simulation": {
    "enabled": true,
    "num_scenarios": 2000,
    "policy_type": "outer",
    "io_channel_capacity": 128
  }
}

7a. Policy Options (config.json → policy)

Controls policy persistence: checkpoint saving and warm-start loading.

7a.1 Checkpointing

Nested under policy.checkpointing:

{
  "policy": {
    "path": "./policy",
    "mode": "warm_start",
    "validate_compatibility": true,
    "checkpointing": {
      "enabled": true,
      "initial_iteration": 10,
      "interval_iterations": 20,
      "store_basis": true
    }
  }
}

7b. Export Options (config.json → exports)

Controls which outputs are written to the results directory.

{
  "exports": {
    "training": true,
    "cuts": true,
    "states": true,
    "simulation": true,
    "stochastic": false,
    "compression": "zstd"
  }
}

7c. Estimation Options (config.json → estimation)

Controls the PAR(p) model estimation pipeline. When the case provides inflow_history.parquet, Cobre can automatically estimate AR coefficients instead of requiring pre-computed inflow_ar_coefficients.parquet.

{
  "estimation": {
    "max_order": 6,
    "order_selection": "pacf",
    "min_observations_per_season": 30
  }
}

8. Complete Example

config.json

{
  "modeling": {
    "inflow_non_negativity": {
      "method": "penalty",
      "penalty_cost": 1000.0
    }
  },
  "training": {
    "enabled": true,
    "tree_seed": 42,
    "forward_passes": 10,
    "cut_formulation": "single",
    "cut_selection": {
      "enabled": true,
      "method": "level1",
      "threshold": 0,
      "check_frequency": 10
    },
    "stopping_rules": [
      { "type": "iteration_limit", "limit": 100 },
      { "type": "bound_stalling", "iterations": 10, "tolerance": 0.0001 }
    ],
    "stopping_mode": "any",
    "scenario_source": {
      "seed": 42,
      "inflow": { "scheme": "in_sample" },
      "load": { "scheme": "in_sample" },
      "ncs": { "scheme": "in_sample" }
    }
  },
  "upper_bound_evaluation": {
    "enabled": true,
    "initial_iteration": 10,
    "interval_iterations": 5
  },
  "policy": {
    "path": "./policy",
    "mode": "fresh",
    "validate_compatibility": true
  },
  "simulation": {
    "enabled": true,
    "num_scenarios": 2000,
    "policy_type": "outer"
  },
  "exports": {
    "training": true,
    "cuts": true,
    "states": false,
    "simulation": true,
    "stochastic": false,
    "compression": "zstd"
  },
  "estimation": {
    "max_order": 6,
    "order_selection": "pacf",
    "min_observations_per_season": 30
  }
}

stages.json (excerpt — per-stage options)

{
  "policy_graph": {
    "type": "finite_horizon",
    "annual_discount_rate": 0.06,
    "transitions": [
      { "source_id": 0, "target_id": 1, "probability": 1.0 },
      { "source_id": 1, "target_id": 2, "probability": 1.0 }
    ]
  },
  "stages": [
    {
      "id": 0,
      "start_date": "2024-01-01",
      "end_date": "2024-02-01",
      "season_id": 0,
      "block_mode": "chronological",
      "risk_measure": { "cvar": { "alpha": 0.95, "lambda": 0.5 } },
      "num_scenarios": 20,
      "blocks": [
        { "id": 0, "name": "LEVE", "hours": 168 },
        { "id": 1, "name": "MEDIA", "hours": 336 },
        { "id": 2, "name": "PESADA", "hours": 240 }
      ],
      "state_variables": { "storage": true, "inflow_lags": true }
    },
    {
      "id": 1,
      "start_date": "2024-02-01",
      "end_date": "2024-03-01",
      "season_id": 1,
      "block_mode": "parallel",
      "risk_measure": "expectation",
      "num_scenarios": 20,
      "blocks": [{ "id": 0, "name": "UNICO", "hours": 696 }],
      "state_variables": { "storage": true, "inflow_lags": true }
    }
  ]
}

Note: block_mode, risk_measure, num_scenarios, and state_variables vary by stage. scenario_source is in config.json under training.scenario_source (global for the run). policy_graph defines the horizon mode and discount rate. See Input Scenarios for the full schema.

8. Formulation-to-Configuration Mapping

This table maps each mathematical formulation to its configuration source and data files.

9. Variable Correspondence

Cross-References

• SDDP Algorithm — Algorithm overview and policy graph structure
• LP Formulation — Objective function, constraints, dual variables
• Block Formulations — Parallel and chronological block modes
• Hydro Production Models — Constant productivity, FPHA, linearized head
• PAR Inflow Model — PAR(p) definition and fitting
• Inflow Non-Negativity — Non-negativity treatment methods
• Cut Management — Cut generation, aggregation, selection
• Cut Management Implementation — Cut selection implementation, serialization
• Stopping Rules — Convergence and stopping criteria
• Discount Rate — Discounted Bellman equation
• Infinite Horizon — Periodic structure and cyclic convergence
• Upper Bound Evaluation — Inner approximation and gap computation
• Risk Measures — CVaR and risk-averse cut generation
• Penalty System — Three-tier penalty cascade
• Binary Formats — FlatBuffers policy data format
• Input Directory Structure — File layout and config.json schema
• Input Scenarios — stages.json schema, scenario_source, policy_graph
• Extension Points — Variant selection, validation rules, dispatch mechanism
• Solver Abstraction — Solver interface, LP construction, compile-time backend selection
• Scenario Generation — Sampling scheme abstraction, opening tree, external scenarios
• Deferred Features — Planned but not yet implemented features