Risk Measures

Purpose

This spec defines the risk-averse SDDP formulation used in Cobre, based on Conditional Value-at-Risk (CVaR). It covers the CVaR definition, the convex combination risk measure, dual representations, the risk-averse subgradient theorem, modified Bellman equation with discount factor, risk-averse cut generation, per-stage risk profiles, and the critical implications for bound validity.

For notation conventions (index sets, parameters, decision variables, dual variables), see Notation Conventions.

Symbol conventions:

• $\alpha$ denotes the CVaR confidence level (matching the alpha field in stages.json). This is the standard convention in the risk measure literature.
• ${\hat{\alpha }}_t(\omega )$ denotes per-scenario cut intercepts within this spec. This corresponds to $\alpha$ in Cut Management §1, renamed here to avoid collision with the CVaR parameter.
• $d$ denotes the discount factor. See Discount Rate.
• $\mu$ denotes the risk-adjusted probability measure (not $q$, which denotes turbined flow).
• $\psi (p,\mu )$ denotes the dual penalty function in the general dual representation.

1 Motivation

Risk-neutral SDDP minimizes expected cost, which can lead to policies that perform poorly in adverse scenarios. Risk-averse SDDP incorporates a coherent risk measure (typically CVaR) to protect against tail risks while maintaining the convexity properties required for valid cut generation.

2 Conditional Value-at-Risk (CVaR)

For a random variable $Z$ representing cost and confidence level $\alpha \in (0,1]$:

$${\text{CVaR}}_{\alpha }(Z)=\underset{\eta \in \mathbb{R}}{\min }\left\{\eta +\frac{1}{\alpha }\mathbb{E}\left[(Z-\eta {)}^{+}\right]\right\}$$

where $(Z-\eta {)}^{+}=\max (0,Z-\eta )$ captures the excess cost above threshold $\eta$.

Interpretation: CVaR${}_{\alpha }$ is the expected cost in the worst $\alpha$-fraction of scenarios.

3 Convex Combination Risk Measure

Cobre uses a convex combination of expectation and CVaR (following the SDDP.jl convention):

$${\rho }^{\lambda ,\alpha }[Z]=(1-\lambda )\mathbb{E}[Z]+\lambda \cdot {\text{CVaR}}_{\alpha }[Z]$$

where:

• $\lambda \in [0,1]$: Risk aversion weight (0 = risk-neutral, 1 = pure CVaR)
• $\alpha \in (0,1]$: CVaR confidence level

This is sometimes called the EAVaR (Expectation + Average Value-at-Risk) risk measure.

4 Dual Representation of Convex Risk Measures

Convex risk measures have a dual representation that is essential for computing risk-averse cuts:

$$\mathbb{F}[Z]=\underset{\mu \in \mathcal{M}(p)}{\sup }{\mathbb{E}}_{\mu }[Z]-\psi (p,\mu )$$

where:

• $\mathcal{M}(p)\subseteq \mathcal{P}$ is a convex subset of the probability simplex
• $\psi (p,\mu )$ is a concave penalty function
• $\mathcal{P}=\{p\ge 0:{\sum }_{\omega }{p}_{\omega }=1\}$

Interpretation: The dual computes the expectation with respect to the worst probability vector $\mu$ within the set $\mathcal{M}$, less a penalty term $\psi (p,\mu )$.

4.1 CVaR Dual Representation

For CVaR${}_{\alpha }$, the dual representation is:

$${\text{CVaR}}_{\alpha }[Z]=\underset{\mu \in {\mathcal{M}}_{\alpha }(p)}{\sup }{\mathbb{E}}_{\mu }[Z]$$

where the risk set ${\mathcal{M}}_{\alpha }(p)$ is:

$${\mathcal{M}}_{\alpha }(p)=\left\{\mu \ge 0:\sum _{\omega }{\mu }_{\omega }=1,{\mu }_{\omega }\le \frac{p_{\omega }}{\alpha }\forall \omega \right\}$$

The penalty $\psi (p,\mu )=0$ for CVaR (no penalty term).

Interpretation: CVaR puts more probability weight on the worst outcomes, with each scenario receiving at most $p_{\omega }/\alpha$ probability mass. For small $\alpha$, only the worst scenarios receive significant weight.

4.2 EAVaR Dual Representation

For the convex combination ${\rho }^{\lambda ,\alpha }[Z]=(1-\lambda )\mathbb{E}[Z]+\lambda \cdot {\text{CVaR}}_{\alpha }[Z]$:

$${\mathcal{M}}^{EAVaR}(p)=\left\{\mu \ge 0:\sum _{\omega }{\mu }_{\omega }=1,{\mu }_{\omega }\le (1-\lambda )p_{\omega }+\frac{\lambda p_{\omega }}{\alpha }\forall \omega \right\}$$

5 Risk-Averse Subgradient Theorem

The key theorem for computing risk-averse cuts:

Theorem (Risk-Averse Subgradient): Let $V(x,\omega )$ be convex with respect to $x$ for all fixed $\omega \in \Omega$, and let $\lambda (\stackrel{,\omega )\; be\; a\; subgradient\; of\; V(x,\omega )\; at\; x=x}{x}$.

If ${\mu }^{\ast }={\text{argmax}}_{\mu \in \mathcal{M}(p)}{\mathbb{E}}_{\mu }[V(\tilde{x},\omega )]-\psi (p,\mu )$, then:

$$\sum _{\omega \in \Omega }{\mu }_{\omega }^{\ast }\cdot \lambda (\tilde{x},\omega )$$

is a subgradient of $\mathbb{F}[V(x,\omega )]$ at $\tilde{x}$.

Application to Cut Generation: In SDDP, the subgradients $\lambda (\tilde{x},\omega )$ are the cut coefficients ${\pi }_t(\omega )$ obtained from LP duals (see Cut Management §2). The risk-averse cut coefficients are computed by replacing the uniform scenario probabilities with risk-adjusted probabilities ${\mu }^{\ast }$:

$${\bar{\pi }}_{t-1,h}=\sum _{\omega \in {\Omega }_t}{\mu }_{\omega }^{\ast }\cdot {\pi }_{t,h}(\omega )$$

where ${\mu }^{\ast }$ is the optimal dual probability vector computed from the scenario costs $\{Q_t(\hat{x},\omega ){\}}_{\omega \in {\Omega }_t}$.

6 Risk-Averse Bellman Equation

The risk-averse value function with discount factor $d$ satisfies:

$$V_t(x_{t-1})={\rho }^{{\lambda }_t,{\alpha }_t}\left[\underset{x_t}{\min }\left\{c_t^{\top }{x}_t+d_{t\to t+1}\cdot V_{t+1}(x_t):(x_t,x_{t-1})\text{ feasible}\right\}\right]$$

This modifies the standard Bellman recursion in two ways:

1. Risk measure replaces expectation: ${\rho }^{{\lambda }_t,{\alpha }_t}[\cdot ]$ replaces $\mathbb{E}[\cdot ]$
2. Discount factor on future cost: $d_{t\to t+1}\cdot V_{t+1}(x_t)$ discounts the cost-to-go (see Discount Rate §2)

Time consistency: The risk measure is applied stage-wise (nested formulation), not to the total cost. This guarantees time consistency, which is essential for the dynamic programming decomposition. Formally: ${\rho }_1[{\rho }_2[\cdots {\rho }_{T-1}[\cdot ]]]$, not $\rho [\text{total cost}]$.

In the LP subproblem at stage $t$, the future cost variable $\theta$ appears in the objective as $d_{t\to t+1}\cdot \theta$ (see Discount Rate §5). Cuts bound $\theta$ (not $d\cdot \theta$), so the discount factor multiplies $\theta$ only in the objective — exactly as in the risk-neutral case.

7 Cut Generation with Risk Measures

For each visited state ${\hat{x}}_{t-1}$, compute the risk-averse cut as follows:

Step 1: Solve subproblems for all realizations $\omega \in {\Omega }_t$:

$$Q_t({\hat{x}}_{t-1},\omega )=\underset{x_t}{\min }\left\{c_t^{\top }{x}_t+d_{t\to t+1}\cdot {\theta }_t:\text{constraints}\right\}$$

Extract dual solutions ${\pi }_t(\omega )$ and compute per-scenario cut coefficients:

• Intercept: ${\hat{\alpha }}_t(\omega )=Q_t({\hat{x}}_{t-1},\omega )-{\pi }_t(\omega {)}^{\top }{\hat{x}}_{t-1}$
• Coefficients: ${\pi }_t(\omega )$ — derived from LP duals (see Cut Management §2)

Step 2: Compute risk-adjusted scenario weights ${\mu }_{\omega }^{\ast }$.

Each scenario $\omega$ has a probability upper bound:

$${\bar{\mu }}_{\omega }=(1-{\lambda }_t)p_{\omega }+\frac{{\lambda }_t{p}_{\omega }}{{\alpha }_t}$$

Since ${\bar{\mu }}_{\omega }>p_{\omega }$ when ${\lambda }_t>0$ and ${\alpha }_t<1$, the total capacity ${\sum }_{\omega }{\bar{\mu }}_{\omega }>1$. The risk-adjusted weights are found by assigning as much weight as possible to the most expensive scenarios:

1. Sort scenarios by cost $Q_{\omega }$ in descending order
2. Walk down the sorted list, assigning ${\mu }_{\omega }^{\ast }={\bar{\mu }}_{\omega }$ (the upper bound) to each scenario
3. When the cumulative weight reaches 1, the current scenario receives the remaining fraction, and all cheaper scenarios get ${\mu }_{\omega }^{\ast }=0$

This is a greedy allocation (continuous knapsack) — it places maximum weight on the worst scenarios and minimum weight on the best.

Special cases:

• Risk-neutral (${\lambda }_t=0$): ${\bar{\mu }}_{\omega }=p_{\omega }$ for all $\omega$, so ${\mu }^{\ast }=p$ and this reduces to the standard aggregation from Cut Management §3.
• Pure CVaR (${\lambda }_t=1$): ${\bar{\mu }}_{\omega }=p_{\omega }/{\alpha }_t$. Only the worst ${\alpha }_t$-fraction of scenarios receive weight; all others get ${\mu }_{\omega }^{\ast }=0$.
• Convex combination ($0<{\lambda }_t<1$): All scenarios receive at least some weight (the $(1-{\lambda }_t)p_{\omega }$ floor), but the worst scenarios receive up to ${\bar{\mu }}_{\omega }$.

Equivalence note: This sorting procedure produces the same ${\mu }^{\ast }$ as solving the dual LP from §4.2, because the LP maximizes a linear objective with per-scenario upper bounds — a structure whose optimal solution is the greedy allocation above.

Step 3: Compute risk-averse cut coefficients using ${\mu }^{\ast }$ (justified by the theorem in §5):

$${\bar{\hat{\alpha }}}_{t-1}=\sum _{\omega \in {\Omega }_t}{\mu }_{\omega }^{\ast }\cdot {\hat{\alpha }}_t(\omega )$$

$${\bar{\pi }}_{t-1}=\sum _{\omega \in {\Omega }_t}{\mu }_{\omega }^{\ast }\cdot {\pi }_t(\omega )$$

Step 4: Add cut to stage $t-1$:

$${\theta }_{t-1}\ge {\bar{\hat{\alpha }}}_{t-1}+{\bar{\pi }}_{t-1}^{\top }{x}_{t-1}$$

Comparison with risk-neutral aggregation: The only difference from Cut Management §3 is that the scenario probabilities $p(\omega )$ are replaced by the risk-adjusted weights ${\mu }_{\omega }^{\ast }$.

8 Per-Stage Risk Profiles

Risk aversion can vary by stage. The risk_measure field in stages.json specifies $({\lambda }_t,{\alpha }_t)$ per stage (see Input Scenarios §1.7):

Example with stage-varying risk:

{
  "stages": [
    {
      "id": 0,
      "risk_measure": { "cvar": { "alpha": 0.95, "lambda": 0.5 } }
    },
    {
      "id": 1,
      "risk_measure": { "cvar": { "alpha": 0.95, "lambda": 0.25 } }
    },
    {
      "id": 2,
      "risk_measure": "expectation"
    }
  ]
}

This allows decreasing risk aversion over the horizon (e.g., higher $\lambda$ for near-term stages, lower for distant stages).

9 Upper Bound with Risk Measures

Monte Carlo simulation cannot directly estimate the upper bound for CVaR problems because:

1. CVaR is computed over the entire distribution, not sample averages
2. The optimal $\eta$ (VaR threshold) changes with the policy

For risk-averse problems, the inner approximation (SIDP) provides deterministic upper bounds that remain valid regardless of the risk measure. See Upper Bound Evaluation §1 for the complete formulation.

10 Lower Bound Validity with Risk Measures

Critical Warning: The lower bound computed during SDDP training is NOT a valid bound for risk-averse problems.

Why the Lower Bound Fails

In risk-neutral SDDP, the lower bound $\underline{z}=V_1(x_0)$ is the optimal value of the first-stage LP, which uses cuts that provide valid outer approximations of the expected future cost. This bound converges to the true optimal expected cost.

In risk-averse SDDP, this property does not hold because:

1. Cuts approximate nested risk measures: Each cut approximates ${\rho }_t[V_{t+1}(x_t,\omega )]$, where the risk measure ${\rho }_t$ depends on the distribution of costs at that stage.

2. The LP optimizes under the wrong distribution: The first-stage LP optimizes ${\rho }_1[V_2]$, but the risk-adjusted distribution used in the cuts was computed for the training states, not for the optimal first-stage decision.

3. Nested risk measures are not time-consistent in expectation: Unlike $\mathbb{E}[\cdot ]$, the nested application of CVaR does not satisfy: $${\rho }_1[{\rho }_2[\dots ]]\ne \rho [\text{total cost}]$$

What the “Lower Bound” Represents

For risk-averse problems, the value $\underline{z}=V_1(x_0)$ computed by SDDP is:

• A convergence indicator: It increases monotonically and plateaus when additional cuts provide no improvement
• NOT a valid lower bound on the true risk-averse optimal cost

Recommendations

Convergence reporting: When risk measures are enabled, convergence reports should label the “lower bound” as “convergence indicator” or explicitly note that it is not a valid bound.

11 References

Philpott, A.B., de Matos, V.L., & Finardi, E.C. (2013). “On solving multistage stochastic programs with coherent risk measures.” Operations Research, 61(4), 957-970. https://doi.org/10.1287/opre.2013.1175

Shapiro, A. (2011). “Analysis of stochastic dual dynamic programming method.” European Journal of Operational Research, 209(1), 63-72.

Cross-References

• Notation Conventions — Symbol definitions, dual variable notation, and sign conventions
• SDDP Algorithm — Bellman recursion and forward/backward pass structure modified by risk measures
• Cut Management — Dual extraction and cut coefficient computation; risk-averse aggregation replaces $p(\omega )$ with ${\mu }_{\omega }^{\ast }$ (§7)
• Stopping Rules — Bound stalling recommended for risk-averse convergence monitoring; simulation-based stopping limitations
• Discount Rate — Discount factor $d$ convention and discounted Bellman equation
• Infinite Horizon — Interaction of risk measures with periodic policy graphs
• Upper Bound Evaluation — SIDP inner approximation for valid upper bounds with CVaR objectives
• Input Scenarios §1.7 — JSON schema for risk_measure field: "expectation" or {"cvar": {"alpha": ..., "lambda": ...}}