mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
fix(resilience): fail closed on missing v2 energy seeds + health CRIT on absent inputs (#3363)
* fix(resilience): fail closed on missing v2 energy seeds + health CRIT on absent inputs PR #3289 shipped the v2 energy construct behind RESILIENCE_ENERGY_V2_ENABLED (default false). Audit on 2026-04-24 after the user flagged "AE only moved 1.49 points — we added nuclear credit, we should see more" revealed two safety gaps that made a future flag flip unsafe: 1. scoreEnergyV2 silently fell back to IMPUTE when any of its three required Redis seeds (low-carbon-generation, fossil-electricity-share, power-losses) was null. A future operator flipping the flag with seeds absent would produce fabricated-looking numbers for every country with zero operator signal. 2. api/health.js had those three seed labels in BOTH SEED_META (CRIT on missing) AND ON_DEMAND_KEYS (which demotes CRIT to WARN). The demotion won. Health has been reporting WARNING on a scorer dependency that has been 100% missing since PR #3289 merged — no paging trail existed. Changes: server/worldmonitor/resilience/v1/_dimension-scorers.ts - Add ResilienceConfigurationError with missingKeys[] payload. - scoreEnergy: preflight the three v2 seeds when flag=true. Throw ResilienceConfigurationError listing the specific absent keys. - scoreAllDimensions: wrap per-dimension dispatch in try/catch so a thrown ResilienceConfigurationError routes to the source-failure shape (imputationClass='source-failure', coverage=0) for that ONE dimension — country keeps scoring other dims normally. Log once per country-dimension pair so the gap is audit-traceable. api/health.js - Remove lowCarbonGeneration / fossilElectricityShare / powerLosses from ON_DEMAND_KEYS. They stay in BOOTSTRAP_KEYS + SEED_META. - Replace the transitional comment with a hard "do NOT add these back" note pointing at the scorer's fail-closed gate. tests/resilience-energy-v2.test.mts - New test: flag on + ALL three seeds missing → throws ResilienceConfigurationError naming all three keys. - New test: flag on + only one seed missing → throws naming ONLY the missing key (operator-clarity guard). - New test: flag on + all seeds present → v2 runs normally. - Update the file-level invariant comment to reflect the new fail-closed contract (replacing the prior "degrade gracefully" wording that codified the silent-IMPUTE bug). - Note: fixture's `??` fallbacks coerce null-overrides into real data, so the preflight tests use a direct-reader helper. docs/methodology/country-resilience-index.mdx - New "Fail-closed semantics" paragraph in the v2 Energy section documenting the throw + source-failure + health-CRIT contract. Non-goals (intentional): - This PR does NOT flip RESILIENCE_ENERGY_V2_ENABLED. - This PR does NOT provision seed-bundle-resilience-energy-v2 on Railway. - This PR does NOT touch RESILIENCE_PILLAR_COMBINE_ENABLED. Operational effect post-merge: - /api/health flips from WARNING → CRITICAL on the three v2 seed-meta entries. That is the intended alarm; it reveals that the Railway bundle was never provisioned. - scoreEnergy behavior with flag=false is unchanged (legacy path). - scoreEnergy behavior with flag=true + seeds present is unchanged. - scoreEnergy behavior with flag=true + seeds absent changes from "silently IMPUTE all 217 countries" to "source-failure on the energy dim for every country, visible in widget + API response". Tests: 511/511 resilience-* pass. Biome clean. Lint:md clean. Related plan: docs/plans/2026-04-24-001-fix-resilience-v2-fail-closed-on-missing-seeds-plan.md * docs(resilience): scrub stale ON_DEMAND_KEYS references for v2 energy seeds Greptile P2 on PR #3363: four stale references implied the three v2 energy seeds were still gated as ON_DEMAND_KEYS (WARN-on-missing) even though this PR's api/health.js change removed them (now strict SEED_META = CRIT on missing). Scrubbing each: - api/health.js:196 (BOOTSTRAP_KEYS comment) — was "ON_DEMAND_KEYS until Railway cron provisions; see below." Updated to cite plan 2026-04-24-001 and the strict-SEED_META posture. - api/health.js:398 (SEED_META comment) — was "Listed in ON_DEMAND_KEYS below until Railway cron provisions..." Updated for same reason. - docs/methodology/country-resilience-index.mdx:635 — v2.1 changelog entry said seed keys were ON_DEMAND_KEYS until graduation. Replaced with the fail-closed contract description. - docs/methodology/energy-v2-flag-flip-runbook.md:25 — step 3 said "ON_DEMAND_KEYS graduation" was required at flag-flip time. Rewrote to explain no graduation step is needed because the posture was removed pre-activation. No code change. Tests still 14/14 on the energy-v2 suite, lint:md clean. * fix(docs): escape MDX-unsafe `<=` in energy-v2 runbook to unblock Mintlify Mintlify deploy on PR #3363 failed with `Unexpected character '=' (U+003D) before name` at `docs/methodology/energy-v2-flag-flip-runbook.md`. Two lines had `<=` in plain prose, which MDX tries to parse as a JSX-tag-start. Replaced both with `≤` (U+2264) — and promoted the two existing `>=` on adjacent lines to `≥` for consistency. Prose is clearer and MDX safe. Same pattern as `mdx-unsafe-patterns-in-md` skill; also adjacent to PR #3344's `(<137 countries)` fix.
This commit is contained in:
Elie Habib
@@ -193,8 +193,11 @@ const STANDALONE_KEYS = {
|
||||
recoveryExternalDebt: 'resilience:recovery:external-debt:v1',
|
||||
recoveryImportHhi: 'resilience:recovery:import-hhi:v1',
|
||||
recoveryFuelStocks: 'resilience:recovery:fuel-stocks:v1',
|
||||
// PR 1 v2 energy-construct seeds. ON_DEMAND_KEYS until Railway cron
|
||||
// provisions; see below.
|
||||
// PR 1 v2 energy-construct seeds. STRICT SEED_META (not ON_DEMAND):
|
||||
// plan 2026-04-24-001 removed these from ON_DEMAND_KEYS so /api/health
|
||||
// reports CRIT (not WARN) when they are absent. This is the intended
|
||||
// alarm on the Railway bundle-not-provisioned state. See the ON_DEMAND_KEYS
|
||||
// comment block below for the full rationale.
|
||||
lowCarbonGeneration: 'resilience:low-carbon-generation:v1',
|
||||
fossilElectricityShare: 'resilience:fossil-electricity-share:v1',
|
||||
powerLosses: 'resilience:power-losses:v1',
|
||||
@@ -395,9 +398,9 @@ const SEED_META = {
|
||||
recoveryImportHhi: { key: 'seed-meta:resilience:recovery:import-hhi', maxStaleMin: 86400 }, // monthly cron; 86400min = 60d = 2x interval
|
||||
recoveryFuelStocks: { key: 'seed-meta:resilience:recovery:fuel-stocks', maxStaleMin: 86400 }, // monthly cron; 86400min = 60d = 2x interval
|
||||
// PR 1 v2 energy seeds — weekly cron (8d * 1440 = 11520min = 2x interval).
|
||||
// Listed in ON_DEMAND_KEYS below until Railway cron provisions and
|
||||
// the first clean run lands; after that they graduate to the normal
|
||||
// SEED_META staleness check like the recovery seeds above.
|
||||
// STRICT SEED_META (not ON_DEMAND): plan 2026-04-24-001 made /api/health
|
||||
// CRIT on absent/stale so operators see the Railway-bundle gap before
|
||||
// the flag flips. See the ON_DEMAND_KEYS "do not add back" note below.
|
||||
lowCarbonGeneration: { key: 'seed-meta:resilience:low-carbon-generation', maxStaleMin: 11520 },
|
||||
fossilElectricityShare: { key: 'seed-meta:resilience:fossil-electricity-share', maxStaleMin: 11520 },
|
||||
powerLosses: { key: 'seed-meta:resilience:power-losses', maxStaleMin: 11520 },
|
||||
@@ -423,13 +426,16 @@ const ON_DEMAND_KEYS = new Set([
|
||||
'resilienceRanking', // on-demand RPC cache populated after ranking requests; missing before first Pro use is expected
|
||||
'recoveryFiscalSpace', 'recoveryReserveAdequacy', 'recoveryExternalDebt',
|
||||
'recoveryImportHhi', 'recoveryFuelStocks', // recovery pillar: stub seeders not yet deployed, keys may be absent
|
||||
// PR 1 v2 energy-construct seeds. TRANSITIONAL: the three seeders
|
||||
// ship with their health registry rows in this PR but Railway cron
|
||||
// is provisioned as a follow-up action. Gated as on-demand until
|
||||
// the first clean run lands; graduate out of this set after ~7 days
|
||||
// of successful production cron runs (verify via
|
||||
// `seed-meta:resilience:{low-carbon-generation,fossil-electricity-share,power-losses}.fetchedAt`).
|
||||
'lowCarbonGeneration', 'fossilElectricityShare', 'powerLosses',
|
||||
// NOTE (2026-04-24, plan 2026-04-24-001): the PR 1 v2 energy seeds
|
||||
// (`lowCarbonGeneration`, `fossilElectricityShare`, `powerLosses`)
|
||||
// are INTENTIONALLY NOT listed in ON_DEMAND_KEYS. They stay strict
|
||||
// SEED_META so `/api/health` returns CRIT (not WARN) when they are
|
||||
// absent — which is the alarm a future operator needs before flipping
|
||||
// `RESILIENCE_ENERGY_V2_ENABLED=true`. The scorer fails closed via
|
||||
// ResilienceConfigurationError if the flag flips before the seeds
|
||||
// populate (server/worldmonitor/resilience/v1/_dimension-scorers.ts
|
||||
// #scoreEnergy). Do NOT add these labels back to ON_DEMAND_KEYS
|
||||
// without revisiting that plan.
|
||||
'displacementPrev', // covered by cascade onto current-year displacement; empty most of the year
|
||||
'fxYoy', // TRANSITIONAL (PR #3071): seed-fx-yoy Railway cron deployed manually after merge —
|
||||
// gate as on-demand so a deploy-order race or first-cron-run failure doesn't
|
||||
|
||||
@@ -166,6 +166,8 @@ Retired under v2: `electricityConsumption` (wealth proxy, §3.1 of repair plan),
|
||||
|
||||
**Deferred under v2 (plan §3.1 open-question):** `reserveMarginPct` does not ship in PR 1. IEA electricity-balance coverage is sparse outside OECD+G20; the indicator will likely ship at `tier='unmonitored'` with weight 0.05 if it lands at all. Its Redis key is reserved in `_dimension-scorers.ts`; when a seeder lands, split 0.10 out of `powerLossesPct` and add `reserveMarginPct` at 0.10 in the scorer blend.
|
||||
|
||||
**Fail-closed semantics (plan `2026-04-24-001`).** When `RESILIENCE_ENERGY_V2_ENABLED=true` but any of the three required seeds (`resilience:fossil-electricity-share:v1`, `resilience:low-carbon-generation:v1`, `resilience:power-losses:v1`) is absent from Redis, the scorer throws `ResilienceConfigurationError` at dispatch rather than silently falling back to IMPUTE. The error is caught per-dimension in `scoreAllDimensions` and surfaces as `imputationClass='source-failure'` with `coverage=0`, visible in the widget and the API response. `/api/health` also reports CRIT on the three `seed-meta:resilience:\{low-carbon-generation,fossil-electricity-share,power-losses\}` entries when they are absent or stale. The flag is only safe to flip AFTER `seed-bundle-resilience-energy-v2` is provisioned on Railway and health reports green on all three.
|
||||
|
||||
### Social & Governance Domain (weight 0.19)
|
||||
|
||||
#### Governance
|
||||
@@ -630,7 +632,7 @@ Self-assessed against the standard composite-indicator review axes on a 0-10 sca
|
||||
- **Indicators retired:** `electricityConsumption` (wealth proxy), `gasShare` / `coalShare` / `dependency` (replaced by `importedFossilDependence`), `renewShare` (absorbed into `lowCarbonGenerationShare`).
|
||||
- **Indicators added (live in PR 1):** `importedFossilDependence` (composite: `EG.ELC.FOSL.ZS × max(EG.IMP.CONS.ZS, 0) / 100`, reusing the existing `resilience:static.iea.energyImportDependency.value` for net-imports), `lowCarbonGenerationShare` (`EG.ELC.NUCL.ZS + EG.ELC.RNEW.ZS + EG.ELC.HYRO.ZS` — hydro summed explicitly because WB RNEW excludes hydroelectric), `powerLossesPct` (`EG.ELC.LOSS.ZS`, weight absorbs the deferred `reserveMarginPct`'s 0.10 share). `accessToElectricityPct` moves to the `infrastructure` domain where it acts as a grid-collapse threshold.
|
||||
- **Indicator deferred in PR 1:** `reserveMarginPct` — IEA electricity-balance seeder is out of scope per plan §3.1 open-question. Redis key name + scorer-plumbing slot reserved for the commit that ships the seeder.
|
||||
- **New seeders (weekly):** `seed-low-carbon-generation.mjs` (EG.ELC.NUCL.ZS + EG.ELC.RNEW.ZS + EG.ELC.HYRO.ZS), `seed-fossil-electricity-share.mjs` (EG.ELC.FOSL.ZS), `seed-power-reliability.mjs` (EG.ELC.LOSS.ZS). Bundled by `seed-bundle-resilience-energy-v2.mjs` for a single Railway cron service. Net-energy-imports (`EG.IMP.CONS.ZS`) is NOT a new seeder — it reuses the existing `seed-resilience-static.mjs` path. All three new seed keys are gated as `ON_DEMAND_KEYS` in `api/health.js` until Railway cron provisions and the first clean run lands; graduate out of the set after ~7 days of clean runs.
|
||||
- **New seeders (weekly):** `seed-low-carbon-generation.mjs` (EG.ELC.NUCL.ZS + EG.ELC.RNEW.ZS + EG.ELC.HYRO.ZS), `seed-fossil-electricity-share.mjs` (EG.ELC.FOSL.ZS), `seed-power-reliability.mjs` (EG.ELC.LOSS.ZS). Bundled by `seed-bundle-resilience-energy-v2.mjs` for a single Railway cron service. Net-energy-imports (`EG.IMP.CONS.ZS`) is NOT a new seeder — it reuses the existing `seed-resilience-static.mjs` path. All three seed-meta keys are registered as STRICT `SEED_META` entries in `api/health.js` (NOT `ON_DEMAND_KEYS`) per plan `2026-04-24-001`: `/api/health` reports CRIT on absence/staleness so the Railway-bundle-not-provisioned state is visible before a future flag flip, and the scorer fails closed (`ResilienceConfigurationError` → source-failure) if the flag flips before seeds populate.
|
||||
- **Acceptance gates (plan §6):** Spearman vs baseline >= 0.85; no country moves >15 points; matched-pair gap signs verified; cohort median shifts capped at 10 points; per-indicator effective influence measured via the PR 0 apparatus. Results committed as `docs/snapshots/resilience-ranking-live-post-pr1-<date>.json` and `docs/snapshots/resilience-energy-v2-acceptance-<date>.json` at flag-flip time.
|
||||
|
||||
### v2.2 (April 2026) — PR 3 dead-signal cleanup
|
||||
|
||||
@@ -22,10 +22,15 @@ All must be green before flipping `RESILIENCE_ENERGY_V2_ENABLED=true`:
|
||||
`HEALTHY` with the three keys in the `lowCarbonGeneration`,
|
||||
`fossilElectricityShare`, `powerLosses` slots. If any shows
|
||||
`EMPTY_DATA` or `STALE_SEED`, the flag cannot flip.
|
||||
3. **ON_DEMAND_KEYS graduation.** After ≥ 7 days of clean weekly cron
|
||||
runs, remove the three entries from `api/health.js` `ON_DEMAND_KEYS`
|
||||
set (transitional block added in this PR). Graduating them out of
|
||||
on-demand puts them under the normal CRIT alerting path.
|
||||
3. **Health-registry state (no code change needed at flip time).** Per
|
||||
plan `2026-04-24-001` the three v2 seed labels are already STRICT
|
||||
`SEED_META` entries — NOT in `ON_DEMAND_KEYS`. `/api/health` reports
|
||||
CRIT on absent/stale data from the moment the Railway bundle is
|
||||
provisioned. No "graduation" step is required at flag-flip time;
|
||||
this transitional posture was removed before the flag-flip activation
|
||||
path to keep the scorer and health layers in fail-closed lockstep
|
||||
(scorer throws `ResilienceConfigurationError` → source-failure;
|
||||
health reports CRIT; both surface the gap independently).
|
||||
4. **Acceptance-gate rerun with flag-off.** Baseline Spearman vs the
|
||||
PR 0 freeze must remain 1.0000:
|
||||
```bash
|
||||
@@ -56,11 +61,11 @@ All must be green before flipping `RESILIENCE_ENERGY_V2_ENABLED=true`:
|
||||
```
|
||||
Every gate must be `pass`. If any is `fail`, STOP and debug before
|
||||
proceeding. Check in order:
|
||||
- `gate-1-spearman`: Spearman vs baseline >= 0.85
|
||||
- `gate-2-country-drift`: max country drift <= 15 points
|
||||
- `gate-6-cohort-median`: cohort median shift <= 10 points
|
||||
- `gate-1-spearman`: Spearman vs baseline ≥ 0.85
|
||||
- `gate-2-country-drift`: max country drift ≤ 15 points
|
||||
- `gate-6-cohort-median`: cohort median shift ≤ 10 points
|
||||
- `gate-7-matched-pair`: every matched pair holds expected direction
|
||||
- `gate-9-effective-influence-baseline`: >= 80% Core indicators measurable
|
||||
- `gate-9-effective-influence-baseline`: ≥ 80% Core indicators measurable
|
||||
|
||||
3. **Bump the score-cache prefix.** Add a new commit to this branch
|
||||
bumping `RESILIENCE_SCORE_CACHE_PREFIX` from `v10` to `v11` in
|
||||
|
||||
@@ -339,6 +339,24 @@ function isEnergyV2EnabledLocal(): boolean {
|
||||
return (process.env.RESILIENCE_ENERGY_V2_ENABLED ?? 'false').toLowerCase() === 'true';
|
||||
}
|
||||
|
||||
/**
|
||||
* Thrown by the v2 energy dispatch when `RESILIENCE_ENERGY_V2_ENABLED=true`
|
||||
* but one or more of the required Redis seeds
|
||||
* (`resilience:low-carbon-generation:v1`, `resilience:fossil-electricity-share:v1`,
|
||||
* `resilience:power-losses:v1`) is absent. Fail-closed surfaces the
|
||||
* misconfiguration via the source-failure path instead of silently
|
||||
* producing IMPUTE scores that look computed. See
|
||||
* `docs/plans/2026-04-24-001-fix-resilience-v2-fail-closed-on-missing-seeds-plan.md`.
|
||||
*/
|
||||
export class ResilienceConfigurationError extends Error {
|
||||
readonly missingKeys: readonly string[];
|
||||
constructor(message: string, missingKeys: readonly string[]) {
|
||||
super(message);
|
||||
this.name = 'ResilienceConfigurationError';
|
||||
this.missingKeys = missingKeys;
|
||||
}
|
||||
}
|
||||
|
||||
const COUNTRY_NAME_ALIASES = new Map<string, Set<string>>();
|
||||
for (const [name, iso2] of Object.entries(countryNames as Record<string, string>)) {
|
||||
const code = String(iso2 || '').toUpperCase();
|
||||
@@ -1309,9 +1327,36 @@ export async function scoreEnergy(
|
||||
countryCode: string,
|
||||
reader: ResilienceSeedReader = defaultSeedReader,
|
||||
): Promise<ResilienceDimensionScore> {
|
||||
return isEnergyV2EnabledLocal()
|
||||
? scoreEnergyV2(countryCode, reader)
|
||||
: scoreEnergyLegacy(countryCode, reader);
|
||||
if (!isEnergyV2EnabledLocal()) {
|
||||
return scoreEnergyLegacy(countryCode, reader);
|
||||
}
|
||||
|
||||
// Flag is ON — preflight the required seeds before routing to v2.
|
||||
// A null from any of these would let scoreEnergyV2 score every country
|
||||
// via the IMPUTE fallback with no signal to the operator (weightedBlend
|
||||
// silently collapses null indicators to the imputation path). Fail-closed:
|
||||
// throw ResilienceConfigurationError, caught at scoreAllDimensions and
|
||||
// surfaced as imputationClass='source-failure' on the energy dimension.
|
||||
// See docs/plans/2026-04-24-001-fix-resilience-v2-fail-closed-on-missing-seeds-plan.md.
|
||||
const [fossilShareRaw, lowCarbonRaw, powerLossesRaw] = await Promise.all([
|
||||
reader(RESILIENCE_FOSSIL_ELEC_SHARE_KEY),
|
||||
reader(RESILIENCE_LOW_CARBON_GEN_KEY),
|
||||
reader(RESILIENCE_POWER_LOSSES_KEY),
|
||||
]);
|
||||
const missing: string[] = [];
|
||||
if (fossilShareRaw == null) missing.push(RESILIENCE_FOSSIL_ELEC_SHARE_KEY);
|
||||
if (lowCarbonRaw == null) missing.push(RESILIENCE_LOW_CARBON_GEN_KEY);
|
||||
if (powerLossesRaw == null) missing.push(RESILIENCE_POWER_LOSSES_KEY);
|
||||
if (missing.length > 0) {
|
||||
throw new ResilienceConfigurationError(
|
||||
`RESILIENCE_ENERGY_V2_ENABLED=true but required v2 energy seeds are absent: ${missing.join(', ')}. ` +
|
||||
`Provision seed-bundle-resilience-energy-v2 on Railway and confirm seeds populate BEFORE flipping the flag. ` +
|
||||
'Or set RESILIENCE_ENERGY_V2_ENABLED=false to revert to the legacy energy construct.',
|
||||
missing,
|
||||
);
|
||||
}
|
||||
|
||||
return scoreEnergyV2(countryCode, reader);
|
||||
}
|
||||
|
||||
export async function scoreGovernanceInstitutional(
|
||||
@@ -1855,10 +1900,41 @@ export async function scoreAllDimensions(
|
||||
const memoizedReader = createMemoizedSeedReader(reader);
|
||||
const [entries, freshnessMap, failedDatasets] = await Promise.all([
|
||||
Promise.all(
|
||||
RESILIENCE_DIMENSION_ORDER.map(async (dimensionId) => [
|
||||
dimensionId,
|
||||
await RESILIENCE_DIMENSION_SCORERS[dimensionId](countryCode, memoizedReader),
|
||||
] as const),
|
||||
RESILIENCE_DIMENSION_ORDER.map(async (dimensionId) => {
|
||||
try {
|
||||
const score = await RESILIENCE_DIMENSION_SCORERS[dimensionId](countryCode, memoizedReader);
|
||||
return [dimensionId, score] as const;
|
||||
} catch (err) {
|
||||
// ResilienceConfigurationError (e.g. v2 energy flag flipped without
|
||||
// seeds) surfaces here. Fail-closed per dimension, not per country:
|
||||
// the country keeps scoring other dims normally, and this dim
|
||||
// carries imputationClass='source-failure' + coverage=0 so the
|
||||
// consumer sees the gap explicitly. The T1.7 decoration pass below
|
||||
// reads this shape and leaves it alone; no double-tagging.
|
||||
if (err instanceof ResilienceConfigurationError) {
|
||||
console.warn(
|
||||
`[Resilience] configuration-error dim=${dimensionId} country=${countryCode} missing=${err.missingKeys.join(',')} — routing to source-failure`,
|
||||
);
|
||||
// Match weightedBlend's empty-data shape (score=0 NOT null
|
||||
// because the type declares score: number; coverage=0 marks
|
||||
// "no data") + explicit source-failure tag so the T1.7
|
||||
// decoration pass downstream recognises this as misconfiguration
|
||||
// rather than IMPUTE. Freshness decorated by the caller
|
||||
// alongside the other scores.
|
||||
const sourceFailureScore: ResilienceDimensionScore = {
|
||||
score: 0,
|
||||
coverage: 0,
|
||||
observedWeight: 0,
|
||||
imputedWeight: 1,
|
||||
imputationClass: 'source-failure',
|
||||
freshness: { lastObservedAtMs: 0, staleness: '' },
|
||||
};
|
||||
return [dimensionId, sourceFailureScore] as const;
|
||||
}
|
||||
// Any other error is a bug, not misconfiguration — let it surface.
|
||||
throw err;
|
||||
}
|
||||
}),
|
||||
),
|
||||
// T1.5 propagation pass: aggregate freshness at the caller level so
|
||||
// the dimension scorers stay mechanical. We share the memoized
|
||||
|
||||
@@ -7,9 +7,14 @@
|
||||
// cross-contamination from the v2 code path into the default
|
||||
// branch is a merge-blocker.
|
||||
// 2. Flag on = v2 composite. Each new indicator must move the score
|
||||
// in the documented direction (monotonicity), and countries
|
||||
// missing a v2 input should degrade gracefully to null per
|
||||
// weighted-blend contract rather than throw.
|
||||
// in the documented direction (monotonicity). When any REQUIRED
|
||||
// v2 seed is absent, the dispatch throws
|
||||
// `ResilienceConfigurationError` (fail-closed) so the operator
|
||||
// sees the misconfiguration via the source-failure path instead
|
||||
// of IMPUTE numbers that look computed. Pre-2026-04-24 the
|
||||
// scorer silently degraded to IMPUTE; plan
|
||||
// `docs/plans/2026-04-24-001-fix-resilience-v2-fail-closed-on-missing-seeds-plan.md`
|
||||
// inverts that contract.
|
||||
//
|
||||
// The tests use stubbed readers instead of Redis so the suite stays
|
||||
// hermetic.
|
||||
@@ -17,7 +22,11 @@
|
||||
import test, { describe, it, before, after } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
|
||||
import { scoreEnergy, type ResilienceSeedReader } from '../server/worldmonitor/resilience/v1/_dimension-scorers.ts';
|
||||
import {
|
||||
ResilienceConfigurationError,
|
||||
scoreEnergy,
|
||||
type ResilienceSeedReader,
|
||||
} from '../server/worldmonitor/resilience/v1/_dimension-scorers.ts';
|
||||
|
||||
const TEST_ISO2 = 'ZZ'; // fictional country so test coverage checks don't flag it
|
||||
|
||||
@@ -159,13 +168,63 @@ describe('scoreEnergy — RESILIENCE_ENERGY_V2_ENABLED=true', () => {
|
||||
assert.ok(deHigh.score > deLow.score, `DE storage 10→90 should raise score; got ${deLow.score} → ${deHigh.score}`);
|
||||
});
|
||||
|
||||
it('missing v2 seed inputs degrade gracefully (no throw, coverage < 1.0)', async () => {
|
||||
const allMissing = await scoreEnergy(TEST_ISO2, makeEnergyReader(TEST_ISO2, {
|
||||
fossilBulk: null, lowCarbonBulk: null, lossesBulk: null,
|
||||
}));
|
||||
// Score may be null/low but must NOT throw. Coverage should be
|
||||
// well below 1.0 because most inputs are absent.
|
||||
assert.ok(allMissing.coverage < 1.0, `all-missing coverage should be < 1.0, got ${allMissing.coverage}`);
|
||||
// Fail-closed contract (plan 2026-04-24-001). When flag=true but any
|
||||
// required v2 seed is absent, the dispatch throws. Silent IMPUTE
|
||||
// fallback would produce fabricated-looking numbers with zero operator
|
||||
// signal — the exact failure the user post-mortem-caught on 2026-04-24.
|
||||
// The shared makeEnergyReader fixture uses `??` fallbacks that coerce
|
||||
// null-overrides back into default data, so these fail-closed tests
|
||||
// need a direct reader that can authoritatively return null for a
|
||||
// specific key without the fixture's defaulting logic.
|
||||
const makeReaderWithMissingV2Seeds = (missing: Set<string>): ResilienceSeedReader => {
|
||||
const base = makeEnergyReader(TEST_ISO2);
|
||||
return async (key: string) => (missing.has(key) ? null : base(key));
|
||||
};
|
||||
|
||||
it('flag on + ALL three v2 seeds missing → throws ResilienceConfigurationError naming all three keys', async () => {
|
||||
const reader = makeReaderWithMissingV2Seeds(new Set([
|
||||
'resilience:fossil-electricity-share:v1',
|
||||
'resilience:low-carbon-generation:v1',
|
||||
'resilience:power-losses:v1',
|
||||
]));
|
||||
await assert.rejects(
|
||||
scoreEnergy(TEST_ISO2, reader),
|
||||
(err: unknown) => {
|
||||
assert.ok(err instanceof ResilienceConfigurationError, `expected ResilienceConfigurationError, got ${err}`);
|
||||
assert.ok(err.message.includes('resilience:fossil-electricity-share:v1'), 'error must name missing fossil-share key');
|
||||
assert.ok(err.message.includes('resilience:low-carbon-generation:v1'), 'error must name missing low-carbon key');
|
||||
assert.ok(err.message.includes('resilience:power-losses:v1'), 'error must name missing power-losses key');
|
||||
assert.equal(err.missingKeys.length, 3, 'missingKeys must enumerate every absent seed');
|
||||
return true;
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it('flag on + partial missing (only fossil-share absent) → throws naming ONLY the missing key', async () => {
|
||||
// Operator-clarity: the error must tell the operator WHICH seed is
|
||||
// broken so they can fix that specific seeder rather than chase
|
||||
// all three.
|
||||
const reader = makeReaderWithMissingV2Seeds(new Set([
|
||||
'resilience:fossil-electricity-share:v1',
|
||||
]));
|
||||
await assert.rejects(
|
||||
scoreEnergy(TEST_ISO2, reader),
|
||||
(err: unknown) => {
|
||||
assert.ok(err instanceof ResilienceConfigurationError);
|
||||
assert.deepEqual([...err.missingKeys].sort(), ['resilience:fossil-electricity-share:v1']);
|
||||
assert.ok(!err.message.includes('low-carbon-generation'), 'must not mention keys that ARE present');
|
||||
assert.ok(!err.message.includes('power-losses'), 'must not mention keys that ARE present');
|
||||
return true;
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it('flag on + ALL three v2 seeds present → v2 runs normally, no throw', async () => {
|
||||
// Regression guard for the happy path — ensuring the preflight
|
||||
// check does not block a correctly-configured v2 activation.
|
||||
const result = await scoreEnergy(TEST_ISO2, makeEnergyReader(TEST_ISO2));
|
||||
assert.equal(typeof result.score, 'number', 'happy-path v2 must produce a numeric score');
|
||||
assert.ok(result.coverage > 0, 'happy-path v2 must report positive coverage');
|
||||
});
|
||||
|
||||
it('reserveMarginPct is NOT read in v2 path (deferred per plan §3.1)', async () => {
|
||||
|
||||
Reference in New Issue
Block a user