fix(energy): strict validation — emptyDataIsFailure on Atlas seeders (#3350)

Adds `emptyDataIsFailure: true` to all 5 curated-registry seeders in the
`seed-bundle-energy-sources` Railway service. File-read-and-validate
seeders whose validateFn returns false (stale container, missing data
file, shape regression, etc.) MUST leave seed-meta stale rather than
stamping fresh `recordCount: 0` via the default `publishResult.skipped`
branch in `_seed-utils.mjs:906-917`.

Why this matters — observed production incident on 2026-04-23 (post
PR #3337 merge):

- Subset of Atlas seeders hit the validation-skip path (for reasons
  involving a Railway container stale vs the merged code + a local
  Option A run during an intermediate-file-state window).
- `_seed-utils.mjs:910` `writeFreshnessMetadata(..., 0, ...)` stamped
  `seed-meta:energy:pipelines-oil` and `seed-meta:energy:storage-facilities`
  with fresh `fetchedAt + recordCount: 0`.
- Bundle runner's interval gate at `_bundle-runner.mjs:210` reads
  `fetchedAt` only, not `recordCount`. With `elapsed < 0.8 × 10080min =
  8064min`, the gate skipped these 2 sections for ~5.5 days. No
  canonical data was written; health reported EMPTY; bundle never
  self-healed.

With `emptyDataIsFailure: true`, the strict branch at
`_seed-utils.mjs:897-905` fires instead:

  FAILURE: validation failed (empty data) — seed-meta NOT refreshed;
  bundle will retry next cycle

Seed-meta stays stale, bundle counts it as `failed++`, next cron tick
retries. Health flips STALE_SEED within max-stale-min. Operator sees
it. Loud-failure instead of silent-skip-with-meta-refresh.

Pattern previously documented for strict-floor validators
(IMF/WEO 180+ country seeders in
`feedback_strict_floor_validate_fail_poisons_seed_meta.md`) — now
applied to all 5 Energy Atlas curated registries for the same reasons.

No functional change in the healthy path — validation-passing runs
still publish canonical + fresh seed-meta as before.

Verification: typecheck clean, 6618/6618 data tests pass.
This commit is contained in:
Elie Habib
2026-04-23 20:43:27 +04:00
committed by GitHub
parent 8278c8e34e
commit 38218db7cd
5 changed files with 21 additions and 0 deletions

View File

@@ -25,6 +25,9 @@ if (isMain) {
declareRecords,
schemaVersion: 1,
maxStaleMin: MAX_STALE_MIN,
// See seed-pipelines-gas.mjs for rationale — strict validation failure
// must leave seed-meta stale so the bundle retries every tick.
emptyDataIsFailure: true,
}).catch((err) => {
const cause = err.cause ? ` (cause: ${err.cause.message || err.cause.code || err.cause})` : '';
console.error('FATAL:', (err.message || err) + cause);

View File

@@ -28,6 +28,9 @@ if (isMain) {
declareRecords,
schemaVersion: 1,
maxStaleMin: MAX_STALE_MIN,
// See seed-pipelines-gas.mjs for rationale — strict validation failure
// must leave seed-meta stale so the bundle retries every tick.
emptyDataIsFailure: true,
}).catch((err) => {
const cause = err.cause ? ` (cause: ${err.cause.message || err.cause.code || err.cause})` : '';
console.error('FATAL:', (err.message || err) + cause);

View File

@@ -31,6 +31,12 @@ if (isMain) {
declareRecords,
schemaVersion: 1,
maxStaleMin: MAX_STALE_MIN,
// File-read-and-validate seeder: if the container can't load/validate the
// registry (stale image, missing data file, shape regression), fail LOUDLY
// rather than refreshing seed-meta with recordCount=0. Without this, the
// bundle's interval gate silently locks the seeder out for ~7 days after
// a single transient validation failure.
emptyDataIsFailure: true,
}).catch((err) => {
const cause = err.cause ? ` (cause: ${err.cause.message || err.cause.code || err.cause})` : '';
console.error('FATAL:', (err.message || err) + cause);

View File

@@ -30,6 +30,9 @@ if (isMain) {
declareRecords,
schemaVersion: 1,
maxStaleMin: MAX_STALE_MIN,
// See seed-pipelines-gas.mjs for rationale — strict validation failure
// must leave seed-meta stale so the bundle retries every tick.
emptyDataIsFailure: true,
}).catch((err) => {
const cause = err.cause ? ` (cause: ${err.cause.message || err.cause.code || err.cause})` : '';
console.error('FATAL:', (err.message || err) + cause);

View File

@@ -29,6 +29,12 @@ if (isMain) {
declareRecords,
schemaVersion: 1,
maxStaleMin: MAX_STALE_MIN,
// File-read-and-validate seeder: if the container can't load/validate the
// registry (stale image, missing data file, shape regression), fail LOUDLY
// rather than refreshing seed-meta with recordCount=0. Without this, the
// bundle gate silently locks the seeder out for ~5.5 days after a single
// validation hiccup. See seed-pipelines-gas.mjs for the canonical incident.
emptyDataIsFailure: true,
}).catch((err) => {
const cause = err.cause ? ` (cause: ${err.cause.message || err.cause.code || err.cause})` : '';
console.error('FATAL:', (err.message || err) + cause);