eliott/worldmonitor
1
0
Fork 0
mirror of https://github.com/koala73/worldmonitor.git synced 2026-04-25 17:14:57 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
84ee2beb3e90f36da747e5acd136610b37490c12
worldmonitor/api/health.js
Elie Habib 84ee2beb3e feat(energy): Energy Atlas end-to-end — pipelines + storage + shortages + disruptions + country drill-down (#3294) 
* feat(energy): pipeline registries (gas + oil) — evidence-based schema

Day 6 of the Energy Atlas Release 1 plan (Week 2). First curated asset
registry for the atlas — the real gap vs GEF.

## Curated data (critical assets only, not global completeness)

scripts/data/pipelines-gas.json — 12 critical gas lines:
  Nord Stream 1/2 (offline; Swedish EEZ sabotage 2022; EU sanctions refs),
  TurkStream, Yamal–Europe (offline; Polish counter-sanctions),
  Brotherhood/Soyuz (offline; Ukraine transit expired 2024-12-31),
  Power of Siberia, Dolphin, Medgaz, TAP, TANAP,
  Central Asia–China, Langeled.

scripts/data/pipelines-oil.json — 12 critical oil lines:
  Druzhba North/South (N offline per EU 2022/879; S under landlocked
  derogation), CPC, ESPO (+ price-cap sanction ref), BTC, TAPS,
  Habshan–Fujairah (Hormuz bypass), Keystone, Kirkuk–Ceyhan (offline
  since 2023 ICC ruling), Baku–Supsa, Trans-Mountain (TMX expansion
  May 2024), ESPO spur to Daqing.

Scope note: 75+ each is Week 2b work via GEM bulk import. Today's cut
is curated from first-hand operator disclosures + regulator filings so
I can stand behind every evidence field.

## Evidence-based schema (not conclusion labels)

Per docs/methodology/pipelines.mdx: no bare `sanctions_blocked` field.
Every pipeline carries an evidence bundle with `physicalState`,
`physicalStateSource`, `operatorStatement`, `commercialState`,
`sanctionRefs[]`, `lastEvidenceUpdate`, `classifierVersion`,
`classifierConfidence`. The public badge (`flowing|reduced|offline|
disputed`) is derived server-side from this bundle at read time.

## Seeder

scripts/seed-pipelines.mjs — single process publishes BOTH keys
(energy:pipelines:{gas,oil}:v1) via two runSeed() calls. Tiny datasets
(<20KB each) so co-location is cheap and guarantees classifierVersion
consistency.

Conventions followed (worldmonitor-bootstrap-registration skill):
- TTL 21d = 3× weekly cadence (gold-standard per
  feedback_seeder_gold_standard.md)
- maxStaleMin 20_160 = 2× cadence (health-maxstalemin-write-cadence skill)
- sourceVersion + schemaVersion + recordCount + declareRecords wired
  (seed-contract-foundation)
- Zero-case explicitly NOT allowed — MIN_PIPELINES_PER_REGISTRY=8 floor

## Health registration (dual, per feedback_two_health_endpoints_must_match)

- api/health.js: BOOTSTRAP_KEYS adds pipelinesGas + pipelinesOil;
  SEED_META adds both with maxStaleMin=20_160.
- api/seed-health.js: mirror entries with intervalMin=10_080 (maxStaleMin/2).

## Bundle registration

scripts/seed-bundle-energy-sources.mjs adds a single Pipelines entry
(not two) because seed-pipelines.mjs publishes both keys in one run —
listing oil separately would double-execute. Monitoring of the oil key
staleness happens in api/health.js instead.

## Tests (tests/pipelines-registry.test.mts)

17 passing node:test assertions covering:
- Schema validation (both registries pass validateRegistry)
- Identity resolution (no id collisions, id matches object key)
- Country ISO2 normalization (from/to/transit all match /^[A-Z]{2}$/)
- Endpoint geometry within Earth bounds
- Evidence rigor: non-flowing badges require at least one supporting
  evidence source (operator statement / sanctionRefs / ais-relay /
  satellite / press)
- ClassifierConfidence in 0..1
- Commodity/capacity pairing (gas uses capacityBcmYr, oil uses
  capacityMbd — mixing = test fail)
- validateRegistry rejects: empty object, null, no-evidence fixtures,
  below-floor counts

Typecheck clean (both tsconfig.json and tsconfig.api.json).

Next: Day 7 will add list-pipelines / get-pipeline-detail RPCs in
supply-chain/v1. Day 8 ships PipelineStatusPanel with DeckGL PathLayer
consuming the registry.

* fix(energy): split seed-pipelines.mjs into two entry points — runSeed hard-exits

High finding from PR review. scripts/seed-pipelines.mjs called runSeed()
twice in one process and awaited Promise.all. But runSeed() in
scripts/_seed-utils.mjs hard-exits via process.exit on ~9 terminal paths
(lines 816, 820, 839, 888, 917, 989, plus fetch-retry 946, fatal 859,
skipped-lock 81). The first runSeed to reach any terminal path exits the
entire node process, so the second runSeed's resolve never fires — only
one of energy:pipelines:{gas,oil}:v1 would ever be written.

Since the bundle scheduled seed-pipelines.mjs exactly once, and both
api/health.js and api/seed-health.js expect both keys populated, the
other registry would stay permanently EMPTY/STALE after deploy.

Fix: split into two entry-point scripts around a shared utility.

- scripts/_pipeline-registry.mjs (NEW, was seed-pipelines.mjs) — shared
  helpers ONLY. Exports GAS_CANONICAL_KEY, OIL_CANONICAL_KEY,
  PIPELINES_TTL_SECONDS, MAX_STALE_MIN, buildGasPayload, buildOilPayload,
  validateRegistry, recordCount, declareRecords. Underscore prefix marks
  it as non-entry-point (matches _seed-utils.mjs / _seed-envelope-source.mjs
  convention).
- scripts/seed-pipelines-gas.mjs (NEW) — imports from the shared module,
  single runSeed('energy','pipelines-gas',…) call.
- scripts/seed-pipelines-oil.mjs (NEW) — same shape, oil.
- scripts/seed-bundle-energy-sources.mjs — register BOTH seeders (not one).
- scripts/seed-pipelines.mjs — deleted.
- tests/pipelines-registry.test.mts — update import path to the shared
  module. All 17 tests still pass.

Typecheck clean (both configs). Tests pass. No other consumers import
from the deleted script.

* fix(energy): complete pipeline bootstrap registration per 4-file checklist

High finding from PR review. My earlier PR description claimed
worldmonitor-bootstrap-registration was complete, but I only touched two
of the four registries (api/health.js + api/seed-health.js). The bootstrap
hydration payload itself (api/bootstrap.js) and the shared cache-keys
registry (server/_shared/cache-keys.ts) still had no entry for either
pipeline key, so any consumer that reads bootstrap data would see
pipelinesGas/pipelinesOil as missing on first load.

Files updated this commit:

- api/bootstrap.js — KEYS map + SLOW_KEYS set both gain pipelinesGas +
  pipelinesOil. Placed next to sprPolicies (same curated-registry cadence
  and tier). Slow tier is correct: weekly cron, not needed on first paint.
- server/_shared/cache-keys.ts — PIPELINES_GAS_KEY + PIPELINES_OIL_KEY
  exported constants (matches SPR_POLICIES_KEY pattern), BOOTSTRAP_KEYS map
  entries, and BOOTSTRAP_TIERS entries (both 'slow').

Not touched (intentional):
- server/gateway.ts — pipeline data is free-tier per the Energy Atlas
  plan; no PREMIUM_RPC_PATHS entry required. Energy Atlas monetization
  hooks (scenario runner, MCP tools, subscriptions) are Release 2.

Full 4-file checklist now complete:
  ✅ server/_shared/cache-keys.ts (this commit)
  ✅ api/bootstrap.js          (this commit)
  ✅ api/health.js             (earlier in PR)
  ✅ api/seed-health.js        (earlier in PR — dual-registry rule)

Typecheck clean (both configs).

* feat(energy): ListPipelines + GetPipelineDetail RPCs with evidence-derived badges

Day 7 of the Energy Atlas Release 1 plan (Week 2). Exposes the pipeline
registries (shipped in Day 6) via two supply-chain RPCs and ships the
evidence-to-badge derivation server-side.

## Proto

proto/worldmonitor/supply_chain/v1/list_pipelines.proto — new:
- ListPipelinesRequest { commodity_type?: 'gas' | 'oil' }
- ListPipelinesResponse { pipelines[], fetched_at, classifier_version, upstream_unavailable }
- GetPipelineDetailRequest { pipeline_id (required, query-param) }
- GetPipelineDetailResponse { pipeline?, revisions[], fetched_at, unavailable }
- PipelineEntry — wire shape mirroring scripts/data/pipelines-{gas,oil}.json
  + a server-derived public_badge field
- PipelineEvidence, OperatorStatement, SanctionRef, LatLon, PipelineRevisionEntry

service.proto adds both rpc methods with HTTP_METHOD_GET + path bindings:
  /api/supply-chain/v1/list-pipelines
  /api/supply-chain/v1/get-pipeline-detail

`make generate` regenerated src/generated/{client,server}/… + docs/api/
OpenAPI json/yaml.

## Evidence-derivation

server/worldmonitor/supply-chain/v1/_pipeline-evidence.ts — new.
derivePublicBadge(evidence) → 'flowing' | 'reduced' | 'offline' | 'disputed'
is deterministic + versioned (DERIVER_VERSION='badge-deriver-v1').

Rules (first match wins):
1. offline + sanctionRef OR expired/suspended commercial → offline
2. offline + operator statement → offline
3. offline + only press/ais/satellite → disputed (single-source negative claim)
4. reduced → reduced
5. flowing → flowing
6. unknown / malformed → disputed

Staleness guard: non-flowing badges on >14d-old evidence demote to
disputed. Flowing is the optimistic default — stale "still flowing" is
safer than stale "offline". Matches seed-pipelines-{gas,oil}.mjs maxStaleMin.

Tests (tests/pipeline-evidence-derivation.test.mts) — 15 passing cases
covering happy paths, disputed fallbacks, staleness guard, versioning.

## Handlers

server/worldmonitor/supply-chain/v1/list-pipelines.ts
- Reads energy:pipelines:{gas,oil}:v1 via getCachedJson.
- projectPipeline() narrows the Upstash `unknown` into PipelineEntry
  shape + calls derivePublicBadge.
- Honors commodity_type filter (skip the opposite registry's Redis read
  when the client pre-filters).
- Returns upstream_unavailable=true when BOTH registries miss.

server/worldmonitor/supply-chain/v1/get-pipeline-detail.ts
- Scans both registries by id (ids are globally unique per
  tests/pipelines-registry.test.mts).
- Empty revisions[] for now; auto-revision log wires up in Week 3.

handler.ts registers both into supplyChainHandler.

## Gateway

server/gateway.ts adds 'static' cache-tier for both new RPC paths
(registry is slow-moving; 'static' matches the other read-mostly
supply-chain endpoints).

## Consumer wiring

Not in this commit — PipelineStatusPanel (Day 8) is what will call
listPipelines/getPipelineDetail via the generated client. pipelinesGas
+ pipelinesOil stay in PENDING_CONSUMERS until Day 8.

Typecheck clean (both configs). 15 new tests + 17 registry tests all pass.

* feat(energy): PipelineStatusPanel — evidence-backed status table + drawer

Day 8 of the Energy Atlas Release 1 plan. First consumer of the Day 6–7
registries + RPCs.

## What this PR adds

- src/components/PipelineStatusPanel.ts — new panel (id=pipeline-status).
  * Bootstrap-hydrates from pipelinesGas + pipelinesOil for instant first
    paint; falls through to listPipelines() RPC if bootstrap misses.
    Background re-fetch runs on every render so a classifier-version bump
    between bootstrap stamp and first view produces a visible update.
  * Table rows sorted non-flowing-first (offline / reduced / disputed
    before flowing) — what an atlas reader cares about.
  * Click-to-expand drawer calls getPipelineDetail() lazily — operator
    statements, sanction refs (with clickable source URLs), commercial
    state, classifier version + confidence %, capacity + route metadata.
  * publicBadge color-chip palette matches the methodology doc.
  * Attribution footer with GEM (CC-BY 4.0) credit + classifier version.

- src/components/index.ts — barrel export.
- src/app/panel-layout.ts — import + createPanel('pipeline-status', …).
- src/config/panels.ts — ENERGY_PANELS adds 'pipeline-status' at priority 1.

## PENDING_CONSUMERS cleanup

tests/bootstrap.test.mjs — removes 'pipelinesGas' + 'pipelinesOil' from
the allowlist. The invariant "every bootstrap key has a getHydratedData
consumer" now enforces real wiring for these keys: the panel literally
calls getHydratedData('pipelinesGas') and getHydratedData('pipelinesOil').
Future regressions that remove the consumer will fail pre-push.

## Consumer contract verified

- 67 tests pass including bootstrap.test.mjs consumer coverage check.
- Typecheck clean.
- No DeckGL PathLayer in this commit — existing 'pipelines-layer' has a
  separate data source, so modifying DeckGLMap.ts to overlay evidence-
  derived badges on the map is a follow-up commit to avoid clobbering.

## Out of scope for Day 8 (next steps on same PR)

- DeckGL PathLayer integration (color pipelines on the main map by
  publicBadge, click-to-open this drawer) — Day 8b commit.
- Storage facility registry + StorageFacilityMapPanel — Days 9-10.

* fix(energy): PipelineStatusPanel bootstrap path — client-side badge derivation

High finding from PR review. The Day-8 panel crashed on first paint
whenever bootstrap hydration succeeded, because:

- Bootstrap hydrates raw scripts/data/pipelines-{gas,oil}.json verbatim.
- That JSON does NOT include publicBadge — that field is only added by
  the server handler's projectPipeline() in list-pipelines.ts.
- PipelineStatusPanel passed raw entries into badgeChip(), which called
  badgeLabel(undefined).charAt(0) → TypeError.

The background RPC refresh that would have repaired the data never ran
because the panel threw before reaching it. So the exact bootstrap path
newly wired in commit 6b01fa537 was broken for the new panel.

Fix: move the evidence→badge deriver to src/shared/pipeline-evidence.ts
so the client panel and the server handler run the identical function on
identical inputs. Panel projects raw bootstrap JSON through the shared
deriver client-side, producing the same publicBadge the RPC would have
returned. No UI flicker on hydration because pre- and post-RPC badges
match exactly (same function, same input).

## Changes

- src/shared/pipeline-evidence.ts (NEW) — pure deriver with duck-typed
  PipelineEvidenceInput (no generated-type dependency, so both client
  and server assign their proto-typed evidence bundles by structural
  subtyping). Exports derivePipelinePublicBadge + version + type.
- server/worldmonitor/supply-chain/v1/_pipeline-evidence.ts — now a thin
  re-export of the shared module under its older name so in-handler
  imports keep working without a sweep.
- src/components/PipelineStatusPanel.ts:
  * Imports derivePipelinePublicBadge from @/shared/pipeline-evidence.
  * NEW projectRawPipeline() defensively coerces every field from
    unknown → PipelineEntry shape, mirroring the server projection.
  * buildBootstrapResponse now routes every raw entry through the
    projection before returning, so the wire-format PipelineEntry[] the
    renderer receives always has publicBadge populated.
  * badgeChip() gained a null-guard fallback to 'disputed' — belt +
    braces so even if a future caller passes an undefined, the UI
    renders safely instead of throwing.
  * BootstrapRegistry renamed RawBootstrapRegistry with a comment
    explaining why the seeder ships raw JSON (not wire format).

## Regression tests

tests/pipeline-panel-bootstrap.test.mts (NEW) — 6 tests that exercise
the bootstrap-first-paint path end-to-end:

- Every gas + oil curated entry produces a valid badge.
- Raw entries never ship with pre-computed publicBadge (contract guard
  on the seed data format).
- Deriver never throws on undefined/null/{} evidence (was the crash).
- Nord Stream 1 regression check (offline + paperwork → offline).
- Druzhba-South staleness behavior (reduced when fresh, disputed after
  60 days without update).

38/38 tests now pass (17 registry + 15 deriver + 6 new bootstrap-path).
Typecheck clean on both configs.

## Invariant preserved

The server handler and the panel render identical badges because:
1. Same pure function (imported from the same module).
2. Same deterministic rules, same staleness window.
3. Same bootstrap data read by both paths (Redis → either bootstrap
   payload or RPC response).

No UI flicker on hydration.

* fix(energy): three PR-review P2s on PipelineStatusPanel + aggregators

## P2-1 — sanitizeUrl on external evidence links (XSS hardening)

Sanction-ref URLs and operator-statement URLs were interpolated with
escapeHtml only. HTML-escaping blocks tag injection but NOT javascript:
or data: URL schemes, so a bad URL in the seeded registry would execute
in-app when a reader clicked the evidence link. Every other panel in
the codebase (NewsPanel, GdeltIntelPanel, GeoHubsPanel, AirlineIntelPanel,
MonitorPanel) uses sanitizeUrl for this exact reason.

Fix: import sanitizeUrl from @/utils/sanitize and route both hrefs
through it. sanitizeUrl() drops non-http(s) schemes + returns '' on
invalid URLs. The renderer now suppresses the <a> entirely when
sanitize rejects — the date label still renders as plain text instead
of becoming an executable link.

## P2-2 — loadDetail catch path missing stale-response guard

The success path at loadDetail() checked `this.selectedId !== pipelineId`
to suppress stale responses when the user has clicked another pipeline
mid-flight. The catch path at line 219 had no such guard: if the user
clicked A, then B, and A's request failed before B resolved, A's error
handler cleared detailLoading and detail, showing "Pipeline detail
unavailable" for B's drawer even though B was still loading.

Fix: mirror the same `if (this.selectedId !== pipelineId) return` guard
in the catch path. The newer request now owns the drawer state
regardless of which path (success OR failure) the older one took.

## P2-3 — always-gas-preference aggregator for classifierVersion + fetchedAt

Three call sites (list-pipelines.ts handler, get-pipeline-detail.ts
handler, PipelineStatusPanel bootstrap projection) computed aggregate
classifier version and fetchedAt by `gas?.x || oil?.x || fallback`.
That was defensible when a single seed-pipelines.mjs wrote both keys
atomically (fix commit 29b4ac78f split this into two separate Railway
cron entry points). Now gas + oil cron independently, so mixed-version
(gas=v1, oil=v2 during classifier rollout) and mixed-timestamp (oil
refreshed 6h after gas) windows are the EXPECTED state, not the
exceptional one. The comment in list-pipelines.ts even said "pick the
newest classifier version" but the code didn't actually compare.

Fix: add two shared helpers in src/shared/pipeline-evidence.ts —

- pickNewerClassifierVersion(a,b) — parses /^v(\\d+)$/ and returns the
  higher-numbered version; falls back to lexicographic for non-v-
  prefixed values; handles single-missing inputs.
- pickNewerIsoTimestamp(a,b) — Date.parse()-compares and returns the
  later ISO; handles missing / malformed inputs gracefully.

Both server RPCs and the panel bootstrap projection now call these
helpers identically, so clients are told the truth about version +
freshness during partial rollouts.

## Tests

Extended tests/pipeline-evidence-derivation.test.mts with 8 new
assertions covering both pickers:

- Higher v-number wins regardless of order (v1 vs v2 → v2 both ways)
- Single-missing falls back to the one present
- Missing + missing → default 'v1' for version / '' for ts
- Non-v-numbered values fall back to lexicographic
- Explicit regression: "gas=v1 + oil=v2 during rollout" returns v2
- Explicit regression: "oil fresher than gas" returns the oil timestamp

38 → 46 tests. All pass. Typecheck clean on both configs.

* feat(energy): DeckGL PathLayer colored by evidence-derived badge + map↔panel link

Day 8b of the Energy Atlas plan. Pipelines now render on the main
DeckGL map of the energy variant colored by their derived publicBadge,
and clicking a pipeline on the map opens the same evidence drawer the
panel row-click opens.

## Why this commit

Day 8 shipped the PipelineStatusPanel as a table + drawer view.
Reviewer flag notwithstanding (fixed in 149d33ec3 + db52965cd), a
table-only pipeline view is a weak product compared to the map-centric
atlas it's meant to rival. The map-layer differentiation is the whole
point of the feature.

## What this adds

src/components/DeckGLMap.ts:
- New createEnergyPipelinesLayer() — reads hydrated pipeline registries
  via getHydratedData, projects raw JSON through the shared deriver
  (src/shared/pipeline-evidence.ts), renders a DeckGL PathLayer colored
  by publicBadge:
    flowing  → green (46,204,113)
    reduced  → amber (243,156,18)
    offline  → red   (231,76,60)
    disputed → purple (155,89,182)
  Offline + disputed get thicker strokes (3px vs 2px) for at-a-glance
  surfacing of disrupted assets. Geometry comes from raw startPoint +
  waypoints[] + endPoint per asset (straight line when no waypoints).
- Branching at line ~1498: SITE_VARIANT === 'energy' routes to the
  new method; other variants keep the static PIPELINES config (colored
  by oil/gas type). Existing commodity/finance/full map layers are
  untouched — no cross-variant leakage.
- onClick handler emits `energy:open-pipeline-detail` as a window
  CustomEvent with { pipelineId }. Loose coupling: the map doesn't
  import the panel, the panel doesn't import the map.
- Fallback: if bootstrap hasn't hydrated yet, createEnergyPipelinesLayer
  falls back to the static createPipelinesLayer() so the pipelines
  toggle always shows *something*.

src/components/PipelineStatusPanel.ts:
- Constructor registers a window event listener for
  'energy:open-pipeline-detail' → calls this.loadDetail(pipelineId) →
  drawer opens on the clicked asset. Map click and row click converge
  on the same drawer, same evidence view.
- destroy() removes the listener to prevent ghost handlers after panel
  unmount.

## Guarantees

- Bootstrap parity: the DeckGL layer calls the SAME derivePipelinePublicBadge
  as the panel and the server handler, so the map color, the table row
  chip, and the RPC response all agree on the badge. No flicker, no
  drift, no confused user.
- Variant isolation: only SITE_VARIANT === 'energy' triggers the new
  path. Commodity / finance / full map layers untouched.
- No cross-component import: the panel doesn't reference the map class
  and vice versa. The event contract is the only coupling — testable,
  swappable, tauri-safe (guarded with `typeof window !== 'undefined'`).

Typecheck clean. PR #3294 now has 8 commits.

Follow-up backlog:
- Add waypoints[] to the curated pipelines-{gas,oil}.json so the map
  draws real routes instead of straight lines (cosmetic; does not
  affect correctness).
- Tooltip case in the picking tooltip registry (line ~3748) so hover
  shows "Nord Stream 1 · OFFLINE" before click.

* fix(energy): three PR-review findings on Day 8b DeckGL integration

## P1 — getHydratedData single-use race between map + panel

src/services/bootstrap.ts:34 — `if (val !== undefined) hydrationCache.delete(key);`
The helper drains its slot on first read. Day 8 (PipelineStatusPanel) and
Day 8b (createEnergyPipelinesLayer) BOTH call getHydratedData('pipelinesGas')
and getHydratedData('pipelinesOil') — whoever renders first drains the cache
and forces the loser onto its fallback path (panel → RPC, map → static
PIPELINES layer). The commit's "shared bootstrap-backed data" guarantee
did not actually hold.

Fix: new src/shared/pipeline-registry-store.ts that reads once and memoizes.
Both consumers read through getCachedPipelineRegistries() — same data, same
reference, unlimited re-reads. When the panel's background RPC fetch lands,
it calls setCachedPipelineRegistries() to back-propagate fresh data into
the store so the map's next re-render sees the newer classifierVersion +
fetchedAt too (no map/panel drift during classifier rollouts).

Test-only injection hook (__setBootstrapReaderForTests) makes the drain-once
semantics observable without a real bootstrap payload.

## P2 — pipelines-layer tooltip regresses to blank label on energy variant

src/components/DeckGLMap.ts:3748 (pipelines-layer tooltip case) still assumed
the static-config shape (obj.type). The new energy layer emits objects with
commodityType + badge fields, so the tooltip's type-ternary fell through to
the generic fallback — hover rendered " pipeline" (empty leading commodity)
instead of "Nord Stream 1 · OFFLINE".

Fix: differentiate by presence of obj.badge (only the energy layer sets it).
On the energy variant, tooltip now reads name + commodity + badge. Static-
config variants (commodity / finance / full) keep their existing format
unchanged.

## P2 — createEnergyPipelinesLayer dropped highlightedAssets behavior

The static createPipelinesLayer() reads this.highlightedAssets.pipeline and
threads it into getColor / getWidth with an updateTrigger on the signature.
Any caller using flashAssets('pipeline', [...]) or highlightAssets([...])
gets a visible red-outline flash on the matching paths. My Day 8b energy
layer ignored the set entirely — those APIs silently no-op'd on the energy
variant.

Fix: createEnergyPipelinesLayer() now reads the same highlight set, applies
HIGHLIGHT_COLOR + wider stroke to matching IDs, and wires
updateTriggers: { getColor: sig, getWidth: sig } so DeckGL actually
recomputes when the set changes.

Also removed the unnecessary layerCache.set() in the energy path: the
store can update via RPC back-propagation, and a cache keyed only on
highlight-signature would serve stale data. With ~25 critical-asset
pipelines, rebuild per render is trivial.

## Tests

tests/pipeline-registry-store.test.mts (NEW) — 5 tests covering the
drain-once read-many invariant: multiple consumers get cached data
without re-draining, RPC back-propagation updates the source, partial
updates preserve the other commodity, and pure RPC-first (no bootstrap)
works without invoking the reader.

All 51 PR tests pass. Typecheck clean on both configs.

* feat(energy): Day 9 — storage facility registry (UGS + SPR + LNG + crude hubs)

Ships 21 critical strategic storage facilities as a curated registry, same
evidence-bundle pattern as the pipeline registries in Day 7/8:

- scripts/data/storage-facilities.json — 4 UGS + 4 SPR + 6 LNG export +
  3 LNG import + 4 crude tank farms. Each carries physicalState +
  sanctionRefs + classifierVersion/Confidence + fillDisclosed/fillSource.
- scripts/_storage-facility-registry.mjs — shared helpers (validator,
  builder, canonical key, MAX_STALE_MIN). Validator enforces facility-type
  × capacity-unit pairing (ugs→TWh, spr/tank-farm→Mb, LNG→Mtpa) and the
  non-operational badge ⇒ evidence invariant.
- scripts/seed-storage-facilities.mjs — single runSeed entry (only one
  key, so no split-seeder dance needed).
- Registered in the 4-file bootstrap checklist: cache-keys.ts
  (STORAGE_FACILITIES_KEY + BOOTSTRAP_CACHE_KEYS + BOOTSTRAP_TIERS),
  api/bootstrap.js (KEYS + SLOW_KEYS), api/health.js (BOOTSTRAP_KEYS +
  SEED_META, 14d threshold = 2× weekly cron), api/seed-health.js (mirror).
- tests/bootstrap.test.mjs PENDING_CONSUMERS adds storageFacilities —
  Day 10 StorageFacilityMapPanel will remove it.
- tests/storage-facilities-registry.test.mts — 20 tests covering schema,
  identity, geometry, type×capacity pairing, evidence contract, and
  negative-input validator rejection.

Registry fields are slow-moving; badge derivation happens at read-time
server-side once the RPC handler lands in Day 10 (panel + deckGL
ScatterplotLayer). Seeded data is live in Redis from this commit so the
Day 10 PR only adds display surfaces.

Tests: 56 pass (36 prior + 20 new). Typecheck + typecheck:api clean.

* feat(energy): Day 10 — storage atlas (ListStorageFacilities RPC + DeckGL ScatterplotLayer + panel)

End-to-end wiring for the strategic storage registry seeded in Day 9. Same
pattern as the pipeline shipping path (Days 7+8+8b): proto → handler →
shared evidence deriver → panel → DeckGL map layer, with a shared
read-once store keeping map + panel aligned.

Proto + generated code:
- list_storage_facilities.proto: ListStorageFacilities +
  GetStorageFacilityDetail messages with StorageFacilityEntry,
  StorageEvidence, StorageSanctionRef, StorageOperatorStatement,
  StorageLatLon, StorageFacilityRevisionEntry.
- service.proto wires both RPCs under /api/supply-chain/v1.
- make generate → regenerated client + server stubs + OpenAPI.

Server handlers:
- src/shared/storage-evidence.ts: shared pure deriver. Duck-typed input
  interface avoids generated-type deps; identical rules to the pipeline
  deriver (sanction/commercial paperwork vs external-signal-only offline,
  14d staleness window, version pin).
- _storage-evidence.ts: thin re-export for server handler import ergonomics.
- list-storage-facilities.ts: reads STORAGE_FACILITIES_KEY from Upstash,
  projects raw → wire format, attaches derived publicBadge, filters by
  optional facilityType query arg.
- get-storage-facility-detail.ts: single-asset lookup for drawer.
- handler.ts registers both new methods.
- gateway.ts: both routes → 'static' cache tier (registry is near-static).

Panel + map:
- src/shared/storage-facility-registry-store.ts: drain-once memo mirroring
  pipeline-registry-store. Both panel and DeckGL layer read through this
  so the single-use getHydratedData drain doesn't race between consumers.
  RPC back-propagation via setCachedStorageFacilityRegistry() keeps map ↔
  panel on the same classifierVersion during rollouts.
- StorageFacilityMapPanel.ts: table + evidence drawer. Bootstrap hot path
  projects raw registry through same deriver as server so first-paint
  badge matches post-RPC badge (no flicker). sanitizeUrl + stale-response
  guards (success + catch paths) carried over from PipelineStatusPanel.
- DeckGLMap.ts createEnergyStorageLayer(): ScatterplotLayer keyed on
  badge color; log-scale radius (6km–26km) keeps Rehden visible next to
  Ras Laffan. Click dispatches 'energy:open-storage-facility-detail' —
  panel listens and opens its drawer (loose coupling, no direct refs).
- Tooltip branch on storage-facilities-layer shows facility type, country,
  capacity unit, and badge.
- Added 'storageFacilities' optional field to MapLayers type (optional so
  existing variant literals across commodity/finance/tech/happy/full/etc.
  don't need touching). Wired into LAYER_REGISTRY + VARIANT_LAYER_ORDER.energy
  + ENERGY_MAP_LAYERS + ENERGY_MOBILE_MAP_LAYERS. Panel entry added to
  ENERGY_PANELS + panel-layout createPanel. PENDING_CONSUMERS entry from
  Day 9 removed — panel + map layer are now real consumers.

Tests:
- storage-evidence-derivation.test.mts (17 tests): covers every curated
  facility yields a valid badge, null/malformed input never throws,
  offline sanction/commercial/operator rules, external-signal-only offline
  → disputed, staleness demotion.
- storage-facility-registry-store.test.mts (4 tests): drain-once, no-data
  drain, RPC update, pure-RPC-first path.

All 6,426 unit tests pass. Typecheck + typecheck:api clean. Pre-existing
src-tauri/sidecar/ test failure is unrelated (no diff touches src-tauri/).

* feat(energy): Day 11 — fuel-shortage registry schema + seed + RPC (classifier post-launch)

Ships v1 of the global fuel-shortage alert registry. Severity is the
CLASSIFIER OUTPUT (confirmed/watch), not a client derivation — we ship
the evidence alongside so readers can audit the grounds. v1 is seeded
from curated JSON; post-launch the proactive-intelligence classifier
(Day 12 work) extends the same key directly.

Data:
- scripts/data/fuel-shortages.json — 15 known active shortages
  (PK, LK, NG×2, CU, VE, LB, ZW, AR, IR, BO, KE, PA, EG, BY)
  spanning petrol/diesel/jet across confirmed + watch tiers. Each entry
  carries evidenceSources[] (regulator/operator/press), firstSeen,
  lastConfirmed, resolvedAt, impactTypes[], causeChain[], classifier
  version + confidence. Confirmed severity enforces authoritative
  evidence at schema level.

Seeder:
- scripts/_fuel-shortage-registry.mjs — shared validator (enforces
  iso2 country, enum products/severities/impacts/causes, authoritative
  evidence for confirmed). MIN_SHORTAGES=10.
- scripts/seed-fuel-shortages.mjs — single runSeed entry.
- Registered in seed-bundle-energy-sources.mjs at DAY cadence (shortages
  move faster than registry assets).

Bootstrap 4-file registration:
- cache-keys.ts: FUEL_SHORTAGES_KEY + BOOTSTRAP_CACHE_KEYS + BOOTSTRAP_TIERS.
- api/bootstrap.js: KEYS + SLOW_KEYS.
- api/health.js: BOOTSTRAP_KEYS + SEED_META (2880min = 2× daily cron).
- api/seed-health.js: mirrors intervalMin=1440.

Proto + RPC:
- list_fuel_shortages.proto: ListFuelShortages (country/product/severity
  query facets) + GetFuelShortageDetail messages with FuelShortageEntry,
  FuelShortageEvidence, FuelShortageEvidenceSource.
- service.proto wires both new RPCs under /api/supply-chain/v1.
- list-fuel-shortages.ts handler projects raw → wire format, supports
  server-side country/product/severity filtering.
- get-fuel-shortage-detail.ts single-shortage lookup.
- handler.ts registers both. gateway.ts: 'medium' cache-tier (daily
  classifier updates warrant moderate freshness).

Shared evidence helper:
- src/shared/shortage-evidence.ts: deriveShortageEvidenceQuality maps
  (confidence + authoritative-source count + freshness) → 'strong' |
  'moderate' | 'thin' for client-side sort/trust indicators. Does NOT
  change severity — classifier owns that decision.
- countEvidenceSources buckets sources for the drawer's "n regulator /
  m press" line.

Tests:
- tests/fuel-shortages-registry.test.mts (19 tests): schema, identity,
  enum coverage, evidence contract (confirmed → authoritative source),
  validateRegistry negative cases.
- tests/shortage-evidence.test.mts (10 tests): quality deriver edge
  cases, source bucketing.
- tests/bootstrap.test.mjs PENDING_CONSUMERS adds fuelShortages —
  FuelShortagePanel arrives Day 12 which will remove the entry.

Typecheck + typecheck:api clean. 64 tests pass.

* feat(energy): Day 12 — FuelShortagePanel + DeckGL shortage pins

End-to-end wiring of the fuel-shortage registry shipped in Day 11: panel
on the Energy variant page, ScatterplotLayer pins on the DeckGL map,
both reading through a shared single-drain store so they don't race on
the bootstrap cache.

Panel:
- src/components/FuelShortagePanel.ts — table sorted by severity (confirmed
  first) then evidence quality (strong → thin) then most-recent lastConfirmed.
  Drawer shows short description, first-seen / last-confirmed / resolved,
  impact types, cause chain, classifier version/confidence, and a typed
  evidence-source list with regulator/operator/press chips. sanitizeUrl on
  every href so classifier-ingested URLs can't render as javascript:. Same
  stale-response guards on success + catch paths as the other detail drawers.
- Consumes deriveShortageEvidenceQuality for client-side trust indicator
  (three-dot ●●● / ●●○ / ●○○), NOT for severity — severity is classifier
  output.
- Registered in ENERGY_PANELS + panel-layout.ts + components barrel.

Shared store:
- src/shared/fuel-shortage-registry-store.ts — same drain-once memoize
  pattern as pipeline- and storage-facility-registry-store. Both the
  panel and the DeckGL shortage-pins layer read through it.

DeckGL layer:
- DeckGLMap.createEnergyShortagePinsLayer: ScatterplotLayer placing one
  pin per active shortage at the country centroid (via getCountryCentroid
  from services/country-geometry). Stacking offset (~0.8° lon) when
  multiple shortages share a country so Nigeria's petrol + diesel don't
  render as a single dot. Confirmed pins 55km radius; watch 38km. Click
  dispatches 'energy:open-fuel-shortage-detail' — panel listens.
- Tooltip branch on fuel-shortages-layer: country · product · short
  description · severity.
- Layer registered in LAYER_REGISTRY, VARIANT_LAYER_ORDER.energy,
  ENERGY_MAP_LAYERS, ENERGY_MOBILE_MAP_LAYERS. MapLayers.fuelShortages
  is optional on the type so other variants' literals remain valid.

Tests:
- tests/fuel-shortage-registry-store.test.mts (4 tests): drain-once,
  no-data, RPC back-prop, pure-RPC-first path.
- tests/bootstrap.test.mjs — fuelShortages removed from PENDING_CONSUMERS.

Typecheck + typecheck:api clean. 39 tests pass (plus full suite in pre-push).

* feat(energy): Day 13 — energy disruption event log + asset timeline drawer

Ships the energy:disruptions:v1 registry that threads together pipelines
and storage facilities: state transitions (sabotage, sanction, maintenance,
mechanical, weather, commercial, war) keyed by assetId so any asset's
drawer can render its history without a second registry lookup.

Data + seeder:
- scripts/data/energy-disruptions.json — 12 curated events spanning
  Nord Stream 1/2 sabotage, Druzhba sanctions, CPC force majeure,
  TurkStream maintenance, Yamal halt, Rehden trusteeship, Arctic LNG 2
  sanction, ESPO drone strikes, BTC fire (historical), Sabine Pass
  Hurricane Beryl, Power of Siberia ramp. Each event links back to a
  seeded asset.
- scripts/_energy-disruption-registry.mjs — validator enforces valid
  assetType/eventType/cause enums, http(s) sources, startAt ≤ endAt,
  MIN_EVENTS=8.
- scripts/seed-energy-disruptions.mjs — runSeed entry (weekly cron).
- Bundle entry at 7×DAY cadence.

Bootstrap 4-file registration (cache-keys.ts + bootstrap.js + health.js +
seed-health.js) — energyDisruptions in PENDING_CONSUMERS because panel
drawers fetch lazily via RPC on drawer-open rather than hydrating from
bootstrap directly.

Proto + handler:
- list_energy_disruptions.proto: ListEnergyDisruptions with
  assetId / assetType / ongoingOnly query facets. Returns events sorted
  newest-first.
- list-energy-disruptions.ts projects raw → wire format, supports all
  three query facets.
- Registered in handler.ts. gateway.ts: 'medium' cache tier.

Shared timeline helper:
- src/shared/disruption-timeline.ts — pure formatters (formatEventWindow,
  formatCapacityOffline, statusForEvent). No generated-type deps so
  PipelineStatusPanel + StorageFacilityMapPanel import the same helpers
  and render the timeline identically.

Panel integration:
- PipelineStatusPanel.loadDetail now fetches getPipelineDetail +
  listEnergyDisruptions({assetId, assetType:'pipeline'}) in parallel.
  Drawer gains "Disruption timeline (N)" section with event type, date
  window, capacity offline, cause chain, and short description per entry.
- StorageFacilityMapPanel gets identical treatment with assetType='storage'.
- Both reset detailEvents on closeDetail and on fresh click (stale-response
  safety).

Tests:
- tests/energy-disruptions-registry.test.mts (17 tests): schema, identity,
  enum coverage, evidence, negative inputs.
- tests/bootstrap.test.mjs — energyDisruptions added to PENDING_CONSUMERS.

Typecheck + typecheck:api clean. 51 tests pass locally (plus full suite
in pre-push).

* feat(energy): Day 14 — country drill-down Atlas exposure section

Extends CountryDeepDivePanel's existing "Energy Profile" card with a
mini Atlas-exposure section that surfaces per-country exposure to the
new registries we shipped in Days 7-13.

For each country:
- Pipelines touching this country (from, to, or transit) — clickable
  rows that dispatch 'energy:open-pipeline-detail' so the PipelineStatusPanel
  drawer opens on the energy variant; no-op on other variants.
- Storage facilities in this country — same loose-coupling pattern
  with 'energy:open-storage-facility-detail'.
- Active fuel shortages in this country — severity breakdown line
  (N confirmed · M watch) plus clickable rows emitting
  'energy:open-fuel-shortage-detail'.

Silent absence: sections render only when the country has matching
assets/events, so countries with no pipeline, storage, or shortage
touchpoints see the existing energy-profile card unchanged.

Lazy stores: reads go through the same shared drain-once stores
(getCachedPipelineRegistries, getCachedStorageFacilityRegistry,
getCachedFuelShortageRegistry) so CountryDeepDivePanel does NOT race
with Atlas panels over the single-drain bootstrap cache. Dynamic
import() keeps the three stores out of the panel's static import graph
so non-energy variants can tree-shake them.

Typecheck clean. No schema changes; purely additive UI read from
already-shipped registries.

* docs(energy): methodology page for energy disruption event log

Fills the /docs/methodology/disruptions URL referenced by
list_energy_disruptions.proto, scripts/_energy-disruption-registry.mjs,
and the panel attribution footers. Explains scope (state transitions
not daily noise), data shape, what counts as a disruption, classifier
evolution path, RPC contract, and ties into the sibling pipeline +
storage + shortage methodology pages.

No code change; pure docs completion for Week 4 launch polish.

* fix(energy): upstreamUnavailable only fires when Redis returned nothing

Two handlers (list-storage-facilities + list-pipelines) conflated "empty
filter result on a healthy registry" with "upstream unavailable". A
caller who queried one facilityType/commodityType and legitimately got
zero matches was told the upstream was down — which may push clients to
error-state rendering or suppress caching instead of showing a valid
empty list.

list-storage-facilities.ts — upstreamUnavailable now only fires when
`raw` is null (Redis miss). Zero filtered rows on a healthy registry
returns upstreamUnavailable: false + empty array. Matches the sibling
list-fuel-shortages handler and the wire contract in
list_storage_facilities.proto.

list-pipelines.ts — same bug, subtler shape. Now checks "requested at
least one side AND received nothing" rather than "zero rows after
collection". A filter that legitimately matches no gas/oil pipelines on
a healthy registry now returns upstreamUnavailable: false.

list-energy-disruptions.ts and list-fuel-shortages.ts already had the
correct shape (only flag unavailable when raw is missing) — left as-is.

Typecheck + typecheck:api clean. No tests added: the existing registry
schema tests cover the projection/filter helpers, and the handler-level
gating change is documented in code comments for future audits.

* fix(energy): three Greptile findings on PR #3294

Two P1 filter bugs (resolved shortages rendered as active) and one P2
contract inconsistency on the disruptions handler.

P1: DeckGLMap createEnergyShortagePinsLayer rendered every shortage in
the registry as an active crisis pin — including entries where the
classifier has written resolvedAt to mark the crisis over. Added a
filter so only entries with a null/empty resolvedAt become map pins.
Curated v1 data has resolvedAt=null everywhere so no visible change
today, but the moment the classifier starts writing resolutions
post-launch, resolved shortages would have appeared as ongoing.

P1: CountryDeepDivePanel renderAtlasExposure had the same bug in the
country drill-down — "N confirmed · M watch" counts included resolved
entries, inflating the active-crisis line per country. Same one-line
filter fix.

P2: list-energy-disruptions.ts gated upstreamUnavailable on
`!raw?.events` — a partial write (top-level object present but `events`
property missing) fired the "upstream down" flag, inconsistent with
the sibling handlers (list-pipelines, list-storage-facilities,
list-fuel-shortages) that only fire on `!raw`. Rewrote to match:
`!raw` → upstreamUnavailable, empty events → normal empty list. This
also aligns with the contract documented on the upstream-unavailable-
vs-empty-filter skill extracted from the earlier P2 review.

Typecheck + typecheck:api clean. All three fixes are one-liner filter
or gate changes; no test additions needed (registry tests still pass
with v1 data since resolvedAt is null throughout).
2026-04-23 07:34:07 +04:00

790 lines
54 KiB
JavaScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { jsonResponse } from './_json-response.js';
// Seed-envelope helper. PR 1 imports it here so PR 2 can wire envelope-aware
// reads at specific call sites without further plumbing. It's a no-op on
// legacy-shape seed-meta values (they have no `_seed` wrapper and pass through
// as `.data`), so importing it is behavior-preserving.
import { unwrapEnvelope } from './_seed-envelope.js';
// @ts-expect-error — JS module, no declaration file
import { redisPipeline, getRedisCredentials } from './_upstash-json.js';
export const config = { runtime: 'edge' };
const BOOTSTRAP_KEYS = {
earthquakes: 'seismology:earthquakes:v1',
outages: 'infra:outages:v1',
sectors: 'market:sectors:v2',
etfFlows: 'market:etf-flows:v1',
climateAnomalies: 'climate:anomalies:v2',
climateDisasters: 'climate:disasters:v1',
climateAirQuality: 'climate:air-quality:v1',
co2Monitoring: 'climate:co2-monitoring:v1',
oceanIce: 'climate:ocean-ice:v1',
wildfires: 'wildfire:fires:v1',
marketQuotes: 'market:stocks-bootstrap:v1',
commodityQuotes: 'market:commodities-bootstrap:v1',
cyberThreats: 'cyber:threats-bootstrap:v2',
techReadiness: 'economic:worldbank-techreadiness:v1',
progressData: 'economic:worldbank-progress:v1',
renewableEnergy: 'economic:worldbank-renewable:v1',
positiveGeoEvents: 'positive_events:geo-bootstrap:v1',
riskScores: 'risk:scores:sebuf:stale:v1',
naturalEvents: 'natural:events:v1',
flightDelays: 'aviation:delays-bootstrap:v1',
newsInsights: 'news:insights:v1',
predictionMarkets: 'prediction:markets-bootstrap:v1',
cryptoQuotes: 'market:crypto:v1',
gulfQuotes: 'market:gulf-quotes:v1',
stablecoinMarkets: 'market:stablecoins:v1',
unrestEvents: 'unrest:events:v1',
iranEvents: 'conflict:iran-events:v1',
ucdpEvents: 'conflict:ucdp-events:v1',
weatherAlerts: 'weather:alerts:v1',
spending: 'economic:spending:v1',
techEvents: 'research:tech-events-bootstrap:v1',
gdeltIntel: 'intelligence:gdelt-intel:v1',
correlationCards: 'correlation:cards-bootstrap:v1',
forecasts: 'forecast:predictions:v2',
securityAdvisories: 'intelligence:advisories-bootstrap:v1',
customsRevenue: 'trade:customs-revenue:v1',
comtradeFlows: 'comtrade:flows:v1',
blsSeries: 'bls:series:v1',
sanctionsPressure: 'sanctions:pressure:v1',
crossSourceSignals: 'intelligence:cross-source-signals:v1',
sanctionsEntities: 'sanctions:entities:v1',
radiationWatch: 'radiation:observations:v1',
consumerPricesOverview: 'consumer-prices:overview:ae',
consumerPricesCategories: 'consumer-prices:categories:ae:30d',
consumerPricesMovers: 'consumer-prices:movers:ae:30d',
consumerPricesSpread: 'consumer-prices:retailer-spread:ae:essentials-ae',
consumerPricesFreshness: 'consumer-prices:freshness:ae',
groceryBasket: 'economic:grocery-basket:v1',
bigmac: 'economic:bigmac:v1',
fuelPrices: 'economic:fuel-prices:v1',
faoFoodPriceIndex: 'economic:fao-ffpi:v1',
nationalDebt: 'economic:national-debt:v1',
defiTokens: 'market:defi-tokens:v1',
aiTokens: 'market:ai-tokens:v1',
otherTokens: 'market:other-tokens:v1',
fredBatch: 'economic:fred:v1:FEDFUNDS:0',
ecbEstr: 'economic:fred:v1:ESTR:0',
ecbEuribor3m: 'economic:fred:v1:EURIBOR3M:0',
ecbEuribor6m: 'economic:fred:v1:EURIBOR6M:0',
ecbEuribor1y: 'economic:fred:v1:EURIBOR1Y:0',
fearGreedIndex: 'market:fear-greed:v1',
breadthHistory: 'market:breadth-history:v1',
euYieldCurve: 'economic:yield-curve-eu:v1',
earningsCalendar: 'market:earnings-calendar:v1',
econCalendar: 'economic:econ-calendar:v1',
cotPositioning: 'market:cot:v1',
hyperliquidFlow: 'market:hyperliquid:flow:v1',
crudeInventories: 'economic:crude-inventories:v1',
natGasStorage: 'economic:nat-gas-storage:v1',
spr: 'economic:spr:v1',
refineryInputs: 'economic:refinery-inputs:v1',
ecbFxRates: 'economic:ecb-fx-rates:v1',
eurostatCountryData: 'economic:eurostat-country-data:v1',
eurostatHousePrices: 'economic:eurostat:house-prices:v1',
eurostatGovDebtQ: 'economic:eurostat:gov-debt-q:v1',
eurostatIndProd: 'economic:eurostat:industrial-production:v1',
euGasStorage: 'economic:eu-gas-storage:v1',
euFsi: 'economic:fsi-eu:v1',
shippingStress: 'supply_chain:shipping_stress:v1',
diseaseOutbreaks: 'health:disease-outbreaks:v1',
healthAirQuality: 'health:air-quality:v1',
socialVelocity: 'intelligence:social:reddit:v1',
wsbTickers: 'intelligence:wsb-tickers:v1',
vpdTrackerRealtime: 'health:vpd-tracker:realtime:v1',
vpdTrackerHistorical: 'health:vpd-tracker:historical:v1',
electricityPrices: 'energy:electricity:v1:index',
gasStorageCountries: 'energy:gas-storage:v1:_countries',
aaiiSentiment: 'market:aaii-sentiment:v1',
cryptoSectors: 'market:crypto-sectors:v1',
ddosAttacks: 'cf:radar:ddos:v1',
economicStress: 'economic:stress-index:v1',
trafficAnomalies: 'cf:radar:traffic-anomalies:v1',
};
const STANDALONE_KEYS = {
serviceStatuses: 'infra:service-statuses:v1',
macroSignals: 'economic:macro-signals:v1',
bisPolicy: 'economic:bis:policy:v1',
bisExchange: 'economic:bis:eer:v1',
fxYoy: 'economic:fx:yoy:v1',
bisCredit: 'economic:bis:credit:v1',
bisDsr: 'economic:bis:dsr:v1',
bisPropertyResidential: 'economic:bis:property-residential:v1',
bisPropertyCommercial: 'economic:bis:property-commercial:v1',
imfMacro: 'economic:imf:macro:v2',
imfGrowth: 'economic:imf:growth:v1',
imfLabor: 'economic:imf:labor:v1',
imfExternal: 'economic:imf:external:v1',
climateZoneNormals: 'climate:zone-normals:v1',
shippingRates: 'supply_chain:shipping:v2',
chokepoints: 'supply_chain:chokepoints:v4',
minerals: 'supply_chain:minerals:v2',
giving: 'giving:summary:v1',
gpsjam: 'intelligence:gpsjam:v2',
theaterPosture: 'theater_posture:sebuf:stale:v1',
theaterPostureLive: 'theater-posture:sebuf:v1',
theaterPostureBackup: 'theater-posture:sebuf:backup:v1',
riskScoresLive: 'risk:scores:sebuf:v1',
usniFleet: 'usni-fleet:sebuf:v1',
usniFleetStale: 'usni-fleet:sebuf:stale:v1',
faaDelays: 'aviation:delays:faa:v1',
intlDelays: 'aviation:delays:intl:v3',
notamClosures: 'aviation:notam:closures:v2',
positiveEventsLive: 'positive-events:geo:v1',
cableHealth: 'cable-health-v1',
cyberThreatsRpc: 'cyber:threats:v2',
militaryBases: 'military:bases:active',
militaryFlights: 'military:flights:v1',
militaryFlightsStale: 'military:flights:stale:v1',
temporalAnomalies: 'temporal:anomalies:v1',
displacement: `displacement:summary:v1:${new Date().getUTCFullYear()}`,
displacementPrev: `displacement:summary:v1:${new Date().getUTCFullYear() - 1}`,
satellites: 'intelligence:satellites:tle:v1',
portwatch: 'supply_chain:portwatch:v1',
portwatchPortActivity: 'supply_chain:portwatch-ports:v1:_countries',
corridorrisk: 'supply_chain:corridorrisk:v1',
chokepointTransits: 'supply_chain:chokepoint_transits:v1',
transitSummaries: 'supply_chain:transit-summaries:v1',
thermalEscalation: 'thermal:escalation:v1',
tariffTrendsUs: 'trade:tariffs:v1:840:all:10',
militaryForecastInputs: 'military:forecast-inputs:stale:v1',
gscpi: 'economic:fred:v1:GSCPI:0',
marketImplications: 'intelligence:market-implications:v1',
hormuzTracker: 'supply_chain:hormuz_tracker:v1',
simulationPackageLatest: 'forecast:simulation-package:latest',
simulationOutcomeLatest: 'forecast:simulation-outcome:latest',
newsThreatSummary: 'news:threat:summary:v1',
climateNews: 'climate:news-intelligence:v1',
pizzint: 'intelligence:pizzint:seed:v1',
resilienceStaticIndex: 'resilience:static:index:v1',
resilienceStaticFao: 'resilience:static:fao',
resilienceRanking: 'resilience:ranking:v10',
productCatalog: 'product-catalog:v2',
energySpineCountries: 'energy:spine:v1:_countries',
energyExposure: 'energy:exposure:v1:index',
energyMixAll: 'energy:mix:v1:_all',
regulatoryActions: 'regulatory:actions:v1',
energyIntelligence: 'energy:intelligence:feed:v1',
ieaOilStocks: 'energy:iea-oil-stocks:v1:index',
oilStocksAnalysis: 'energy:oil-stocks-analysis:v1',
eiaPetroleum: 'energy:eia-petroleum:v1',
jodiGas: 'energy:jodi-gas:v1:_countries',
lngVulnerability: 'energy:lng-vulnerability:v1',
jodiOil: 'energy:jodi-oil:v1:_countries',
chokepointBaselines: 'energy:chokepoint-baselines:v1',
portwatchChokepointsRef: 'portwatch:chokepoints:ref:v1',
chokepointFlows: 'energy:chokepoint-flows:v1',
emberElectricity: 'energy:ember:v1:_all',
resilienceIntervals: 'resilience:intervals:v1:US',
sprPolicies: 'energy:spr-policies:v1',
pipelinesGas: 'energy:pipelines:gas:v1',
pipelinesOil: 'energy:pipelines:oil:v1',
storageFacilities: 'energy:storage-facilities:v1',
fuelShortages: 'energy:fuel-shortages:v1',
energyDisruptions: 'energy:disruptions:v1',
energyCrisisPolicies: 'energy:crisis-policies:v1',
regionalSnapshots: 'intelligence:regional-snapshots:summary:v1',
regionalBriefs: 'intelligence:regional-briefs:summary:v1',
recoveryFiscalSpace: 'resilience:recovery:fiscal-space:v1',
recoveryReserveAdequacy: 'resilience:recovery:reserve-adequacy:v1',
recoveryExternalDebt: 'resilience:recovery:external-debt:v1',
recoveryImportHhi: 'resilience:recovery:import-hhi:v1',
recoveryFuelStocks: 'resilience:recovery:fuel-stocks:v1',
// PR 1 v2 energy-construct seeds. ON_DEMAND_KEYS until Railway cron
// provisions; see below.
lowCarbonGeneration: 'resilience:low-carbon-generation:v1',
fossilElectricityShare: 'resilience:fossil-electricity-share:v1',
powerLosses: 'resilience:power-losses:v1',
goldExtended: 'market:gold-extended:v1',
goldEtfFlows: 'market:gold-etf-flows:v1',
goldCbReserves: 'market:gold-cb-reserves:v1',
// Relay-side loop heartbeats. ais-relay.cjs writes these on successful child
// exit for the two execFile-spawned seeders (chokepoint-flows, climate-news).
// A stale heartbeat means the relay loop itself is broken (child dying at
// import, parent event-loop blocked, container in a restart loop, etc.)
// and alarms earlier than the underlying seed-meta staleness window.
chokepointFlowsRelayHeartbeat: 'relay:heartbeat:chokepoint-flows',
climateNewsRelayHeartbeat: 'relay:heartbeat:climate-news',
telegramFeed: 'intelligence:telegram-feed:v1',
};
const SEED_META = {
earthquakes: { key: 'seed-meta:seismology:earthquakes', maxStaleMin: 30 },
wildfires: { key: 'seed-meta:wildfire:fires', maxStaleMin: 360 }, // FIRMS NRT resets at midnight UTC; new-day data takes 3-6h to accumulate
outages: { key: 'seed-meta:infra:outages', maxStaleMin: 30 },
climateAnomalies: { key: 'seed-meta:climate:anomalies', maxStaleMin: 240 }, // runs as independent Railway cron (0 */2 * * *); 240 = 2x interval
climateDisasters: { key: 'seed-meta:climate:disasters', maxStaleMin: 720 }, // runs every 6h; 720min = 2x interval
climateAirQuality:{ key: 'seed-meta:health:air-quality', maxStaleMin: 180 }, // hourly cron; 180 = 3x interval — shares meta key with healthAirQuality (same seeder run)
climateZoneNormals: { key: 'seed-meta:climate:zone-normals', maxStaleMin: 89280 }, // monthly cron on the 1st; 62d = 2x 31-day cadence
co2Monitoring: { key: 'seed-meta:climate:co2-monitoring', maxStaleMin: 4320 }, // daily cron at 06:00 UTC; 72h tolerates two missed runs
oceanIce: { key: 'seed-meta:climate:ocean-ice', maxStaleMin: 2880 }, // daily cron at 08:00 UTC; 48h = 2× interval, tolerates one missed run
climateNews: { key: 'seed-meta:climate:news-intelligence', maxStaleMin: 90 }, // relay loop every 30min; 90 = 3× interval
unrestEvents: { key: 'seed-meta:unrest:events', maxStaleMin: 120 }, // 45min cron; 120 = 2h grace (was 75 = 30min buffer, too tight)
cyberThreats: { key: 'seed-meta:cyber:threats', maxStaleMin: 240 }, // 2h interval; 240min = 2x interval
cryptoQuotes: { key: 'seed-meta:market:crypto', maxStaleMin: 30 },
etfFlows: { key: 'seed-meta:market:etf-flows', maxStaleMin: 60 },
gulfQuotes: { key: 'seed-meta:market:gulf-quotes', maxStaleMin: 30 },
stablecoinMarkets:{ key: 'seed-meta:market:stablecoins', maxStaleMin: 60 },
naturalEvents: { key: 'seed-meta:natural:events', maxStaleMin: 360 }, // 2h cron; 3x interval; was 120 (TTL was 60min — panel went dark before health alarmed)
flightDelays: { key: 'seed-meta:aviation:faa', maxStaleMin: 90 }, // CACHE_TTL=7200s; matches notamClosures from same cron
notamClosures: { key: 'seed-meta:aviation:notam', maxStaleMin: 240 }, // 2h interval; 240min = 2x interval
predictionMarkets: { key: 'seed-meta:prediction:markets', maxStaleMin: 90 },
newsInsights: { key: 'seed-meta:news:insights', maxStaleMin: 30 },
marketQuotes: { key: 'seed-meta:market:stocks', maxStaleMin: 30 },
commodityQuotes: { key: 'seed-meta:market:commodities', maxStaleMin: 30 },
goldExtended: { key: 'seed-meta:market:gold-extended', maxStaleMin: 30 },
goldEtfFlows: { key: 'seed-meta:market:gold-etf-flows', maxStaleMin: 2880 }, // SPDR publishes daily; 2× = 48h tolerance
goldCbReserves: { key: 'seed-meta:market:gold-cb-reserves', maxStaleMin: 44640 }, // IMF IFS is monthly w/ ~2-3mo lag; 31d tolerance
// RPC/warm-ping keys — seed-meta written by relay loops or handlers
// serviceStatuses: moved to ON_DEMAND — RPC-populated, no dedicated seed, goes stale when no users visit
cableHealth: { key: 'seed-meta:cable-health', maxStaleMin: 90 }, // ais-relay warm-ping runs every 30min; 90min = 3× interval catches missed pings without false positives
macroSignals: { key: 'seed-meta:economic:macro-signals', maxStaleMin: 150 }, // seed-economy cron; primary key energy-prices has same 150min threshold
bisPolicy: { key: 'seed-meta:economic:bis', maxStaleMin: 10080 }, // runSeed('economic','bis',...) writes seed-meta:economic:bis
// seed-bis-extended.mjs writes per-dataset seed-meta keys ONLY when that
// specific dataset published fresh entries — so a single-dataset BIS outage
// (e.g. WS_DSR 500s) goes stale in health without falsely dragging down the
// healthy ones. 24h = 2× 12h cron.
bisDsr: { key: 'seed-meta:economic:bis-dsr', maxStaleMin: 1440 },
bisPropertyResidential:{ key: 'seed-meta:economic:bis-property-residential', maxStaleMin: 1440 },
bisPropertyCommercial: { key: 'seed-meta:economic:bis-property-commercial', maxStaleMin: 1440 },
imfMacro: { key: 'seed-meta:economic:imf-macro', maxStaleMin: 100800 }, // monthly seed; 100800min = 70 days = 2× interval (absorbs one missed run)
imfGrowth: { key: 'seed-meta:economic:imf-growth', maxStaleMin: 100800 }, // monthly seed; 70d threshold matches imfMacro (same WEO release cadence)
imfLabor: { key: 'seed-meta:economic:imf-labor', maxStaleMin: 100800 }, // monthly seed; 70d threshold matches imfMacro
imfExternal: { key: 'seed-meta:economic:imf-external', maxStaleMin: 100800 }, // monthly seed; 70d threshold matches imfMacro
shippingRates: { key: 'seed-meta:supply_chain:shipping', maxStaleMin: 420 },
chokepoints: { key: 'seed-meta:supply_chain:chokepoints', maxStaleMin: 60, minRecordCount: 13 }, // 13 canonical chokepoints; get-chokepoint-status writes covered-count → < 13 = upstream partial (portwatch/ArcGIS dropped some)
// minerals + giving: on-demand cachedFetchJson only, no seed-meta writer — freshness checked via TTL
// bisExchange + bisCredit: extras written by same BIS script via writeExtraKey, no dedicated seed-meta
fxYoy: { key: 'seed-meta:economic:fx-yoy', maxStaleMin: 1500 }, // daily cron; 25h tolerance + 1h drift
gpsjam: { key: 'seed-meta:intelligence:gpsjam', maxStaleMin: 720 },
positiveGeoEvents:{ key: 'seed-meta:positive-events:geo', maxStaleMin: 60 },
riskScores: { key: 'seed-meta:intelligence:risk-scores', maxStaleMin: 30 }, // CII warm-ping every 8min; 30min = ~3.5x interval,
iranEvents: { key: 'seed-meta:conflict:iran-events', maxStaleMin: 20160 }, // manual seed from LiveUAMap; 20160 = 14d = 2× weekly cadence
ucdpEvents: { key: 'seed-meta:conflict:ucdp-events', maxStaleMin: 420 },
militaryFlights: { key: 'seed-meta:military:flights', maxStaleMin: 30 }, // cron ~10min (LIVE_TTL=600s); 30min = 3x interval,
satellites: { key: 'seed-meta:intelligence:satellites', maxStaleMin: 240 }, // CelesTrak every 120min; 240min = absorbs one missed cycle
weatherAlerts: { key: 'seed-meta:weather:alerts', maxStaleMin: 45 }, // relay loop every 15min; 45 = 3× interval (was 30 = 2×, too tight on relay hiccup)
spending: { key: 'seed-meta:economic:spending', maxStaleMin: 120 },
techEvents: { key: 'seed-meta:research:tech-events', maxStaleMin: 480 },
gdeltIntel: { key: 'seed-meta:intelligence:gdelt-intel', maxStaleMin: 420 }, // 6h cron + 1h grace; CACHE_TTL is 24h so per-topic merge always has a prior snapshot
telegramFeed: { key: 'seed-meta:intelligence:telegram-feed:v1', maxStaleMin: 10 }, // 60s poll interval; 10min grace catches poll failures before they go stale in the panel
forecasts: { key: 'seed-meta:forecast:predictions', maxStaleMin: 90 },
sectors: { key: 'seed-meta:market:sectors', maxStaleMin: 30 },
techReadiness: { key: 'seed-meta:economic:worldbank-techreadiness:v1', maxStaleMin: 10080 },
progressData: { key: 'seed-meta:economic:worldbank-progress:v1', maxStaleMin: 10080 },
renewableEnergy: { key: 'seed-meta:economic:worldbank-renewable:v1', maxStaleMin: 10080 },
intlDelays: { key: 'seed-meta:aviation:intl', maxStaleMin: 90 },
// faaDelays shares seed-meta key with flightDelays — no duplicate entry needed here
theaterPosture: { key: 'seed-meta:theater-posture', maxStaleMin: 60 },
correlationCards: { key: 'seed-meta:correlation:cards', maxStaleMin: 15 },
portwatch: { key: 'seed-meta:supply_chain:portwatch', maxStaleMin: 720 },
portwatchPortActivity: { key: 'seed-meta:supply_chain:portwatch-ports', maxStaleMin: 2160 }, // 12h cron; 2160min = 36h = 3x interval
corridorrisk: { key: 'seed-meta:supply_chain:corridorrisk', maxStaleMin: 120 },
chokepointTransits: { key: 'seed-meta:supply_chain:chokepoint_transits', maxStaleMin: 30 }, // relay every 10min; 30min = 3x interval,
transitSummaries: { key: 'seed-meta:supply_chain:transit-summaries', maxStaleMin: 30 }, // relay every 10min; 30min = 3x interval,
usniFleet: { key: 'seed-meta:military:usni-fleet', maxStaleMin: 720 }, // relay loop every 6h; 720 = 2× interval (was 480 = 1.3×, too tight)
securityAdvisories: { key: 'seed-meta:intelligence:advisories', maxStaleMin: 120 },
customsRevenue: { key: 'seed-meta:trade:customs-revenue', maxStaleMin: 1440 },
comtradeFlows: { key: 'seed-meta:trade:comtrade-flows', maxStaleMin: 2880 }, // 24h cron; 2880min = 48h = 2x interval
blsSeries: { key: 'seed-meta:economic:bls-series', maxStaleMin: 2880 }, // daily seed; 2880min = 48h = 2x interval
sanctionsPressure: { key: 'seed-meta:sanctions:pressure', maxStaleMin: 720 },
crossSourceSignals: { key: 'seed-meta:intelligence:cross-source-signals', maxStaleMin: 30 }, // 15min cron; 30min = 2x interval
regionalSnapshots: { key: 'seed-meta:intelligence:regional-snapshots', maxStaleMin: 720 }, // 6h cron via seed-bundle-derived-signals; 720min = 12h = 2x interval
regionalBriefs: { key: 'seed-meta:intelligence:regional-briefs', maxStaleMin: 20160 }, // weekly cron; 20160min = 14 days = 2x interval
sanctionsEntities: { key: 'seed-meta:sanctions:entities', maxStaleMin: 1440 }, // 12h cron; 1440min = 24h = 2x interval
radiationWatch: { key: 'seed-meta:radiation:observations', maxStaleMin: 30 },
groceryBasket: { key: 'seed-meta:economic:grocery-basket', maxStaleMin: 10080 }, // weekly seed; 10080 = 7 days
bigmac: { key: 'seed-meta:economic:bigmac', maxStaleMin: 10080 }, // weekly seed; 10080 = 7 days
fuelPrices: { key: 'seed-meta:economic:fuel-prices', maxStaleMin: 10080 }, // weekly seed; 10080 = 7 days
faoFoodPriceIndex: { key: 'seed-meta:economic:fao-ffpi', maxStaleMin: 86400 }, // monthly seed; 86400 = 60 days (2x interval)
thermalEscalation: { key: 'seed-meta:thermal:escalation', maxStaleMin: 360 }, // cron every 2h; 360 = 3x interval (was 240 = 2x)
nationalDebt: { key: 'seed-meta:economic:national-debt', maxStaleMin: 86400 }, // monthly seed (seed-bundle-macro intervalMs: 30 * DAY); 60d = 2x interval absorbs one missed run. Prior 10080 (7d) was narrower than the cron interval so every cron past day 7 alarmed STALE_SEED.
tariffTrendsUs: { key: 'seed-meta:trade:tariffs:v1:840:all:10', maxStaleMin: 900 },
// publish.ts runs once daily (02:30 UTC); seed-meta TTL=52h — maxStaleMin must cover the full 24h cycle
consumerPricesOverview: { key: 'seed-meta:consumer-prices:overview:ae', maxStaleMin: 1500 }, // 25h = 24h cadence + 1h grace
consumerPricesCategories: { key: 'seed-meta:consumer-prices:categories:ae:30d', maxStaleMin: 1500 },
consumerPricesMovers: { key: 'seed-meta:consumer-prices:movers:ae:30d', maxStaleMin: 1500 },
consumerPricesSpread: { key: 'seed-meta:consumer-prices:retailer-spread:ae:essentials-ae', maxStaleMin: 1500 },
consumerPricesFreshness: { key: 'seed-meta:consumer-prices:freshness:ae', maxStaleMin: 1500 },
// defiTokens/aiTokens/otherTokens all share one seed run (seed-token-panels cron, every 30min)
defiTokens: { key: 'seed-meta:market:token-panels', maxStaleMin: 90 },
aiTokens: { key: 'seed-meta:market:token-panels', maxStaleMin: 90 },
otherTokens: { key: 'seed-meta:market:token-panels', maxStaleMin: 90 },
fredBatch: { key: 'seed-meta:economic:fred:v1:FEDFUNDS:0', maxStaleMin: 1500 }, // daily cron
ecbEstr: { key: 'seed-meta:economic:ecb-short-rates', maxStaleMin: 4320 }, // daily ECB publish; 4320min = 3d = TTL/interval
ecbEuribor3m: { key: 'seed-meta:economic:ecb-short-rates', maxStaleMin: 4320 }, // shared meta key with ecbEstr
ecbEuribor6m: { key: 'seed-meta:economic:ecb-short-rates', maxStaleMin: 4320 }, // shared meta key with ecbEstr
ecbEuribor1y: { key: 'seed-meta:economic:ecb-short-rates', maxStaleMin: 4320 }, // shared meta key with ecbEstr
gscpi: { key: 'seed-meta:economic:gscpi', maxStaleMin: 2880 }, // 24h interval; 2880min = 48h = 2x interval
fearGreedIndex: { key: 'seed-meta:market:fear-greed', maxStaleMin: 720 }, // 6h cron; 720min = 12h = 2x interval
breadthHistory: { key: 'seed-meta:market:breadth-history', maxStaleMin: 5760 }, // cron at 02:00 UTC, Tue-Sat (captures Mon-Fri market close); max gap Sat→Tue = 72h + 24h miss buffer = 96h = 5760min. 48h was wrong — alarmed every Monday morning when Sun+Mon are intentionally skipped.
hormuzTracker: { key: 'seed-meta:supply_chain:hormuz_tracker', maxStaleMin: 2880 }, // daily cron; 2880min = 48h = 2x interval
earningsCalendar: { key: 'seed-meta:market:earnings-calendar', maxStaleMin: 1440 }, // 12h cron; 1440min = 24h = 2x interval
econCalendar: { key: 'seed-meta:economic:econ-calendar', maxStaleMin: 1440 }, // 12h cron; 1440min = 24h = 2x interval
cotPositioning: { key: 'seed-meta:market:cot', maxStaleMin: 14400 }, // weekly CFTC release; 14400min = 10d = 1.4x interval (weekend + delay buffer)
hyperliquidFlow: { key: 'seed-meta:market:hyperliquid-flow', maxStaleMin: 15 }, // Railway cron 5min; 15min = 3x interval
crudeInventories: { key: 'seed-meta:economic:crude-inventories', maxStaleMin: 20160 }, // weekly EIA data; 20160min = 14 days = 2x weekly cadence
natGasStorage: { key: 'seed-meta:economic:nat-gas-storage', maxStaleMin: 20160 }, // weekly EIA data; 20160min = 14 days = 2x weekly cadence
spr: { key: 'seed-meta:economic:spr', maxStaleMin: 20160 }, // weekly EIA data; 20160min = 14 days = 2x weekly cadence
refineryInputs: { key: 'seed-meta:economic:refinery-inputs', maxStaleMin: 20160 }, // weekly EIA data; 20160min = 14 days = 2x weekly cadence
ecbFxRates: { key: 'seed-meta:economic:ecb-fx-rates', maxStaleMin: 5760 }, // daily seed (weekdays + holidays); 5760min = 96h = covers Wed→Mon Easter gap
eurostatCountryData: { key: 'seed-meta:economic:eurostat-country-data', maxStaleMin: 4320 }, // daily seed; 4320min = 3 days = 3x interval
eurostatHousePrices: { key: 'seed-meta:economic:eurostat-house-prices', maxStaleMin: 60 * 24 * 50 }, // weekly cron, annual data; 50d threshold = 35d TTL + 15d buffer
eurostatGovDebtQ: { key: 'seed-meta:economic:eurostat-gov-debt-q', maxStaleMin: 60 * 24 * 14 }, // 2d cron, quarterly data; 14d threshold matches TTL + quarterly release drift
eurostatIndProd: { key: 'seed-meta:economic:eurostat-industrial-production', maxStaleMin: 60 * 24 * 5 }, // daily cron, monthly data; 5d threshold matches TTL
euGasStorage: { key: 'seed-meta:economic:eu-gas-storage', maxStaleMin: 2880 }, // daily seed (T+1); 2880min = 48h = 2x interval
euYieldCurve: { key: 'seed-meta:economic:yield-curve-eu', maxStaleMin: 4320 }, // daily seed (weekdays only); 4320min = 72h = covers Fri→Mon gap
euFsi: { key: 'seed-meta:economic:fsi-eu', maxStaleMin: 20160 }, // weekly seed (Saturday); 20160min = 14d = 2x interval
newsThreatSummary: { key: 'seed-meta:news:threat-summary', maxStaleMin: 60 }, // relay classify every ~20min; 60min = 3x interval
shippingStress: { key: 'seed-meta:supply_chain:shipping_stress', maxStaleMin: 45 }, // relay loop every 15min; 45 = 3x interval (was 30 = 2×, too tight on relay hiccup)
diseaseOutbreaks: { key: 'seed-meta:health:disease-outbreaks', maxStaleMin: 2880 }, // daily seed; 2880 = 48h = 2x interval
healthAirQuality: { key: 'seed-meta:health:air-quality', maxStaleMin: 180 }, // hourly cron; 180 = 3x interval for shared health/climate seed
socialVelocity: { key: 'seed-meta:intelligence:social-reddit', maxStaleMin: 180 }, // relay loop every 60min (hourly, bumped from 10min to reduce Reddit IP blocking); 180 = 3x interval
wsbTickers: { key: 'seed-meta:intelligence:wsb-tickers', maxStaleMin: 180 }, // relay loop every 60min; 180 = 3x interval
pizzint: { key: 'seed-meta:intelligence:pizzint', maxStaleMin: 30 }, // relay loop every 10min; 30 = 3x interval
productCatalog: { key: 'seed-meta:product-catalog', maxStaleMin: 1080 }, // relay loop every 6h; 1080 = 18h = 3x interval
vpdTrackerRealtime: { key: 'seed-meta:health:vpd-tracker', maxStaleMin: 2880 }, // daily seed (0 2 * * *); 2880min = 48h = 2x interval
vpdTrackerHistorical: { key: 'seed-meta:health:vpd-tracker', maxStaleMin: 2880 }, // shares seed-meta key with vpdTrackerRealtime (same run)
resilienceStaticIndex: { key: 'seed-meta:resilience:static', maxStaleMin: 576000 }, // annual October snapshot; 400d threshold matches TTL and preserves prior-year data on source outages
resilienceStaticFao: { key: 'seed-meta:resilience:static', maxStaleMin: 576000 }, // same seeder + same heartbeat as resilienceStaticIndex; required so EMPTY_DATA_OK + missing data degrades to STALE_SEED instead of silent OK
resilienceRanking: { key: 'seed-meta:resilience:ranking', maxStaleMin: 720 }, // RPC cache (12h TTL, refreshed every 6h by seed-resilience-scores cron via refreshRankingAggregate); 12h staleness threshold = 2 missed cron ticks
resilienceIntervals: { key: 'seed-meta:resilience:intervals', maxStaleMin: 20160 }, // weekly cron; 20160min = 14d = 2x interval
energyExposure: { key: 'seed-meta:economic:owid-energy-mix', maxStaleMin: 50400 }, // monthly cron on 1st; 50400min = 35d = TTL matches cron cadence + 5d buffer
energyMixAll: { key: 'seed-meta:economic:owid-energy-mix', maxStaleMin: 50400 }, // same seed run as energyExposure; shares seed-meta key
regulatoryActions: { key: 'seed-meta:regulatory:actions', maxStaleMin: 360 }, // 2h cron; 360min = 3x interval
energySpineCountries: { key: 'seed-meta:energy:spine', maxStaleMin: 2880 }, // daily cron (06:00 UTC); 2880min = 48h = 2x interval
electricityPrices: { key: 'seed-meta:energy:electricity-prices', maxStaleMin: 2880 }, // daily cron (14:00 UTC); 2880min = 48h = 2x interval
gasStorageCountries: { key: 'seed-meta:energy:gas-storage-countries', maxStaleMin: 2880 }, // daily cron at 10:30 UTC; 2880min = 48h = 2x interval
energyIntelligence: { key: 'seed-meta:energy:intelligence', maxStaleMin: 720 }, // 6h cron; 720min = 2x interval
jodiOil: { key: 'seed-meta:energy:jodi-oil', maxStaleMin: 60 * 24 * 40 }, // monthly cron on 25th; 40d threshold matches 35d TTL + 5d buffer
ieaOilStocks: { key: 'seed-meta:energy:iea-oil-stocks', maxStaleMin: 60 * 24 * 40 }, // monthly cron on 15th; 40d threshold = TTL_SECONDS
oilStocksAnalysis: { key: 'seed-meta:energy:oil-stocks-analysis', maxStaleMin: 60 * 24 * 50 }, // afterPublish of ieaOilStocks; 50d = matches seed-meta TTL (exceeds 40d data TTL)
eiaPetroleum: { key: 'seed-meta:energy:eia-petroleum', maxStaleMin: 4320 }, // daily bundle cron (seed-bundle-energy-sources); 72h = 3× interval, well under 7d data TTL
jodiGas: { key: 'seed-meta:energy:jodi-gas', maxStaleMin: 60 * 24 * 40 }, // monthly cron on 25th; 40d threshold matches 35d TTL + 5d buffer
lngVulnerability: { key: 'seed-meta:energy:jodi-gas', maxStaleMin: 60 * 24 * 40 }, // written by jodi-gas seeder afterPublish; shares seed-meta key
chokepointBaselines: { key: 'seed-meta:energy:chokepoint-baselines', maxStaleMin: 60 * 24 * 400 }, // 400 days
sprPolicies: { key: 'seed-meta:energy:spr-policies', maxStaleMin: 60 * 24 * 400 }, // 400 days; static registry, same cadence as chokepoint baselines
pipelinesGas: { key: 'seed-meta:energy:pipelines-gas', maxStaleMin: 20_160 }, // 14d — weekly cron (7d) × 2 headroom
pipelinesOil: { key: 'seed-meta:energy:pipelines-oil', maxStaleMin: 20_160 }, // 14d — same seed-pipelines.mjs publishes both keys
storageFacilities: { key: 'seed-meta:energy:storage-facilities', maxStaleMin: 20_160 }, // 14d — weekly cron (7d) × 2 headroom
fuelShortages: { key: 'seed-meta:energy:fuel-shortages', maxStaleMin: 2880 }, // 2d — daily cron × 2 headroom (classifier-driven post-launch)
energyDisruptions: { key: 'seed-meta:energy:disruptions', maxStaleMin: 20_160 }, // 14d — weekly cron × 2 headroom
energyCrisisPolicies: { key: 'seed-meta:energy:crisis-policies', maxStaleMin: 60 * 24 * 400 }, // static data, ~400d TTL matches seeder
aaiiSentiment: { key: 'seed-meta:market:aaii-sentiment', maxStaleMin: 20160 }, // weekly cron; 20160min = 14 days = 2x weekly cadence
portwatchChokepointsRef: { key: 'seed-meta:portwatch:chokepoints-ref', maxStaleMin: 60 * 24 * 14 }, // seed-bundle-portwatch runs this at WEEK cadence; 14d = 2× interval
chokepointFlows: { key: 'seed-meta:energy:chokepoint-flows', maxStaleMin: 720 }, // 6h cron; 720min = 2x interval
// Relay-side heartbeat written by ais-relay.cjs on successful child exit.
// Detects "relay loop fires but child dies at import/runtime" failures
// (e.g. ERR_MODULE_NOT_FOUND from a missing Dockerfile COPY) 4h earlier
// than the 720min seed-meta threshold above. TTL is 18h on the writer.
chokepointFlowsRelayHeartbeat: { key: 'relay:heartbeat:chokepoint-flows', maxStaleMin: 480 }, // 6h loop; 8h alarm
climateNewsRelayHeartbeat: { key: 'relay:heartbeat:climate-news', maxStaleMin: 60 }, // 30m loop; 60m alarm
emberElectricity: { key: 'seed-meta:energy:ember', maxStaleMin: 2880 }, // daily cron (08:00 UTC); 2880min = 48h = 2x interval
cryptoSectors: { key: 'seed-meta:market:crypto-sectors', maxStaleMin: 120 }, // relay loop every ~30min; 120min = 2h = 4x interval
ddosAttacks: { key: 'seed-meta:cf:radar:ddos', maxStaleMin: 60 }, // written by seed-internet-outages afterPublish; outages cron ~15min; 60 = 4x interval
economicStress: { key: 'seed-meta:economic:stress-index', maxStaleMin: 180 }, // computed in seed-economy afterPublish; cron ~1h; 180min = 3x interval
marketImplications: { key: 'seed-meta:intelligence:market-implications', maxStaleMin: 120 }, // LLM-generated in seed-forecasts; cron ~1h; 120min = 2x interval
trafficAnomalies: { key: 'seed-meta:cf:radar:traffic-anomalies', maxStaleMin: 60 }, // written by seed-internet-outages afterPublish; outages cron ~15min; 60 = 4x interval
chokepointExposure: { key: 'seed-meta:supply_chain:chokepoint-exposure', maxStaleMin: 2880 }, // daily cron; 2880min = 48h = 2x interval
recoveryFiscalSpace: { key: 'seed-meta:resilience:recovery:fiscal-space', maxStaleMin: 86400 }, // monthly cron; 86400min = 60d = 2x interval
recoveryReserveAdequacy: { key: 'seed-meta:resilience:recovery:reserve-adequacy', maxStaleMin: 86400 }, // monthly cron; 86400min = 60d = 2x interval
recoveryExternalDebt: { key: 'seed-meta:resilience:recovery:external-debt', maxStaleMin: 86400 }, // monthly cron; 86400min = 60d = 2x interval
recoveryImportHhi: { key: 'seed-meta:resilience:recovery:import-hhi', maxStaleMin: 86400 }, // monthly cron; 86400min = 60d = 2x interval
recoveryFuelStocks: { key: 'seed-meta:resilience:recovery:fuel-stocks', maxStaleMin: 86400 }, // monthly cron; 86400min = 60d = 2x interval
// PR 1 v2 energy seeds — weekly cron (8d * 1440 = 11520min = 2x interval).
// Listed in ON_DEMAND_KEYS below until Railway cron provisions and
// the first clean run lands; after that they graduate to the normal
// SEED_META staleness check like the recovery seeds above.
lowCarbonGeneration: { key: 'seed-meta:resilience:low-carbon-generation', maxStaleMin: 11520 },
fossilElectricityShare: { key: 'seed-meta:resilience:fossil-electricity-share', maxStaleMin: 11520 },
powerLosses: { key: 'seed-meta:resilience:power-losses', maxStaleMin: 11520 },
};
// Standalone keys that are populated on-demand by RPC handlers (not seeds).
// Empty = WARN not CRIT since they only exist after first request.
const ON_DEMAND_KEYS = new Set([
'riskScoresLive',
'usniFleetStale', 'positiveEventsLive',
'bisPolicy', 'bisExchange', 'bisCredit',
// bisDsr/bisPropertyResidential/bisPropertyCommercial have dedicated SEED_META
// entries (seed-bis-extended.mjs), so they are not on-demand.
'macroSignals', 'shippingRates', 'chokepoints', 'minerals', 'giving',
'cyberThreatsRpc', 'militaryBases', 'temporalAnomalies', 'displacement',
'corridorrisk', // intermediate key; data flows through transit-summaries:v1
'serviceStatuses', // RPC-populated; seed-meta written on fresh fetch only, goes stale between visits
'militaryForecastInputs', // intermediate seed-to-seed pipeline key; only populated after seed-military-flights runs
'marketImplications', // LLM-generated inside forecast cron; can fail silently on LLM errors — degrade to WARN not CRIT
'simulationPackageLatest', // written by writeSimulationPackage after deep forecast runs; only present after first successful deep run
'simulationOutcomeLatest', // written by writeSimulationOutcome after simulation runs; only present after first successful simulation
'newsThreatSummary', // relay classify loop — only written when mergedByCountry has entries; absent on quiet news periods
'resilienceRanking', // on-demand RPC cache populated after ranking requests; missing before first Pro use is expected
'recoveryFiscalSpace', 'recoveryReserveAdequacy', 'recoveryExternalDebt',
'recoveryImportHhi', 'recoveryFuelStocks', // recovery pillar: stub seeders not yet deployed, keys may be absent
// PR 1 v2 energy-construct seeds. TRANSITIONAL: the three seeders
// ship with their health registry rows in this PR but Railway cron
// is provisioned as a follow-up action. Gated as on-demand until
// the first clean run lands; graduate out of this set after ~7 days
// of successful production cron runs (verify via
// `seed-meta:resilience:{low-carbon-generation,fossil-electricity-share,power-losses}.fetchedAt`).
'lowCarbonGeneration', 'fossilElectricityShare', 'powerLosses',
'displacementPrev', // covered by cascade onto current-year displacement; empty most of the year
'fxYoy', // TRANSITIONAL (PR #3071): seed-fx-yoy Railway cron deployed manually after merge —
// gate as on-demand so a deploy-order race or first-cron-run failure doesn't
// fire a CRIT health alarm. Remove from this set after ~7 days of clean
// production cron runs (verify via `seed-meta:economic:fx-yoy.fetchedAt`).
'hyperliquidFlow', // TRANSITIONAL: seed-hyperliquid-flow runs inside seed-bundle-market-backup on
// Railway; gate as on-demand so initial deploy-order race or first cold-start
// snapshot doesn't CRIT. Remove after ~7 days of clean production cron runs.
'chokepointFlowsRelayHeartbeat', // TRANSITIONAL (PR #3133): ais-relay.cjs writes this on the
// first successful child exit after a deploy. Vercel deploys
// api/health.js instantly, but Railway rebuild + 6h initial
// loop interval means the key is absent for up to ~6h post-merge.
// Gate as on-demand so the deploy window doesn't CRIT. Remove
// after ~7 days of clean production runs (verify via
// `relay:heartbeat:chokepoint-flows.fetchedAt`).
'climateNewsRelayHeartbeat', // TRANSITIONAL (PR #3133): same deploy-order rationale.
// 30min initial loop, so window is shorter but still present.
// Remove after ~7 days alongside the chokepoint-flows entry.
'eiaPetroleum', // TRANSITIONAL: gold-standard migration of /api/eia/petroleum
// from live Vercel fetch to Redis-reader (seed-bundle-energy-sources
// daily cron). SEED_META entry above enforces 72h staleness — this
// ON_DEMAND gate only softens the absent-on-deploy case (Vercel
// deploys instantly; Railway EIA_API_KEY + first daily tick ~24h
// behind). STALE_SEED still fires if data goes stale after first seed.
// Remove from this set after ~7 days of clean cron runs so
// never-provisioned Railway promotes EMPTY_ON_DEMAND → EMPTY (CRIT).
]);
// Keys where 0 records is a valid healthy state (e.g. no airports closed,
// no earnings events this week, econ calendar quiet between seasons).
// The key must still exist in Redis; only the record count can be 0.
const EMPTY_DATA_OK_KEYS = new Set([
'notamClosures', 'faaDelays', 'gpsjam', 'positiveGeoEvents', 'weatherAlerts',
'earningsCalendar', 'econCalendar', 'cotPositioning',
'usniFleet', // usniFleetStale covers the fallback; relay outages → WARN not CRIT
'newsThreatSummary', // only written when classify produces country matches; quiet news periods = 0 countries, no write
'recoveryFiscalSpace',
'recoveryImportHhi', 'recoveryFuelStocks', // recovery pillar seeds: stub seeders write empty payloads until real sources are wired
'ddosAttacks', 'trafficAnomalies', // zero events during quiet periods is valid, not critical
'resilienceStaticFao', // empty aggregate = no IPC Phase 3+ countries this year (possible in theory); the key must exist but count=0 is fine
'cableHealth', // `cables: {}` = no active subsea cable disruptions per NGA NAVAREA warnings — all cables implicitly healthy. Also covers NGA-upstream-down windows where get-cable-health writes back the fallback response (empty cables); without this, those would alarm EMPTY_DATA.
]);
// Cascade groups: if any key in the group has data, all empty siblings are OK.
// Theater posture uses live → stale → backup fallback chain.
const CASCADE_GROUPS = {
theaterPosture: ['theaterPosture', 'theaterPostureLive', 'theaterPostureBackup'],
theaterPostureLive: ['theaterPosture', 'theaterPostureLive', 'theaterPostureBackup'],
theaterPostureBackup: ['theaterPosture', 'theaterPostureLive', 'theaterPostureBackup'],
militaryFlights: ['militaryFlights', 'militaryFlightsStale'],
militaryFlightsStale: ['militaryFlights', 'militaryFlightsStale'],
// Displacement key embeds UTC year — on Jan 1 the new-year key may be empty
// for hours until the seed runs. Cascade onto the previous-year snapshot.
displacement: ['displacement', 'displacementPrev'],
displacementPrev: ['displacement', 'displacementPrev'],
};
const NEG_SENTINEL = '__WM_NEG__';
function parseRedisValue(raw) {
if (!raw || raw === NEG_SENTINEL) return null;
try { return JSON.parse(raw); } catch { return raw; }
}
// Real data is always >0 bytes. The negative-cache sentinel is exactly
// NEG_SENTINEL.length bytes (10), so any strlen > 0 that is NOT exactly that
// length counts as data. The previous `> 10` heuristic misclassified
// legitimately small payloads (`{}`, `[]`, `0`) as missing.
function strlenIsData(strlen) {
return strlen > 0 && strlen !== NEG_SENTINEL.length;
}
function readSeedMeta(seedCfg, keyMetaValues, keyMetaErrors, now) {
if (!seedCfg) {
return { seedAge: null, seedStale: null, seedError: false, metaReadFailed: false, metaCount: null };
}
// Per-command Redis errors on the GET seed-meta half of the pipeline must
// not silently fall through to STALE_SEED — promote to REDIS_PARTIAL.
if (keyMetaErrors.get(seedCfg.key)) {
return { seedAge: null, seedStale: null, seedError: false, metaReadFailed: true, metaCount: null };
}
// Unwrap through the envelope helper. Legacy seed-meta is a bare
// `{ fetchedAt, recordCount, sourceVersion, status? }` object with no `_seed`
// wrapper, so `unwrapEnvelope` returns it as `.data` unchanged. PR 2 wires
// true envelope reads at the canonical-key layer; this import establishes
// the dependency so behavior stays byte-identical in PR 1.
const meta = unwrapEnvelope(parseRedisValue(keyMetaValues.get(seedCfg.key))).data;
if (meta?.status === 'error') {
return { seedAge: null, seedStale: true, seedError: true, metaReadFailed: false, metaCount: null };
}
let seedAge = null;
let seedStale = true;
if (meta?.fetchedAt) {
seedAge = Math.round((now - meta.fetchedAt) / 60_000);
seedStale = seedAge > seedCfg.maxStaleMin;
}
const metaCount = meta?.count ?? meta?.recordCount ?? null;
return { seedAge, seedStale, seedError: false, metaReadFailed: false, metaCount };
}
function isCascadeCovered(name, hasData, keyStrens, keyErrors) {
const siblings = CASCADE_GROUPS[name];
if (!siblings || hasData) return false;
for (const sibling of siblings) {
if (sibling === name) continue;
const sibKey = STANDALONE_KEYS[sibling] ?? BOOTSTRAP_KEYS[sibling];
if (!sibKey) continue;
if (keyErrors.get(sibKey)) continue;
if (strlenIsData(keyStrens.get(sibKey) ?? 0)) return true;
}
return false;
}
function classifyKey(name, redisKey, opts, ctx) {
const { keyStrens, keyErrors, keyMetaValues, keyMetaErrors, now } = ctx;
const seedCfg = SEED_META[name];
const isOnDemand = !!opts.allowOnDemand && ON_DEMAND_KEYS.has(name);
const meta = readSeedMeta(seedCfg, keyMetaValues, keyMetaErrors, now);
// Per-command Redis errors (data STRLEN or seed-meta GET) propagate as their
// own bucket — don't conflate with "key missing", since ops needs to know if
// the read itself failed.
if (keyErrors.get(redisKey) || meta.metaReadFailed) {
const entry = { status: 'REDIS_PARTIAL', records: null };
if (seedCfg) entry.maxStaleMin = seedCfg.maxStaleMin;
return entry;
}
const strlen = keyStrens.get(redisKey) ?? 0;
const hasData = strlenIsData(strlen);
const { seedAge, seedStale, seedError, metaCount } = meta;
// When the data key is gone the meta count is meaningless; force records=0
// so we never display the contradictory "EMPTY records=N>0" pair (item 1).
const records = hasData ? (metaCount ?? 1) : 0;
const cascadeCovered = isCascadeCovered(name, hasData, keyStrens, keyErrors);
let status;
if (seedError) status = 'SEED_ERROR';
else if (!hasData) {
if (cascadeCovered) status = 'OK_CASCADE';
else if (EMPTY_DATA_OK_KEYS.has(name)) status = seedStale === true ? 'STALE_SEED' : 'OK';
else if (isOnDemand) status = 'EMPTY_ON_DEMAND';
else status = 'EMPTY';
} else if (records === 0) {
// hasData is true in this branch, so cascade can never apply (isCascadeCovered
// short-circuits when hasData=true). Cascade only shields wholly absent keys.
if (EMPTY_DATA_OK_KEYS.has(name)) status = seedStale === true ? 'STALE_SEED' : 'OK';
else if (isOnDemand) status = 'EMPTY_ON_DEMAND';
else status = 'EMPTY_DATA';
} else if (seedStale === true) status = 'STALE_SEED';
// Coverage threshold: producers that know their canonical shape size can
// declare minRecordCount. When the writer reports a count below threshold
// (e.g., 10/13 chokepoints because portwatch dropped some), this degrades
// to COVERAGE_PARTIAL (warn) instead of reporting OK. Producer must write
// seed-meta.recordCount using the *covered* count, not the shape size.
else if (seedCfg?.minRecordCount != null && records < seedCfg.minRecordCount) status = 'COVERAGE_PARTIAL';
else status = 'OK';
const entry = { status, records };
if (seedAge !== null) entry.seedAgeMin = seedAge;
if (seedCfg) entry.maxStaleMin = seedCfg.maxStaleMin;
if (seedCfg?.minRecordCount != null) entry.minRecordCount = seedCfg.minRecordCount;
return entry;
}
const STATUS_COUNTS = {
OK: 'ok',
OK_CASCADE: 'ok',
STALE_SEED: 'warn',
SEED_ERROR: 'warn',
EMPTY_ON_DEMAND: 'warn',
REDIS_PARTIAL: 'warn',
COVERAGE_PARTIAL: 'warn',
EMPTY: 'crit',
EMPTY_DATA: 'crit',
};
export default async function handler(req, ctx) {
const headers = {
'Content-Type': 'application/json',
'Cache-Control': 'private, no-store, max-age=0',
'CDN-Cache-Control': 'no-store',
'CF-Cache-Status': 'BYPASS',
'Access-Control-Allow-Origin': '*',
};
if (req.method === 'OPTIONS') {
return new Response(null, { status: 204, headers });
}
const now = Date.now();
const allDataKeys = [
...Object.values(BOOTSTRAP_KEYS),
...Object.values(STANDALONE_KEYS),
];
const allMetaKeys = Object.values(SEED_META).map(s => s.key);
// STRLEN for data keys avoids loading large blobs into memory (OOM prevention).
// NEG_SENTINEL ('__WM_NEG__') is 10 bytes — strlenIsData() rejects exactly
// that length while accepting any other non-zero strlen as data.
let results;
try {
const commands = [
...allDataKeys.map(k => ['STRLEN', k]),
...allMetaKeys.map(k => ['GET', k]),
];
if (!getRedisCredentials()) throw new Error('Redis not configured');
results = await redisPipeline(commands, 8_000);
if (!results) throw new Error('Redis request failed');
} catch (err) {
return jsonResponse({
status: 'REDIS_DOWN',
error: err.message,
checkedAt: new Date(now).toISOString(),
}, 200, headers);
}
// keyStrens: byte length per data key (0 = missing/empty/sentinel)
// keyErrors: per-command Redis errors so we can surface REDIS_PARTIAL
const keyStrens = new Map();
const keyErrors = new Map();
for (let i = 0; i < allDataKeys.length; i++) {
const r = results[i];
if (r?.error) keyErrors.set(allDataKeys[i], r.error);
keyStrens.set(allDataKeys[i], r?.result ?? 0);
}
// keyMetaValues: parsed seed-meta objects (GET, small payloads)
// keyMetaErrors: per-command errors so a single GET failure surfaces as
// REDIS_PARTIAL instead of silently degrading to STALE_SEED.
const keyMetaValues = new Map();
const keyMetaErrors = new Map();
for (let i = 0; i < allMetaKeys.length; i++) {
const r = results[allDataKeys.length + i];
if (r?.error) keyMetaErrors.set(allMetaKeys[i], r.error);
keyMetaValues.set(allMetaKeys[i], r?.result ?? null);
}
const classifyCtx = { keyStrens, keyErrors, keyMetaValues, keyMetaErrors, now };
const checks = {};
const counts = { ok: 0, warn: 0, onDemandWarn: 0, crit: 0 };
let totalChecks = 0;
const sources = [
[BOOTSTRAP_KEYS, { allowOnDemand: false }],
[STANDALONE_KEYS, { allowOnDemand: true }],
];
for (const [registry, opts] of sources) {
for (const [name, redisKey] of Object.entries(registry)) {
totalChecks++;
const entry = classifyKey(name, redisKey, opts, classifyCtx);
checks[name] = entry;
const bucket = STATUS_COUNTS[entry.status] ?? 'warn';
counts[bucket]++;
if (entry.status === 'EMPTY_ON_DEMAND') counts.onDemandWarn++;
}
}
// On-demand keys that simply haven't been requested yet should not flip
// overall to WARNING — they're warn-level only for visibility.
const realWarnCount = counts.warn - counts.onDemandWarn;
const critCount = counts.crit;
let overall;
if (critCount === 0 && realWarnCount === 0) overall = 'HEALTHY';
else if (critCount === 0) overall = 'WARNING';
// Degraded threshold scales with registry size so adding keys doesn't
// silently raise the page-out bar. ~3% of total keys (was hardcoded 3).
else if (critCount / totalChecks <= 0.03) overall = 'DEGRADED';
else overall = 'UNHEALTHY';
const httpStatus = 200;
if (overall !== 'HEALTHY') {
// problemKeys includes seedAgeMin for the snapshot (useful for post-mortem),
// but the dedupe signature uses only key:status (no age) so a long STALE_SEED
// window doesn't produce a new log entry on every poll.
const problemKeys = Object.entries(checks)
.filter(([, c]) => c.status !== 'OK' && c.status !== 'OK_CASCADE' && c.status !== 'EMPTY_ON_DEMAND')
.map(([k, c]) => `${k}:${c.status}${c.seedAgeMin != null ? `(${c.seedAgeMin}min)` : ''}`);
const sigKeys = Object.entries(checks)
.filter(([, c]) => c.status !== 'OK' && c.status !== 'OK_CASCADE' && c.status !== 'EMPTY_ON_DEMAND')
.map(([k, c]) => `${k}:${c.status}`)
.sort();
console.log('[health] %s problems=[%s]', overall, problemKeys.join(', '));
const snapshot = {
at: new Date(now).toISOString(),
status: overall,
critCount,
warnCount: realWarnCount,
problems: problemKeys,
};
// Dedupe: only LPUSH when the incident signature (status + problem set,
// excluding seedAgeMin) changes. Read the previous sig first, then write
// everything (last-failure + sig + LPUSH) in one atomic pipeline so the
// sig only advances when the LPUSH succeeds. If the pipeline fails, the
// sig stays stale and the next poll retries the append.
const sig = `${overall}|${sigKeys.join(',')}`;
const prevSigResult = await redisPipeline([['GET', 'health:failure-log-sig']], 4_000).catch(() => null);
const prevSig = prevSigResult?.[0]?.result ?? '';
const persistCmds = [
['SET', 'health:last-failure', JSON.stringify(snapshot), 'EX', 86400],
];
if (sig !== prevSig) {
persistCmds.push(
['LPUSH', 'health:failure-log', JSON.stringify(snapshot)],
['LTRIM', 'health:failure-log', 0, 49],
['EXPIRE', 'health:failure-log', 86400 * 7],
['SET', 'health:failure-log-sig', sig, 'EX', 86400],
);
}
const persist = redisPipeline(persistCmds, 4_000).catch(() => {});
if (ctx && typeof ctx.waitUntil === 'function') ctx.waitUntil(persist);
} else {
// Clear the sig on recovery so a recurrence of the same problem set
// after a healthy gap is logged as a new incident, not deduped against
// the previous one.
const clear = redisPipeline([['DEL', 'health:failure-log-sig']], 4_000).catch(() => {});
if (ctx && typeof ctx.waitUntil === 'function') ctx.waitUntil(clear);
}
const url = new URL(req.url);
const compact = url.searchParams.get('compact') === '1';
const body = {
status: overall,
summary: {
total: totalChecks,
ok: counts.ok,
// `warn` excludes on-demand-empty (cosmetic warns); `onDemandWarn` is
// surfaced separately so readers can reconcile against `overall`.
warn: realWarnCount,
onDemandWarn: counts.onDemandWarn,
crit: critCount,
},
checkedAt: new Date(now).toISOString(),
};
if (!compact) {
body.checks = checks;
} else {
const problems = {};
for (const [name, check] of Object.entries(checks)) {
if (check.status !== 'OK' && check.status !== 'OK_CASCADE') problems[name] = check;
}
if (Object.keys(problems).length > 0) body.problems = problems;
}
return new Response(JSON.stringify(body, null, compact ? 0 : 2), {
status: httpStatus,
headers,
});
}
Reference in New Issue View Git Blame Copy Permalink