mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
3d2dce3be10fa9a393d96a71ce6cebb6d3e3e925
3592 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Elie Habib
|
3d2dce3be1 |
feat(energy-atlas): promote Atlas map layers to FULL variant (§R #3 = B) (#3366)
* feat(energy-atlas): promote Atlas map layers to FULL variant (§R #3 = B)
Per plan §R/#3 decision B: the Redis-backed evidence registries
(75 gas + 75 oil pipelines, 200 storage facilities, 29 fuel shortages)
are now toggleable on the main worldmonitor.app map. Previously they
were hardcoded energy-variant-only, and FULL users who toggled
`pipelines: true` got the ~20-entry legacy static PIPELINES list.
Changes:
- `src/components/DeckGLMap.ts`: drop the `SITE_VARIANT === 'energy'`
gates at :1511-1541. The pipelines layer now always uses
`createEnergyPipelinesLayer()` (Redis-backed evidence registry);
`createPipelinesLayer` (legacy static) is left in the file as dead
code pending a separate cleanup PR that also retires
`src/config/pipelines.ts`. Storage and fuel-shortage layers are
now gated only on the variant's `mapLayers.storageFacilities` /
`mapLayers.fuelShortages` booleans.
- `src/config/panels.ts`: add `storageFacilities: false` +
`fuelShortages: false` to FULL_MAP_LAYERS (desktop + mobile) so
the keys exist for toggle dispatch; default off so users opt in.
- `src/config/map-layer-definitions.ts`: extend the `full` variant's
VARIANT_LAYER_ORDER to include `storageFacilities` and
`fuelShortages`, so `getAllowedLayerKeys('full')` admits them and
the layer picker surfaces them.
- `src/config/commands.ts`: add CMD+K toggles
`layer:storageFacilities` and `layer:fuelShortages` next to the
existing `layer:pipelines`.
Finance + commodity variants already had `pipelines: true`; they
now render the more comprehensive Redis-backed 150-entry dataset
instead of the ~20-entry legacy list. If a variant doesn't want
this, they set `pipelines: false` in their MAP_LAYERS config.
Part of docs/internal/energy-atlas-registry-expansion.md §R.
* fix(energy-atlas): restrict storageFacilities + fuelShortages to flat renderer
Reviewer (Codex) found two gaps in PR #3366:
1. GlobeMap 3D toggles did nothing. LAYER_REGISTRY declared both new
layers with the default ['flat', 'globe'] renderers, so the toggle
showed up in globe mode. But GlobeMap.ts has no rendering support:
ensureStaticDataForLayer (:2160) only handles cables/pipelines/etc.,
and the layer-channel map (:2484) has no entries for either. Users
in globe mode saw the toggle and got silent no-ops.
2. SVG/mobile fallback (Map.ts fullLayers at :381) also has no render
path for these data types. The existing cyberThreats precedent at
:387 documents this as an intentional DeckGL-only pattern.
Fix:
- Restrict both LAYER_REGISTRY entries to ['flat'] explicitly. The
layer picker hides the toggle in globe mode instead of exposing a
no-op. Comment points to the GlobeMap gap so a future globe-rendering
PR knows what to undo.
- Extend the existing cyberThreats note in Map.ts:387 to cover
storageFacilities + fuelShortages too, noting they're already
hidden from globe mode via the LAYER_REGISTRY restriction.
This is the smallest possible fix consistent with the pre-existing
pattern. Full globe-mode rendering for these layers is out of scope —
tracked separately as a follow-up.
* fix(energy-atlas): gate layer:* CMD+K by current renderer + DeckGL state
Reviewer follow-up on PR #3366: the previous fix restricted
LAYER_REGISTRY renderers to ['flat'] so the globe-mode layer picker
hides storageFacilities / fuelShortages toggles. But CMD+K was still
callable — SearchModal.matchCommands didn't filter `layer:*` commands
by renderer, so a user could CMD+K "storage layer" in globe or SVG
mode and trigger a silent no-op.
Fix — centralize "can this layer render right now?" in one helper:
- Add `deckGLOnly?: boolean` to LayerDefinition. `renderers: ['flat']`
is not enough because `'flat'` covers both DeckGL-flat and SVG-flat,
and the SVG/mobile fallback has no render path for either layer.
Mark both as `deckGLOnly: true`.
- New `isLayerExecutable(key, renderer, isDeckGLActive)` helper in
map-layer-definitions.ts. Returns true iff renderers include the
current renderer AND (if deckGLOnly) DeckGL is active.
- `SearchModal.setLayerExecutableFn(fn)`: caller-supplied predicate
used in both `matchCommands` (search results) and
`renderAllCommandsList` (full picker).
- `search-manager` wires the predicate using `ctx.map.isGlobeMode()`
+ `ctx.map.isDeckGLActive()`, and also adds a symmetric guard in
the `layer:` dispatch case so direct activations (keyboard
accelerator, programmatic invocation) bail the same way.
Pre-existing resilienceScore DeckGL gate at search-manager:494 kept as
a belt-and-suspenders — the new isLayerExecutable check already
covers it since resilienceScore has `renderers: ['flat']` (though it
lacks deckGLOnly). Left the specific check in place to avoid scope
creep on a working guard.
Typecheck clean, 6694/6694 tests pass.
* fix(energy-atlas): filter CMD+K layer commands by variant too
Greptile P2 on commit
|
||
|
Elie Habib
|
73cd8a9c92 |
feat(energy-atlas): EnergyDisruptionsPanel standalone timeline (§L #4) (#3378)
* feat(energy-atlas): EnergyDisruptionsPanel standalone timeline (§L #4) Closes gap #4 from docs/internal/energy-atlas-registry-expansion.md §L. Before this PR, the 52 disruption events in `energy:disruptions:v1` were only reachable by drilling into a specific pipeline or storage facility — PipelineStatusPanel and StorageFacilityMapPanel each render an asset-scoped slice of the log inside their drawers, but no surface listed the global event log. This panel makes the full log first-class. Shape: - Reverse-chronological table (newest first) of every event. - Filter chips: event type (sabotage, sanction, maintenance, mechanical, weather, war, commercial, other) + "ongoing only" toggle. - Row click dispatches the existing `energy:open-pipeline-detail` or `energy:open-storage-facility-detail` CustomEvent with `{assetId, highlightEventId}` — no new open-panel protocol introduced. Mirrors the CountryDeepDivePanel disruption row contract from PR #3377. - Uses `src/shared/disruption-timeline.ts` formatters (formatEventWindow, formatCapacityOffline, statusForEvent) that PipelineStatus/StorageFacilityMap already use — consistent UI across all three disruption surfaces. Wiring: - `src/components/EnergyDisruptionsPanel.ts` — new (~230 lines). - `src/components/index.ts` — export. - `src/app/panel-layout.ts` — `this.createPanel('energy-disruptions', () => new EnergyDisruptionsPanel())` alongside the other three atlas panels at :892. - `src/config/panels.ts` — add to `FULL_PANELS` (priority 2, next to fuel-shortages) + `ENERGY_PANELS` (priority 1, top tier) + `PANEL_CATEGORY_MAP.marketsFinance` list alongside the other atlas panels. - `src/config/commands.ts` — CMD+K entry `panel:energy-disruptions` with keywords matching the user vocabulary (sabotage, sanctions events, force majeure, drone strike, nord stream sabotage). Not done in this PR: - No new map pin layer — per plan §Q (Codex approved), disruptions stay a tabular/timeline surface; map assets (pipelines + storage) already show disruption markers on click. - No direct globe-mode or SVG-fallback rendering needs — panel is pure DOM, not a map layer. Test plan: - [x] npm run typecheck (clean) - [x] npm run test:data (6694/6694 pass) - [ ] Manual: CMD+K "disruption log" → panel opens with 52 events, newest first. Click "Sabotage" chip → narrows to sabotage events only. Click a Nord Stream row → PipelineStatusPanel opens with that event highlighted. * fix(energy-atlas): drop highlightEventId emission + respect empty-state (review P2) Two Codex P2 findings on this PR: 1. Row click dispatched `highlightEventId` but neither PipelineStatusPanel nor StorageFacilityMapPanel consumes it. The UI's implicit promise (event-specific highlighting) wasn't delivered — clickthrough was asset-generic, and the extra field on the wire was a misleading API surface. Fix: drop `highlightEventId` from the dispatched detail. Row click now opens the asset drawer with just {pipelineId, facilityId}, the fields the receivers actually consume. User sees the full disruption timeline for that asset and locates the event visually. A future PR can add real highlight support by: - drawers accept `highlightEventId` in their openDetailHandler - loadDetail stores it and renderDisruptionTimeline scrolls + emphasises the matching event - re-add `highlightEventId` to the dispatch here, symmetrically in CountryDeepDivePanel (which has the same wire emission) The internal `_eventId` parameter is kept as a plumb-through so that future work is a drawer-side change, not a re-plumb. 2. `events.length === 0` was conflated with `upstreamUnavailable` and triggered the error UI. The server contract (list-energy-disruptions handler) returns `upstreamUnavailable: false` with an empty events array when Redis is up but has no entries matching the filter — a legitimate empty state, not a fetch failure. Fix: gate `showError` on `upstreamUnavailable` alone. Empty results fall through to the normal render, where the table's `No events match the current filter` row already handles the case. Typecheck clean, test:data 6694/6694 pass. * fix(energy-atlas): event delegation on persistent content (review P1) Codex P1: Panel.setContent() debounces the DOM write by 150ms (see Panel.ts:1025), so attaching listeners in render() via `this.element.querySelector(...)` targets the STALE DOM — chips, rows, and the ongoing-toggle button are silently non-interactive. Visually the panel renders correctly after the debounce fires, but every click is permanently dead. Fix: register a single delegated click handler on `this.content` (persistent element) in the constructor. The handler uses `closest('[data-filter-type]')`, `closest('[data-toggle-ongoing]')`, and `closest('tr.ed-row')` to route by data-attribute. Works regardless of when setContent flushes or how many times render() re-rewrites the inner HTML. Also fixes Codex P2 on the same PR: filterEvents() was called twice per render (once for row HTML, again for filteredCount). Now computed once, reused. Trivial for 52 events but eliminates the redundant sort. Typecheck clean. * fix(energy-atlas): remap orphan disruption assetIds to real pipelines Two events referenced pipeline ids that do not exist in scripts/data/pipelines-oil.json: - cpc-force-majeure-2022: assetId "cpc-pipeline" → "cpc" - pdvsa-designation-2019: assetId "ve-petrol-2026-q1" → "venezuela-anzoategui-puerto-la-cruz" Without this, clicking those rows in EnergyDisruptionsPanel dead-ends at "Pipeline detail unavailable", so the panel shipped with broken navigation on real data. Mirrors the same fix on PR #3377 (gap #5a registry); applying it on this branch as well so PR #3378 is independently correct regardless of merge order. The two changes will dedupe cleanly on rebase since the edits are byte-identical. |
||
|
Elie Habib
|
7c0c08ad89 |
feat(energy-atlas): seed-side countries[] denorm on disruptions + CountryDeepDive row (§R #5 = B) (#3377)
* feat(energy-atlas): seed-side countries[] denorm + CountryDeepDive row (§R #5 = B)
Per plan §R/#5 decision B: denormalise countries[] at seed time on each
disruption event so CountryDeepDivePanel can filter events per country
without an asset-registry round trip. Schema join (pipeline/storage
→ event.assetId) happens once in the weekly cron, not on every panel
render. The alternative (client-side join) was rejected because it
couples UI logic to asset-registry internals and duplicates the join
for every surface that wants a per-country filter.
Changes:
- `proto/.../list_energy_disruptions.proto`: add `repeated string
countries = 15` to EnergyDisruptionEntry with doc comment tying it
to the plan decision and the always-non-empty invariant.
- `scripts/_energy-disruption-registry.mjs`:
• Load pipeline-gas + pipeline-oil + storage-facilities registries
once per seed cycle; index by id.
• `deriveCountriesForEvent()` resolves assetId to {fromCountry,
toCountry, transitCountries} (pipeline) or {country} (storage),
deduped + alpha-sorted so byte-diff stability holds.
• `buildPayload()` attaches the computed countries[] to every
event before writing.
• `validateRegistry()` now requires non-empty countries[] of
ISO2 codes. Combined with the seeder's `emptyDataIsFailure:
true`, this surfaces orphaned assetIds loudly — the next cron
tick fails validation and seed-meta stays stale, tripping
health alarms.
- `scripts/data/energy-disruptions.json`: fix two orphaned assetIds
that the new join caught:
• `cpc-force-majeure-2022`: `cpc-pipeline` → `cpc` (matches the
entry in pipelines-oil.json).
• `pdvsa-designation-2019`: `ve-petrol-2026-q1` (non-existent) →
`venezuela-anzoategui-puerto-la-cruz`.
- `server/.../list-energy-disruptions.ts`: project countries[] into
the RPC response via coerceStringArray. Legacy pre-denorm rows
surface as empty array (always present on wire, length 0 => old).
- `src/components/CountryDeepDivePanel.ts`: add 4th Atlas row —
"Energy disruptions in {iso2}" — filtered by `iso2 ∈ countries[]`.
Failure is silent; EnergyDisruptionsPanel (upcoming) is the
primary disruption surface.
- `tests/energy-disruptions-registry.test.mts`: switch to validating
the buildPayload output (post-denorm), add §R #5 B invariant
tests, plus a raw-JSON invariant ensuring curators don't hand-edit
countries[] (it's derived, not declared).
Proto regen note: `make generate` currently fails with a duplicate
openapi plugin collision in buf.gen.yaml (unrelated bug — 3 plugin
entries emit to the same out dir). Worked around by temporarily
trimming buf.gen.yaml to just the TS plugins for this regen. Added
only the `countries: string[]` wire field to both service_client and
service_server; no other generated-file drift in this PR.
* chore(proto): regenerate openapi specs for countries[] field
Runs `make generate` with the sebuf v0.11.1 plugin now correctly
resolved via the PATH fix (cherry-picked from fix/makefile-generate-path-prefix).
The new `countries` field on EnergyDisruptionEntry propagates into:
- docs/api/SupplyChainService.openapi.yaml (primary per-service spec)
- docs/api/SupplyChainService.openapi.json (machine-readable variant)
- docs/api/worldmonitor.openapi.yaml (consolidated bundle)
No TypeScript drift beyond the already-committed service_client.ts /
service_server.ts updates in
|
||
|
Elie Habib
|
a04c53fe26 |
fix(build): pin sebuf plugin via PATH in make generate (#3371)
* fix(build): pin sebuf plugin via PATH in `make generate`
Without this, developers who have an older sebuf protoc-gen-openapiv3
binary installed via a package manager (Homebrew ships v0.7.0 at
/opt/homebrew/bin) hit this failure on a fresh `make generate`:
Failure: file ".../AviationService.openapi.yaml" was generated
multiple times: once by plugin "protoc-gen-openapiv3" and again
by plugin "protoc-gen-openapiv3"
Root cause: `buf.gen.yaml` declares three `protoc-gen-openapiv3`
invocations — the default yaml, a `format=json` variant, and a
`bundle_only=true, strategy: all` unified bundle. Sebuf v0.11.x
honors both `format=json` (emits .json extension) and `bundle_only=true`
(skips per-service emission), so the three invocations write distinct
files. Sebuf v0.7.x does NOT honor either option — it silently emits
the same per-service .yaml filenames from all three plugins and buf
rejects the collision.
`Makefile: install-plugins` installs v0.11.1 (SEBUF_VERSION) to
$HOME/go/bin. But the `generate` target doesn't prepend that to PATH,
so `which protoc-gen-openapiv3` resolves to the stale Homebrew binary
for anyone with both installed.
Verified by `go version -m`:
/opt/homebrew/bin/protoc-gen-openapiv3 — mod sebuf v0.7.0
/Users/eliehabib/go/bin/protoc-gen-openapiv3 — mod sebuf v0.11.1
Fix: prepend $$HOME/go/bin to PATH in the `generate` recipe. Matches
what .husky/pre-push:151-153 already does before invoking this target,
so CI and local behavior converge. No sebuf upstream bug.
* fix(build): follow GOBIN-then-GOPATH/bin for plugin PATH prefix
Reviewer (Codex) on PR #3371: the previous patch hardcoded
\$HOME/go/bin, which is only the default fallback when GOBIN is unset
AND GOPATH defaults to \$HOME/go. On machines with a custom GOBIN or
a non-default GOPATH, `go install` targets a different directory — so
hardcoding \$HOME/go/bin can force a stale binary from there to win
over the freshly-installed SEBUF_VERSION sitting at the actual install
location.
Fix: resolve the install dir the same way `go install` does:
GOBIN first, then GOPATH/bin.
Shell expression: `go env GOBIN` returns an empty string (exit 0) when
unset, so `||` alone doesn't cascade. Using explicit `[ -n "$gobin" ]`
instead.
Also dropped the misleading comment that claimed the pre-push hook
used the same rule — it still hardcodes \$HOME/go/bin. Called that out
in a note, but left the hook alone because its PATH prepend is
belt-and-suspenders (only matters for locating `buf` itself; the
Makefile's own recipe-level prepend decides plugin resolution).
Verified on a machine with empty GOBIN:
resolved → /Users/eliehabib/go/bin
And \`make generate\` succeeds without manual PATH overrides.
* fix(build): use first GOPATH entry for plugin PATH prefix
Reviewer (Codex) on commit
|
||
|
Sebastien Melki
|
e68a7147dd |
chore(api): sebuf migration follow-ups (post-#3242) (#3287)
* chore(api-manifest): rewrite brief-why-matters reason as proper internal-helper justification Carried in from #3248 merge as a band-aid (called out in #3242 review followup checklist item 7). The endpoint genuinely belongs in internal-helper — RELAY_SHARED_SECRET-bearer auth, cron-only caller, never reached by dashboards or partners. Same shape constraint as api/notify.ts. Replaces the apologetic "filed here to keep the lint green" framing with a proper structural justification: modeling it as a generated service would publish internal cron plumbing as user-facing API surface. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lint): premium-fetch parity check for ServiceClients (closes #3279) Adds scripts/enforce-premium-fetch.mjs — AST-walks src/, finds every `new <ServiceClient>(...)` (variable decl OR `this.foo =` assignment), tracks which methods each instance actually calls, and fails if any called method targets a path in src/shared/premium-paths.ts PREMIUM_RPC_PATHS without `{ fetch: premiumFetch }` on the constructor. Per-call-site analysis (not class-level) keeps the trade/index.ts pattern clean — publicClient with globalThis.fetch + premiumClient with premiumFetch on the same TradeServiceClient class — since publicClient never calls a premium method. Wired into: - npm run lint:premium-fetch - .husky/pre-push (right after lint:rate-limit-policies) - .github/workflows/lint-code.yml (right after lint:api-contract) Found and fixed three latent instances of the HIGH(new) #1 class from #3242 review (silent 401 → empty fallback for signed-in browser pros): - src/services/correlation-engine/engine.ts — IntelligenceServiceClient built with no fetch option called deductSituation. LLM-assessment overlay on convergence cards never landed for browser pros without a WM key. - src/services/economic/index.ts — EconomicServiceClient with globalThis.fetch called getNationalDebt. National-debt panel rendered empty for browser pros. - src/services/sanctions-pressure.ts — SanctionsServiceClient with globalThis.fetch called listSanctionsPressure. Sanctions-pressure panel rendered empty for browser pros. All three swap to premiumFetch (single shared client, mirrors the supply-chain/index.ts justification — premiumFetch no-ops safely on public methods, so the public methods on those clients keep working). Verification: - lint:premium-fetch clean (34 ServiceClient classes, 28 premium paths, 466 src/ files analyzed) - Negative test: revert any of the three to globalThis.fetch → exit 1 with file:line and called-premium-method names - typecheck + typecheck:api clean - lint:api-contract / lint:rate-limit-policies / lint:boundaries clean - tests/sanctions-pressure.test.mjs + premium-fetch.test.mts: 16/16 pass Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(military): fetchStaleFallback NEG_TTL=30s parity (closes #3277) The legacy /api/military-flights handler had NEG_TTL = 30_000ms — a short suppression window after a failed live + stale read so we don't Redis-hammer the stale key during sustained relay+seed outages. Carried into the sebuf list-military-flights handler: - Module-scoped `staleNegUntil` timestamp (per-isolate on Vercel Edge, which is fine — each warm isolate gets its own 30s suppression window). - Set whenever fetchStaleFallback returns null (key missing, parse fail, empty array after staleToProto filter, or thrown error). - Checked at the entry of fetchStaleFallback before doing the Redis read. - Test seam `_resetStaleNegativeCacheForTests()` exposed for unit tests. Test pinned in tests/redis-caching.test.mjs: drives a stale-empty cycle three times — first read hits Redis, second within window doesn't, after test-only reset it does again. Verified: 18/18 redis-caching tests pass, typecheck:api clean, lint:premium-fetch clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lint): rate-limit-policies regex → import() (closes #3278) The previous lint regex-parsed ENDPOINT_RATE_POLICIES from the source file. That worked because the literal happens to fit a single line per key today, but a future reformat (multi-line key wrap, formatter swap, etc.) would silently break the lint without breaking the build — exactly the failure mode that's worse than no lint at all. Fix: - Export ENDPOINT_RATE_POLICIES from server/_shared/rate-limit.ts. - Convert scripts/enforce-rate-limit-policies.mjs to async + dynamic import() of the policy object directly. Same TS module that the gateway uses at runtime → no source-of-truth drift possible. - Run via tsx (already a dev dep, used by test:data) so the .mjs shebang can resolve a .ts import. - npm script swapped to `tsx scripts/...`. .husky/pre-push uses `npm run lint:rate-limit-policies` so no hook change needed. Verified: - Clean: 6 policies / 182 gateway routes. - Negative test (rename a key to the original sanctions typo /api/sanctions/v1/lookup-entity): exit 1 with the same incident- attributed remedy message as before. - Reformat test (split a single-line entry across multiple lines): still passes — the property is what's read, not the source layout. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(shipping/v2): alertThreshold: 0 preserved; drop dead validation branch (#3242 followup) Before: alert_threshold was a plain int32. proto3 scalar default is 0, so the handler couldn't distinguish "partner explicitly sent 0 (deliver every disruption)" from "partner omitted the field (apply legacy default 50)" — both arrived as 0 and got coerced to 50 by `> 0 ? : 50`. Silent intent-drop for any partner who wanted every alert. The subsequent `alertThreshold < 0` branch was also unreachable after that coercion. After: - Proto field is `optional int32 alert_threshold` — TS type becomes `alertThreshold?: number`, so omitted = undefined and explicit 0 stays 0. - Handler uses `req.alertThreshold ?? 50` — undefined → 50, any number passes through unchanged. - Dead `< 0 || > 100` runtime check removed; buf.validate `int32.gte = 0, int32.lte = 100` already enforces the range at the wire layer. Partner wire contract: identical for the omit-field and 1..100 cases. Only behavioural change is explicit 0 — previously impossible to request, now honored per proto3 optional semantics. Scoped `buf generate --path worldmonitor/shipping/v2` to avoid the full- regen `@ts-nocheck` drift Seb documented in the #3242 PR comments. Re-applied `@ts-nocheck` on the two regenerated files manually. Tests: - `alertThreshold 0 coerces to 50` flipped to `alertThreshold 0 preserved`. - New test: `alertThreshold omitted (undefined) applies legacy default 50`. - `rejects > 100` test removed — proto/wire validation handles it; direct handler calls intentionally bypass wire and the handler no longer carries a redundant runtime range check. Verified: 18/18 shipping-v2-handler tests pass, typecheck + typecheck:api clean, all 4 custom lints clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(shipping/v2): document missing webhook delivery worker + DNS-rebinding contract (#3242 followup) #3242 followup checklist item 6 from @koala73 — sanity-check that the delivery worker honors the re-resolve-and-re-check contract that isBlockedCallbackUrl explicitly delegates to it. Audit finding: no delivery worker for shipping/v2 webhooks exists in this repo. Grep across the entire tree (excluding generated/dist) shows the only readers of webhook:sub:* records are the registration / inspection / rotate-secret handlers themselves. No code reads them and POSTs to the stored callbackUrl. The delivery worker is presumed to live in Railway (separate repo) or hasn't been built yet — neither is auditable from this repo. Refreshes the comment block at the top of webhook-shared.ts to: - explicitly state DNS rebinding is NOT mitigated at registration - spell out the four-step contract the delivery worker MUST follow (re-validate URL, dns.lookup, re-check resolved IP against patterns, fetch with resolved IP + Host header preserved) - flag the in-repo gap so anyone landing delivery code can't miss it Tracking the gap as #3288 — acceptance there is "delivery worker imports the patterns + helpers from webhook-shared.ts and applies the four steps before each send." Action moves to wherever the delivery worker actually lives (Railway likely). No code change. Tests + lints unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci(lint): add rate-limit-policies step (greptile P1 #3287) Pre-push hook ran lint:rate-limit-policies but the CI workflow did not, so fork PRs and --no-verify pushes bypassed the exact drift check the lint was added to enforce (closes #3278). Adding it right after lint:api-contract so it runs in the same context the lint was designed for. * refactor(lint): premium-fetch regex → import() + loop classRe (greptile P2 #3287) Two fragilities greptile flagged on enforce-premium-fetch.mjs: 1. loadPremiumPaths regex-parsed src/shared/premium-paths.ts with /'(\/api\/[^']+)'/g — same class of silent drift we just removed from enforce-rate-limit-policies in #3278. Reformatting the source Set (double quotes, spread, helper-computed entries) would drop paths from the lint while leaving the runtime untouched. Fix: flip the shebang to `#!/usr/bin/env -S npx tsx` and dynamic-import PREMIUM_RPC_PATHS directly, mirroring the rate-limit pattern. package.json lint:premium-fetch now invokes via tsx too so the npm-script path matches direct execution. 2. loadClientClassMap ran classRe.exec once, silently dropping every ServiceClient after the first if a file ever contained more than one. Current codegen emits one class per file so this was latent, but a template change would ship un-linted classes. Fix: collect every class-open match with matchAll, slice each class body with the next class's start as the boundary, and scan methods per-body so method-to-class binding stays correct even with multiple classes per file. Verification: - lint:premium-fetch clean (34 classes / 28 premium paths / 466 files — identical counts to pre-refactor, so no coverage regression). - Negative test: revert src/services/economic/index.ts to globalThis.fetch → exit 1 with file:line, bound var name, and premium method list (getNationalDebt). Restore → clean. - lint:rate-limit-policies still clean. * fix(shipping/v2): re-add alertThreshold handler range guard (greptile nit 1 #3287) Wire-layer buf.validate enforces 0..100, but direct handler invocation (internal jobs, test harnesses, future transports) bypasses it. Cheap invariant-at-the-boundary — rejects < 0 or > 100 with ValidationError before the record is stored. Tests: restored the rejects-out-of-range cases that were dropped when the branch was (correctly) deleted as dead code on the previous commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lint): premium-fetch method-regex → TS AST (greptile nits 2+5 #3287) loadClientClassMap: The method regex `async (\w+)\s*\([^)]*\)\s*:\s*Promise<[^>]+>\s*\{\s*let path = "..."` assumed (a) no nested `)` in arg types, (b) no nested `>` in the return type, (c) `let path = "..."` as the literal first statement. Any codegen template shift would silently drop methods with the lint still passing clean — the same silent-drift class #3287 just closed on the premium-paths side. Now walks the service_client.ts AST, matches `export class *ServiceClient`, iterates `MethodDeclaration` members, and reads the first `let path: string = '...'` variable statement as a StringLiteral. Tolerant to any reformatting of arg/return types or method shape. findCalls scope-blindness: Added limitation comment — the walker matches `<varName>.<method>()` anywhere in the file without respecting scope. Two constructions in different function scopes sharing a var name merge their called-method sets. No current src/ file hits this; the lint errs cautiously (flags both instances). Keeping the walker simple until scope-aware binding is needed. webhook-shared.ts: Inlined issue reference (#3288) so the breadcrumb resolves without bouncing through an MDX that isn't in the diff. Verification: - lint:premium-fetch clean — 34 classes / 28 premium paths / 489 files. Pre-refactor: 34 / 28 / 466. Class + path counts identical; file bump is from the main-branch rebase, not the refactor. - Negative test: revert src/services/economic/index.ts premiumFetch → globalThis.fetch. Lint exits 1 at `src/services/economic/index.ts:64:7` with `premium method(s) called: getNationalDebt`. Restore → clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lint): rate-limit OpenAPI regex → yaml parser (greptile nit 3 #3287) Input side (ENDPOINT_RATE_POLICIES) was flipped to live `import()` in |
||
|
Elie Habib
|
1dc807e70f |
docs(resilience): PR 4a — SWF classification rubric (tiers + precedents, no manifest changes) (#3376)
* docs(resilience): PR 4a — SWF classification rubric (tiers + precedents, no manifest changes)
PR 4a of cohort-audit plan 2026-04-24-002. First half of the plan's PR 4
(full-manifest re-rate) split into:
- PR 4a (this): pure documentation — central rubric defining tiers
+ concrete precedents per axis. No manifest changes.
- PR 4b (deferred): apply the rubric to revise specific coefficients
in `scripts/shared/swf-classification-manifest.yaml`. Behaviour-
changing; belongs in a separate PR with cohort snapshots and
methodology review.
This split addresses the plan's concern that PR 4 "may not be outcome-
predetermined" by separating the evaluative framework from its
application. PR 4a makes every current manifest value evaluable against
a benchmark; PR 4b applies the benchmark.
Shipped
- `docs/methodology/swf-classification-rubric.md` — new doc.
Sections:
1. Introduction + scope (rubric vs manifest boundary)
2. Access axis: 5 named tiers (0.1, 0.3, 0.5, 0.7, 0.9) w/
concrete precedents per tier, plus edge cases for
fiscal-rule caps (Norway GPFG) and state holding
companies (Temasek)
3. Liquidity axis: 6 tiers (0.1, 0.3, 0.5, 0.7, 0.9, 1.0) w/
precedents + listed-vs-directly-owned real-estate edge case
4. Transparency axis: 6 tiers grounded in LM Transparency
Index + IFSWF membership + annual-report granularity, plus
edge cases for LM=10 w/o holdings-level disclosure and
sealed filings (KIA)
5. Current manifest × rubric alignment — 24 coefficients reviewed;
6 flagged as "arguably higher/lower under the rubric" with
directional-impact analysis marked INFORMATIONAL, not
motivation for revision
6. How-to-use playbook for manifest-edit PRs (add/revise/rubric-
revise workflows)
Findings (informational only — no PR changes)
Six ratings flagged as potentially under-/over-stated against the
rubric. Per the plan's anti-pattern note (rank-targeted acceptance
criteria), the flags are INFORMATIONAL: a future manifest-edit PR
should revise only when the rubric + cited evidence support the
change, not to hit a target ranking.
Flagged (with directional impact if revised upward):
- Mubadala access 0.4 → arguably 0.5; transparency 0.6 → 0.7
(haircut 0.12 → 0.175, +46% access × transparency product)
- PIF access 0.4 → arguably 0.5; liquidity 0.4 → arguably 0.3
(net small effect — opposite directions partially cancel)
- KIA transparency 0.4 → arguably 0.5 (haircut +25%)
- QIA access 0.4 → arguably 0.5; transparency 0.4 → arguably 0.5
(haircut +56%)
- GIC access 0.6 → arguably 0.7 (haircut +17%)
Not flagged: GPFG, ADIA, Temasek (all 9 coefficients align with
their rubric tiers).
Verified
- `npm run test:data` — 6694 pass / 0 fail (unchanged — pure docs PR)
- `npm run typecheck` / `typecheck:api` — green
- `npm run lint:md` — clean
Not in this PR
- Manifest coefficient changes (PR 4b)
- Cohort-sanity snapshot before/after (PR 4b)
- Live-data audit of IFSWF engagement + LM index current values
(requires web fetch — not in scope for a doc PR)
* fix(resilience): PR 4a review — resolve GIC/ADIA rubric contradictions + flag-count
Addresses P1 + 2 P2 Greptile findings on #3376 (draft).
1. **P1 — GIC tier contradiction.** GIC was listed as a canonical 0.7
("Explicit stabilization with rule") precedent AND rated 0.6 in
the alignment table with an "arguably 0.7" note. That inconsistency
makes the rubric unusable as-is for PR 4b review. Removed GIC from
the 0.7 precedent list and explicitly marked it as a 0.7 *candidate*
(pending PR 4b evaluation), not a 0.7 *precedent*. KIA General
Reserve Fund stays as the canonical 0.7 example; Norway GPFG
remains the borderline case for fiscal-rule caps.
2. **P2 — ADIA liquidity midpoint inconsistency.** Methodology text
said the rubric uses "midpoint" for ranged disclosures and cited
ADIA 55-70% → 0.7 tier. But midpoint(55-70) = 62.5%, which sits
in the 0.5 tier band (50-65%). Fixed the methodology to state the
rubric uses the **upper-bound** of a disclosed range (fund's own
statement of maximum public-market allocation), which keeps ADIA
at 0.7 tier (70% upper bound fits 65-85% band). Added forward-
compatibility note: if future ADIA disclosures tighten the range
so the upper bound drops below 65%, the rubric directs the rating
to 0.5.
3. **P2 — Flag-count header.** "(6 of 24 coefficients)" was wrong;
the enumeration below lists 8 coefficients across 5 funds.
Corrected to "8 coefficients across 5 funds" with the fund-by-fund
count inline so the header math is self-verifying.
Verified
- `npm run lint:md` — clean
- `npm run typecheck` — green (pure docs PR, no behaviour change)
This PR remains in draft pending #3380 (PR 3A — net-imports denominator)
merge per the plan's PR 4 → after PR 3A sequencing.
|
||
|
Elie Habib
|
b4198a52c3 |
docs(resilience): PR 5.1 — sanctions construct audit (designated-party domicile question) (#3375)
* docs(resilience): PR 5.1 — sanctions construct audit (designated-party domicile question) PR 5.1 of cohort-audit plan 2026-04-24-002. Stacked on PR 5.3 (#3374) so the known-limitations.md section append is additive. Read-only static audit of scoreTradeSanctions + the sanctions:country-counts:v1 seed — framed around the Codex-reformulated construct question: should designated-party domicile count penalize resilience? Findings 1. The count is "OFAC-designated-party domicile locations," NOT "sanctions against this country." Seeder (`scripts/seed-sanctions- pressure.mjs:85-93`) parses OFAC Advanced XML SDN + Consolidated, extracts each designated party's Locations, and increments `map[countryCode]` by 1 for every location country on that party. 2. The count conflates three semantically distinct categories a resilience construct might treat differently: (a) Country-level sanction target (NK SDN listings) — correct penalty (b) Domiciled sanctioned entity (RU bank in Moscow, post-2022) — debatable, country hosts the actor (c) Transit / shell entity (UAE trading co listed under SDGT for Iran evasion; CY SPV for a Russian oligarch) — country is NOT the target, but takes the penalty 3. Observed GCC cohort impact: AE scores 54 vs KW/QA 82. The −28 gap is almost entirely driven by category (c) listings — AE is a financial hub where sanctioned parties incorporate shells. 4. Three options documented for the construct decision (NOT decided in this PR): - Option 1: Keep flat count (status quo, defensible via secondary- sanctions / FATF argument) - Option 2: Program-weighted count — weight DPRK/IRAN/SYRIA/etc. at 1.0, SDGT/SDNTK/CYBER/etc. at 0.3-0.5. Recommended; seeder already captures `programs` per entry — data is there, scorer just doesn't read it. - Option 3: Transit-hub exclusion list (AE, SG, HK, CY, VG, KY) — brittle + normative, not recommended 5. Recommendation documented: Option 2. Implementation deferred to a separate methodology-decision PR (outside auto-mode authority). Shipped - `docs/methodology/known-limitations.md` — new section extending the file: "tradeSanctions — designated-party domicile construct question." Covers what the count represents, the three categories with examples, observed GCC impact, three options w/ trade-offs, recommendation, follow-up audit list (entity-sample gated on API-key access), and file references. - `tests/resilience-sanctions-field-mapping.test.mts` (new) — 10 regression-guard tests pinning CURRENT behavior: 1-6. normalizeSanctionCount piecewise anchors: count=0→100, 1→90, 10→75, 50→50, 200→25, 500→≤1 7. Monotonicity: strictly decreasing across the ramp 8. Country absent from map defaults to count=0 → score 100 (intentional "no designated parties here" semantics) 9. Seed outage (raw=null) → null score slot, NOT imputed (protects against silent data-outage scoring) 10. Construct anchor: count=1 is exactly 10 points below count=0 (pins the "first listing drops 10" design choice) Verified - `npx tsx --test tests/resilience-sanctions-field-mapping.test.mts` — 10 pass / 0 fail - `npm run test:data` — 6721 pass / 0 fail - `npm run typecheck` / `typecheck:api` — green - `npm run lint` / `lint:md` — clean * fix(resilience): PR 5.1 review — tighten count=500 assertion; clarify weightedBlend weights Addresses 2 P2 Greptile findings on #3375: 1. Tighten count=500 assertion. Was `<= 1` with a comment stating the exact value is 0. That loose bound silently tolerates roundScore / boundary drift that would be the very signal this regression guard exists to catch. Changed to strict equality `=== 0`. 2. Clarify the "zero weight" comment on the sanctions-only harness. The other slots DO contribute their declared weights (0.15 + 0.15 + 0.25 = 0.55) to weightedBlend's `totalWeight` denominator — only `availableWeight` (the score-computation denominator) drops to 0.45 because their score is null. The previous comment elided this distinction and could mislead a reader into thinking the null slots contributed nothing at all. Expanded to state exactly how `coverage` and `score` each behave. Verified - `npx tsx --test tests/resilience-sanctions-field-mapping.test.mts` — 10 pass / 0 fail (count=500 now pins the exact 0 floor) |
||
|
Elie Habib
|
a97ba83833 |
docs(resilience): PR 5.3 — foodWater scorer audit (construct-deterministic GCC identity) (#3374)
* docs(resilience): PR 5.3 — foodWater scorer audit (construct-deterministic GCC identity) PR 5.3 of cohort-audit plan 2026-04-24-002. Stacked on PR 5.2 (#3373) so the known-limitations.md section append is additive. Read-only static audit of scoreFoodWater. Findings 1. The observed GCC-all-score-53 is CONSTRUCT-DETERMINISTIC, not a regional-default leak. Pinned mathematically: - IPC/HDX doesn't publish active food-crisis data for food-secure states → scorer's fao-null branch imputes IMPUTE.ipcFood=88 (class='stable-absence', cov=0.7) at combined weight 0.6 - WB indicator ER.H2O.FWST.ZS (labelled 'water stress') for GCC is EXTREME (KW ~3200%, BH ~3400%, UAE ~2080%, QA ~770%) — all clamp to sub-score 0 under the scorer's lower-better 0..100 normaliser at weight 0.4 - Blended with peopleInCrisis=0 (fao block present with zero): (100 * 0.45 + 0 * 0.4) / (0.45 + 0.4) = 45 / 0.85 ≈ 53 Every GCC country has the same inputs → same outputs. That's construct math, not a regional lookup. 2. Indicator-keyword routing is code-correct. `'water stress'`, `'withdrawal'`, `'dependency'` route to lower-better; `'availability'`, `'renewable'`, `'access'` route to higher-better; unrecognized indicators fall through to a value-range heuristic with a WARN log. 3. No bug or methodology decision required. The 53-all-GCC output is a correct summary statement: "non-crisis food security + severe water-withdrawal stress." A future construct decision might split foodWater into separate food and water dims so one saturated sub-signal doesn't dominate the combined dim for desert economies — but that's a construct redesign, not a bug. Shipped - `docs/methodology/known-limitations.md` — extended with a new section documenting the foodWater audit findings, the exact blend math that yields ~53 for GCC, cohort-determinism vs regional-default, and a follow-up data-side spot-check list gated on API-key access. - `tests/resilience-foodwater-field-mapping.test.mts` — 8 new regression-guard tests: 1. indicator='water stress' routes to lower-better 2. GCC extreme-withdrawal anchor (value=2000 → blended score 53) 3. indicator='renewable water availability' routes to higher-better 4. fao=null with static record → imputes 88; imputationClass=null because observed AQUASTAT wins (weightedBlend T1.7 rule) 5. fully-imputed (fao=null + aquastat=null) surfaces imputationClass='stable-absence' 6. static-record absent entirely → coverage=0, NOT impute 7. Cohort determinism — identical inputs → identical scores 8. Different water-profile inputs → different scores (rules out regional-default hypothesis) Verified - `npx tsx --test tests/resilience-foodwater-field-mapping.test.mts` — 8 pass / 0 fail - `npm run test:data` — 6711 pass / 0 fail (PR 5.2's 9 + PR 5.3's 8 = 17 new stacked) - `npm run typecheck` / `typecheck:api` — green - `npm run lint` / `lint:md` — clean * fix(resilience): PR 5.3 review — pin IMPUTE branch for GCC anchor; fix comment math Addresses 3 P2 Greptile findings on #3374 — all variations of the same root cause: the test fixture + doc described two different code paths that coincidentally both produce ~53 for GCC inputs. Changes 1. GCC anchor test now drives the IMPUTE branch (`fao: null`), matching what the static seeder emits for GCC in production. The else branch (`fao: { peopleInCrisis: 0 }`) happens to converge on ~52.94 by coincidence but is NOT the live code path for GCC. 2. Doc finding #4 updated to show the IMPUTE-branch math `(88×0.6 + 0×0.4) / 1.0 = 52.8 → 53` and explicitly notes the else-branch convergence as a coincidence — not the construct's intent. 3. Comment math off-by-one fix at line 107: (88×0.6 + 80×0.4) / (0.6+0.4) = 52.8 + 32.0 = 84.8 → 85 (was incorrectly stated as 85.6 → 86) Test assertion `>= 80 && <= 90` still accepts 85 so behaviour is unchanged; this was a comment-only error that would have misled anyone reproducing the math by hand. Verified - `npx tsx --test tests/resilience-foodwater-field-mapping.test.mts` — 8 pass / 0 fail (IMPUTE-branch anchor test produces 53 as expected) - `npm run lint:md` — clean Also rebased onto updated #3373 (which landed a backtick-escape fix). |
||
|
Elie Habib
|
6807a9c7b9 |
docs(resilience): PR 5.2 — displacement field-mapping audit + known-limitations (#3373)
* docs(resilience): PR 5.2 — displacement field-mapping audit + known-limitations
PR 5.2 of cohort-audit plan 2026-04-24-002. Read-only static audit of
the UNHCR displacement field mapping consumed by scoreSocialCohesion,
scoreBorderSecurity, and scoreStateContinuity.
Findings
1. Field mapping is CODE-CORRECT. The plan's concern — that
`totalDisplaced` might inadvertently include labor migrants — is
negative at the source. The UNHCR Population API does not publish
labor migrant data at all; it covers only four categories
(refugees, asylum seekers, IDPs, stateless), all of which the
seeder sums correctly. Labor-migrant-dominated cohorts (GCC, SG)
legitimately register as "no UNHCR footprint" — that's UNHCR
semantics, not a bug.
2. NEW finding during audit — `scoreBorderSecurity` fallback at
_dimension-scorers.ts:1412 is effectively dead code. The
`hostTotal ?? totalDisplaced` fallback never fires in production
for two compounding reasons:
(a) `safeNum(null)` returns 0 (JS `Number(null) === 0`), so the
`??` short-circuits on 0 — the nullish-coalescing only falls
back on null/undefined.
(b) `scripts/seed-displacement-summary.mjs` ALWAYS writes
`hostTotal: 0` explicitly for origin-only countries (lines
141-144). There's no production shape where `hostTotal` is
undefined, so the `??` can never select the fallback path.
Observable consequence: origin-only high-outflow countries
(Syria, Venezuela, Ukraine, Afghanistan) score 100 on
borderSecurity's displacement sub-component (35% of the dim
blend). The outflow signal is effectively silenced.
3. NOT fixing this in this PR. A one-line change (`||` or an
explicit `> 0` check) would flip the borderSecurity score for
~6 high-outflow origin countries by a material amount — a
methodology change, not a pure bug-fix. Belongs in a construct-
decision PR with before/after cohort snapshots. Opening this as
a follow-up discussion instead of bundling into an audit-doc PR.
Shipped
- `docs/methodology/known-limitations.md` — new file. Sections:
"Displacement field-mapping" covering source semantics (what
UNHCR provides vs does not), the GCC labor-migrant-cohort
implication, the `??` short-circuit finding, and the decision
to not fix in this PR. Includes a follow-up audit list of 11
countries (high host-pressure + high origin-outflow + labor-
migrant cohorts) for a live-data spot-check against UNHCR
Refugee Data Finder — gated on API-key access.
- `tests/resilience-displacement-field-mapping.test.mts` —
9-test regression guard. Pins:
(1) `totalDisplaced` = sum of all four UNHCR categories;
(2) `hostTotal` = asylum-side sum (no IDPs/stateless);
(3) stateless population flows into totalDisplaced (guards
against a future seeder refactor that drops the term);
(4) labor-migrant-cohort (UNHCR-empty) entry scores 100 on
the displacement sub-component — the correct-per-UNHCR-
semantics outcome, intentionally preserved;
(5) CURRENT scoreBorderSecurity behaviour: hostTotal=0
short-circuits `??` (Syria-pattern scores 100);
(6) `??` fallback ONLY fires when hostTotal is undefined
(academic; seeder never emits this shape today);
(7) `safeNum(null)` returns 0 quirk pinned as a numeric-
coercion contract;
(8) absent-from-UNHCR country imputes `stable-absence`;
(9) scoreStateContinuity reads `totalDisplaced` origin-side.
Verified
- `npx tsx --test tests/resilience-displacement-field-mapping.test.mts` — 9 pass / 0 fail
- `npm run test:data` — 6703 pass / 0 fail
- `npm run typecheck` / `typecheck:api` — green
- `npm run lint` / `lint:md` — no warnings on new files
* fix(resilience): PR 5.2 review — escape backticks in assertion message
Addresses Greptile P2 on #3373. The unescaped backticks around the
nullish-coalescing operator in a template literal caused JavaScript to
parse the string as 'prefix' ?? 'suffix' — truncating the assertion
message to the prefix alone on failure. Escaping the backticks preserves
the full diagnostic so a future regression shows the complete context.
Semantics unchanged; test still passes.
|
||
|
Elie Habib
|
184e82cb40 |
feat(resilience): PR 3A — net-imports denominator for sovereignFiscalBuffer (#3380)
PR 3A of cohort-audit plan 2026-04-24-002. Construct correction for
re-export hubs: the SWF rawMonths denominator was gross imports, which
double-counted flow-through trade that never represents domestic
consumption. Net-imports fix:
rawMonths = aum / (grossImports × (1 − reexportShareOfImports)) × 12
applied to any country in the re-export share manifest. Countries NOT
in the manifest get gross imports unchanged (status-quo fallback).
Plan acceptance gates — verified synthetically in this PR:
Construct invariant. Two synthetic countries, same SWF, same gross
imports. A re-exports 60%; B re-exports 0%. Post-fix, A's rawMonths
is 2.5× B's (1/(1-0.6) = 2.5). Pinned in
tests/resilience-net-imports-denominator.test.mts.
SWF-heavy exporter invariant. Country with share ≤ 5%: rawMonths
lift < 5% vs baseline (negligible). Pinned.
What shipped
1. Re-export share manifest infrastructure.
- scripts/shared/reexport-share-manifest.yaml (new, empty) — schema
committed; entries populated in follow-up PRs with UNCTAD
Handbook citations.
- scripts/shared/reexport-share-loader.mjs (new) — loader + strict
validator, mirrors swf-manifest-loader.mjs.
- scripts/seed-recovery-reexport-share.mjs (new) — publishes
resilience:recovery:reexport-share:v1 from manifest. Empty
manifest = valid (no countries, no adjustment).
2. SWF seeder uses net-imports denominator.
- scripts/seed-sovereign-wealth.mjs exports computeNetImports(gross,
share) — pure helper, unit-tested.
- Per-country loop: reads manifest, computes denominatorImports,
applies to rawMonths math.
- Payload records annualImports (gross, audit), denominatorImports
(used in math), reexportShareOfImports (provenance).
- Summary log reports which countries had a net-imports adjustment
applied with source year.
3. Bundle wiring.
- Reexport-Share runs BEFORE Sovereign-Wealth in the recovery
bundle so the SWF seeder reads fresh re-export data in the same
cron tick.
- tests/seed-bundle-resilience-recovery.test.mjs expected-entries
updated (6 → 7) with ordering preservation.
4. Cache-prefix bump (per cache-prefix-bump-propagation-scope skill).
- RESILIENCE_SCORE_CACHE_PREFIX: v11 → v12
- RESILIENCE_RANKING_CACHE_KEY: v11 → v12
- RESILIENCE_HISTORY_KEY_PREFIX: v6 → v7 (history rotation prevents
30-day rolling window from mixing pre/post-fix scores and
manufacturing false "falling" trends on deploy day).
- Source of truth: server/worldmonitor/resilience/v1/_shared.ts
- Mirrored in: scripts/seed-resilience-scores.mjs,
scripts/validate-resilience-correlation.mjs,
scripts/backtest-resilience-outcomes.mjs,
scripts/validate-resilience-backtest.mjs,
scripts/benchmark-resilience-external.mjs, api/health.js
- Test literals bumped in 4 test files (26 line edits).
- EXTENDED tests/resilience-cache-keys-health-sync.test.mts with
a parity pass that reads every known mirror file and asserts
both (a) canonical prefix present AND (b) no stale v<older>
literals in non-comment code. Found one legacy log-line that
still referenced v9 (scripts/seed-resilience-scores.mjs:342)
and refactored it to use the RESILIENCE_RANKING_CACHE_KEY
constant so future bumps self-update.
Explicitly NOT in this PR
- liquidReserveAdequacy denominator fix. The plan's PR 3A wording
mentions both dims, but the RESERVES ratio (WB FI.RES.TOTL.MO) is a
PRE-COMPUTED WB series; applying a post-hoc net-imports adjustment
mixes WB's denominator year with our manifest-year, and the math
change belongs in PR 3B (unified liquidity) where the α calibration
is explicit. This PR stays scoped to sovereignFiscalBuffer.
- Live re-export share entries. The manifest ships EMPTY in this PR;
entries with UNCTAD citations are one-per-PR follow-ups so each
figure is individually auditable.
Verified
- tests/resilience-net-imports-denominator.test.mts — 9 pass (construct
contract: 2.5× ratio gate, monotonicity, boundary rejections,
backward-compat on missing manifest entry, cohort-proportionality,
SWF-heavy-exporter-unchanged)
- tests/reexport-share-loader.test.mts — 7 pass (committed-manifest
shape + 6 schema-violation rejections)
- tests/resilience-cache-keys-health-sync.test.mts — 5 pass (existing 3
+ 2 new parity checks across all mirror files)
- tests/seed-bundle-resilience-recovery.test.mjs — 17 pass (expected
entries bumped to 7)
- npm run test:data — 6714 pass / 0 fail
- npm run typecheck / typecheck:api — green
- npm run lint / lint:md — clean
Deployment notes
Score + ranking + history cache prefixes all bump in the same deploy.
Per established v10→v11 precedent (and the cache-prefix-bump-
propagation-scope skill):
- Score / ranking: 6h TTL — the new prefix populates via the Railway
resilience-scores cron within one tick.
- History: 30d ring — the v7 ring starts empty; the first 30 days
post-deploy lack baseline points, so trend / change30d will read as
"no change" until v7 accumulates a window.
- Legacy v11 keys can be deleted from Redis at any time post-deploy
(no reader references them). Leaving them in place costs storage
but does no harm.
|
||
|
Elie Habib
|
0081da4148 |
fix(resilience): widen Comtrade period to 4y + surface picked year (#3372)
PR 1 of cohort-audit plan 2026-04-24-002. Unblocks UAE, Oman, Bahrain
(and any other late-reporter) on the importConcentration dimension.
Problem
- seed-recovery-import-hhi.mjs queries Comtrade with `period=Y-1,Y-2`
(currently "2025,2024"). Several reporters publish Comtrade 1-2y
behind — their 2024/2025 rows are empty while 2023 is populated.
- With no data in the queried window, parseRecords() returned [] for
the reporter, the seeder counted a "skip", the scorer fell through
to IMPUTE (score=50, coverage=0.3, imputationClass="unmonitored"),
and the cohort-sanity audit flagged AE as a coverage-outlier inside
the GCC — exactly the class of silent gap the audit is designed to
catch.
Fix
1. Widen the Comtrade period parameter to a 4-year window Y-1..Y-4
via a new `buildPeriodParam(now)` helper. On-time reporters still
pick their latest year via the existing completeness tiebreak in
parseRecords(); late reporters now pick up whatever year they
actually published in (2023 for UAE, etc.).
2. parseRecords() now returns { rows, year } — the year surfaces in
the per-country payload as `year: number | null` for operator
freshness audit. The scorer already expects this shape
(_dimension-scorers.ts:1524 RecoveryImportHhiCountry.year); this
PR actually populates it.
3. `buildPeriodParam` + `parseRecords` are exported so their unit
tests can pin year-selection behaviour without hitting Comtrade.
Note on PR 2 of the same plan
The plan calls out "PR 2 — externalDebtCoverage re-goalpost to
Greenspan-Guidotti" as unshipped. It IS shipped: commit
|
||
|
Elie Habib
|
df392b0514 |
feat(resilience): PR 0 — cohort-sanity release-gate harness (#3369)
* feat(resilience): PR 0 — cohort-sanity release-gate harness Lands the audit infrastructure for the resilience cohort-ranking structural audit (plan 2026-04-24-002). Release gate, not merge gate: the audit tells release review what to look at before publishing a ranking; it does not block a PR. What's new - scripts/audit-resilience-cohorts.mjs — Markdown report generator. Fetches the live ranking + per-country scores (or reads a fixture in offline mode), emits per-cohort per-dimension tables, contribution decomposition, saturated / outlier / identical-score flags, and a top-N movers comparison vs a baseline snapshot. - tests/resilience-construct-invariants.test.mts — 12 formula-level anchor-value assertions with synthetic inputs. Covers HHI, external debt (Greenspan-Guidotti anchor), and sovereign fiscal buffer (saturating transform). Tests the MATH, not a country's rank. - tests/fixtures/resilience-audit-fixture.json — offline fixture that mirrors the 2026-04-24 GCC state (KW>QA>AE) so the audit tool can be smoke-tested without API-key access. - docs/methodology/cohort-sanity-release-gate.md — operational doc explaining when to run, how to read the report, and the explicit anti-pattern note on rank-targeted acceptance criteria. Verified - `npx tsx --test tests/resilience-construct-invariants.test.mts` — 12 pass (HHI, debt, SWF invariants all green against current scorer) - `npm run test:data` — 6706 pass / 0 fail - `FIXTURE=tests/fixtures/resilience-audit-fixture.json OUT=/tmp/audit.md node scripts/audit-resilience-cohorts.mjs` runs to completion and correctly flags: (a) coverage-outlier on AE.importConcentration (0.3 vs peers 1.0) (b) saturated-high on GCC.externalDebtCoverage (all 6 at 100) — the two top cohort-sanity findings from the plan. Not in this PR - The live-API baseline snapshot (docs/snapshots/resilience-ranking-live-pre-cohort-audit-2026-04-24.json) is deferred to a manual release-prep step: run `WORLDMONITOR_API_KEY=wm_xxx API_BASE=https://api.worldmonitor.app node scripts/freeze-resilience-ranking.mjs` before the first methodology PR (PR 1 HHI period widening) so its movers table has something to compare against. - No scorer changes. No cache-prefix bumps. This PR is pure tooling. * fix(resilience): fail-closed on fetch failures + pillar-combine formula mode Addresses review P1 + P2 on PR #3369. P1 — fetch-failure silent-drop. Per-country score fetches that failed were logged to stderr, silently stored as null, and then filtered out of cohort tables via `codes.filter((cc) => scoreMap.get(cc))`. A transient 403/500 on the very country carrying the ranking anomaly could produce a Markdown report that looked valid — wrong failure mode for a release gate. Fix: - `fetchScoresConcurrent` now tracks failures in a dedicated Map and does NOT insert null placeholders; missing cohort members are computed against the requested cohort code set. - The report has a ⛔ blocker banner at top AND an always-rendered "Fetch failures / missing members" section (shown even when empty, so an operator learns to look). - `STRICT=1` writes the report, then exits code 3 on any fetch failure or missing cohort member, code 4 on formula-mode drift, code 0 otherwise. Automation can differentiate the two. P2 — pillar-combine formula mode invalidates contribution rows. `docs/methodology/cohort-sanity-release-gate.md:63` tells operators to run this audit before activating `RESILIENCE_PILLAR_COMBINE_ENABLED`, but the contribution decomposition is a domain-weighted roll-up that is ONLY valid when `overallScore = sum(domain.score * domain.weight)`. Once pillar combine is on, `overallScore = penalizedPillarScore(pillars)` (non-linear in dim scores); decomposition rows become materially misleading for exactly the release-gate scenario the doc prescribes. Fix: - Added `detectFormulaMode(scoreMap)` that takes countries with: (a) `sum(domain.weight)` within 0.05 of 1.0 (complete response), AND (b) every dim at `coverage ≥ 0.9` (stable share math) and compares `|Σ contributions - overallScore|` against `CONTRIB_TOLERANCE` (default 1.5). If > 50% of ≥ 3 eligible countries drift, pillar combine is flagged. - Report emits a ⛔ blocker banner at top, a "Formula mode" line in the header, and a "Formula-mode diagnostic" section with the first three offenders. Under `STRICT=1` exits code 4. - Methodology doc updated: new "Fail-closed semantics" section, "Formula mode" operator guide, ENV table entries for STRICT + CONTRIB_TOLERANCE. Verified: - `tests/audit-cohort-formula-detection.test.mts` (NEW) — 3 child-process smoke tests: missing-members banner + STRICT exit 3, all-clear exit 0, pillar-mode banner + STRICT exit 4. All pass. - `npx tsx --test tests/resilience-construct-invariants.test.mts tests/audit-cohort-formula-detection.test.mts` — 15 pass / 0 fail - `npm run test:data` — 6709 pass / 0 fail - `npm run typecheck` / `typecheck:api` — green - `npm run lint` / `lint:md` — no warnings on new / changed files (refactor split buildReport complexity from 51 → under 50 by extracting `renderCohortSection` + `renderDimCell`) - Fixture smoke: AE.importConcentration coverage-outlier and GCC.externalDebtCoverage saturated-high flags still fire correctly. * fix(resilience): PR 0 review — fixture-mode source label, try/catch country-names, ASCII minus Addresses 3 P2 Greptile findings on #3369: 1. **Misleading Source: line in fixture mode.** `FIXTURE_PATH` sets `API_BASE=''`, so the report header showed a bare "/api/..." path that never resolved — making a fixture run visually indistinguishable from a live run. Now surfaces `Source: fixture://<path>` in fixture mode. 2. **`loadCountryNameMap` crashes without useful diagnostics.** A missing or unparseable `shared/country-names.json` produced a raw unhandled rejection. Now the read and the parse are each wrapped in their own try/catch; on either failure the script logs a developer-friendly warning and falls back to ISO-2 codes (report shows "AE" instead of "Uae"). Keeps the audit operable in CI-offline scenarios. 3. **Unicode minus `−` (U+2212) instead of ASCII `-` in `fmtDelta`.** Downstream operators diff / grep / CSV-pipe the report; the Unicode minus breaks byte-level text tooling. Replaced with ASCII hyphen- minus. Left the U+2212 in the formula-mode diagnostic prose (`|Σ contributions − overallScore|`) where it's mathematical notation, not data. Verified - `npx tsx --test tests/audit-cohort-formula-detection.test.mts tests/resilience-construct-invariants.test.mts` — 15 pass / 0 fail - Fixture-mode run produces `Source: fixture://tests/fixtures/...` - Movers-table negative deltas now use ASCII `-` |
||
|
Elie Habib
|
34dfc9a451 |
fix(news): ground LLM surfaces on real RSS description end-to-end (#3370)
* feat(news/parser): extract RSS/Atom description for LLM grounding (U1)
Add description field to ParsedItem, extract from the first non-empty of
description/content:encoded (RSS) or summary/content (Atom), picking the
longest after HTML-strip + entity-decode + whitespace-normalize. Clip to
400 chars. Reject empty, <40 chars after strip, or normalize-equal to the
headline — downstream consumers fall back to the cleaned headline on '',
preserving current behavior for feeds without a description.
CDATA end is anchored to the closing tag so internal ]]> sequences do not
truncate the match. Preserves cached rss:feed:v1 row compatibility during
the 1h TTL bleed since the field is additive.
Part of fix: pipe RSS description end-to-end so LLM surfaces stop
hallucinating named actors (docs/plans/2026-04-24-001-...).
Covers R1, R7.
* feat(news/story-track): persist description on story:track:v1 HSET (U2)
Append description to the story:track:v1 HSET only when non-empty. Additive
— no key version bump. Old rows and rows from feeds without a description
return undefined on HGETALL, letting downstream readers fall back to the
cleaned headline (R6).
Extract buildStoryTrackHsetFields as a pure helper so the inclusion gate is
unit-testable without Redis.
Update the contract comment in cache-keys.ts so the next reader of the
schema sees description as an optional field.
Covers R2, R6.
* feat(proto): NewsItem.snippet + SummarizeArticleRequest.bodies (U3)
Add two additive proto fields so the article description can ride to every
LLM-adjacent consumer without a breaking change:
- NewsItem.snippet (field 12): RSS/Atom description, HTML-stripped,
≤400 chars, empty when unavailable. Wired on toProtoItem.
- SummarizeArticleRequest.bodies (field 8): optional article bodies
paired 1:1 with headlines for prompt grounding. Empty array is today's
headline-only behavior.
Regenerated TS client/server stubs and OpenAPI YAML/JSON via sebuf v0.11.1
(PATH=~/go/bin required — Homebrew's protoc-gen-openapiv3 is an older
pre-bundle-mode build that collides on duplicate emission).
Pre-emptive bodies:[] placeholders at the two existing SummarizeArticle
call sites in src/services/summarization.ts; U6 replaces them with real
article bodies once SummarizeArticle handler reads the field.
Covers R3, R5.
* feat(brief/digest): forward RSS description end-to-end through brief envelope (U4)
Digest accumulator reader (seed-digest-notifications.mjs::buildDigest) now
plumbs the optional `description` field off each story:track:v1 HGETALL into
the digest story object. The brief adapter (brief-compose.mjs::
digestStoryToUpstreamTopStory) prefers the real RSS description over the
cleaned headline; when the upstream row has no description (old rows in the
48h bleed, feeds that don't carry one), we fall back to the cleaned headline
so today behavior is preserved (R6).
This is the upstream half of the description cache path. U5 lands the LLM-
side grounding + cache-prefix bump so Gemini actually sees the article body
instead of hallucinating a named actor from the headline.
Covers R4 (upstream half), R6.
* feat(brief/llm): RSS grounding + sanitisation + 4 cache prefix bumps (U5)
The actual fix for the headline-only named-actor hallucination class:
Gemini 2.5 Flash now receives the real article body as grounding context,
so it paraphrases what the article says instead of filling role-label
headlines from parametric priors ("Iran's new supreme leader" → "Ali
Khamenei" was the 2026-04-24 reproduction; with grounding, it becomes
the actual article-named actor).
Changes:
- buildStoryDescriptionPrompt interpolates a `Context: <body>` line
between the metadata block and the "One editorial sentence" instruction
when description is non-empty AND not normalise-equal to the headline.
Clips to 400 chars as a second belt-and-braces after the U1 parser cap.
No Context line → identical prompt to pre-fix (R6 preserved).
- sanitizeStoryForPrompt extended to cover `description`. Closes the
asymmetry where whyMatters was sanitised and description wasn't —
untrusted RSS bodies now flow through the same injection-marker
neutraliser before prompt interpolation. generateStoryDescription wraps
the story in sanitizeStoryForPrompt before calling the builder,
matching generateWhyMatters.
- Four cache prefixes bumped atomically to evict pre-grounding rows:
scripts/lib/brief-llm.mjs:
brief:llm:description:v1 → v2 (Railway, description path)
brief:llm:whymatters:v2 → v3 (Railway, whyMatters fallback)
api/internal/brief-why-matters.ts:
brief:llm:whymatters:v6 → v7 (edge, primary)
brief:llm:whymatters:shadow:v4 → shadow:v5 (edge, shadow)
hashBriefStory already includes description in the 6-field material
(v5 contract) so identity naturally drifts; the prefix bump is the
belt-and-braces that guarantees a clean cold-start on first tick.
- Tests: 8 new + 2 prefix-match updates on tests/brief-llm.test.mjs.
Covers Context-line injection, empty/dup-of-headline rejection,
400-char clip, sanitisation of adversarial descriptions, v2 write,
and legacy-v1 row dark (forced cold-start).
Covers R4 + new sanitisation requirement.
* feat(news/summarize): accept bodies + bump summary cache v5→v6 (U6)
SummarizeArticle now grounds on per-headline article bodies when callers
supply them, so the dashboard "News summary" path stops hallucinating
across unrelated headlines when the upstream RSS carried context.
Three coordinated changes:
1. SummarizeArticleRequest handler reads req.bodies, sanitises each entry
through sanitizeForPrompt (same trust treatment as geoContext — bodies
are untrusted RSS text), clips to 400 chars, and pads to the headlines
length so pair-wise identity is stable.
2. buildArticlePrompts accepts optional bodies and interleaves a
` Context: <body>` line under each numbered headline that has a
non-empty body. Skipped in translate mode (headline[0]-only) and when
all bodies are empty — yielding a byte-identical prompt to pre-U6
for every current caller (R6 preserved).
3. summary-cache-key bumps CACHE_VERSION v5→v6 so the pre-grounding rows
(produced from headline-only prompts) cold-start cleanly. Extends
canonicalizeSummaryInputs + buildSummaryCacheKey with a pair-wise
bodies segment `:bd<hash>`; the prefix is `:bd` rather than `:b` to
avoid colliding with `:brief:` when pattern-matching keys. Translate
mode is headline[0]-only and intentionally does not shift on bodies.
Dedup reorder preserved: the handler re-pairs bodies to the deduplicated
top-5 via findIndex, so layout matches without breaking cache identity.
New tests: 7 on buildArticlePrompts (bodies interleave, partial fill,
translate-mode skip, clip, short-array tolerance), 8 on
buildSummaryCacheKey (pair-wise sort, cache-bust on body drift, translate
skip). Existing summary-cache-key assertions updated v5→v6.
Covers R3, R4.
* feat(consumers): surface RSS snippet across dashboard, email, relay, MCP + audit (U7)
Thread the RSS description from the ingestion path (U1-U5) into every
user-facing LLM-adjacent surface. Audit the notification producers so
RSS-origin and domain-origin events stay on distinct contracts.
Dashboard (proto snippet → client → panel):
- src/types/index.ts NewsItem.snippet?:string (client-side field).
- src/app/data-loader.ts proto→client mapper propagates p.snippet.
- src/components/NewsPanel.ts renders snippet as a truncated (~200 chars,
word-boundary ellipsis) `.item-snippet` line under each headline.
- NewsPanel.currentBodies tracks per-headline bodies paired 1:1 with
currentHeadlines; passed as options.bodies to generateSummary so the
server-side SummarizeArticle LLM grounds on the article body.
Summary plumbing:
- src/services/summarization.ts threads bodies through SummarizeOptions
→ generateSummary → runApiChain → tryApiProvider; cache key now includes
bodies (via U6's buildSummaryCacheKey signature).
MCP world-brief:
- api/mcp.ts pairs headlines with their RSS snippets and POSTs `bodies`
to /api/news/v1/summarize-article so the MCP tool surface is no longer
starved.
Email digest:
- scripts/seed-digest-notifications.mjs plain-text formatDigest appends
a ~200-char truncated snippet line under each story; HTML formatDigestHtml
renders a dim-grey description div between title and meta. Both gated
on non-empty description (R6 — empty → today's behavior).
Real-time alerts:
- src/services/breaking-news-alerts.ts BreakingAlert gains optional
description; checkBatchForBreakingAlerts reads item.snippet; dispatchAlert
includes `description` in the /api/notify payload when present.
Notification relay:
- scripts/notification-relay.cjs formatMessage gated on
NOTIFY_RELAY_INCLUDE_SNIPPET=1 (default off). When on, RSS-origin
payloads render a `> <snippet>` context line under the title. When off
or payload.description absent, output is byte-identical to pre-U7.
Audit (RSS vs domain):
- tests/notification-relay-payload-audit.test.mjs enforces file-level
@notification-source tags on every producer, rejects `description:` in
domain-origin payload blocks, and verifies the relay codepath gates
snippet rendering under the flag.
- Tag added to ais-relay.cjs (domain), seed-aviation.mjs (domain),
alert-emitter.mjs (domain), breaking-news-alerts.ts (rss).
Deferred (plan explicitly flags): InsightsPanel + cluster-producer
plumbing (bodies default to [] — will unlock gradually once news:insights:v1
producer also carries primarySnippet).
Covers R5, R6.
* docs+test: grounding-path note + bump pinned CACHE_VERSION v5→v6 (U8)
Final verification for the RSS-description-end-to-end fix:
- docs/architecture.mdx — one-paragraph "News Grounding Pipeline"
subsection tracing parser → story:track:v1.description → NewsItem.snippet
→ brief / SummarizeArticle / dashboard / email / relay / MCP, with the
empty-description R6 fallback rule called out explicitly.
- tests/summarize-reasoning.test.mjs — Fix-4 static-analysis pin updated
to match the v6 bump from U6. Without this the summary cache bump silently
regressed CI's pinned-version assertion.
Final sweep (2026-04-24):
- grep -rn 'brief:llm:description:v1' → only in the U5 legacy-row test
simulation (by design: proves the v2 bump forces cold-start).
- grep -rn 'brief:llm:whymatters:v2/v6/shadow:v4' → no live references.
- grep -rn 'summary:v5' → no references.
- CACHE_VERSION = 'v6' in src/utils/summary-cache-key.ts.
- Full tsx --test sweep across all tests/*.test.{mjs,mts}: 6747/6747 pass.
- npm run typecheck + typecheck:api: both clean.
Covers R4, R6, R7.
* fix(rss-description): address /ce:review findings before merge
14 fixes from structured code review across 13 reviewer personas.
Correctness-critical (P1 — fixes that prevent R6/U7 contract violations):
- NewsPanel signature covers currentBodies so view-mode toggles that leave
headlines identical but bodies different now invalidate in-flight summaries.
Without this, switching renderItems → renderClusters mid-summary let a
grounded response arrive under a stale (now-orphaned) cache key.
- summarize-article.ts re-pairs bodies with headlines BEFORE dedup via a
single zip-sanitize-filter-dedup pass. Previously bodies[] was indexed by
position in light-sanitized headlines while findIndex looked up the
full-sanitized array — any headline that sanitizeHeadlines emptied
mispaired every subsequent body, grounding the LLM on the wrong story.
- Client skips the pre-chain cache lookup when bodies are present, since
client builds keys from RAW bodies while server sanitizes first. The
keys diverge on injection content, which would silently miss the
server's authoritative cache every call.
Test + audit hardening:
- Legacy v1 eviction test now uses the real hashBriefStory(story()) suffix
instead of a literal "somehash", so a bug where the reader still queried
the v1 prefix at the real key would actually be caught.
- tests/summary-cache-key.test.mts adds 400-char clip identity coverage so
the canonicalizer's clip and any downstream clip can't silently drift.
- tests/news-rss-description-extract.test.mts renames the well-formed
CDATA test and adds a new test documenting the malformed-]]> fallback
behavior (plain regex captures, article content survives).
Safe_auto cleanups:
- Deleted dead SNIPPET_PUSH_MAX constant in notification-relay.cjs.
- BETA-mode groq warm call now passes bodies, warming the right cache slot.
- seed-digest shares a local normalize-equality helper for description !=
headline comparison, matching the parser's contract.
- Pair-wise sort in summary-cache-key tie-breaks on body so duplicate
headlines produce stable order across runs.
- buildSummaryCacheKey gained JSDoc documenting the client/server contract
and the bodies parameter semantics.
- MCP get_world_brief tool description now mentions RSS article-body
grounding so calling agents see the current contract.
- _shared.ts `opts.bodies![i]!` double-bang replaced with `?? ''`.
- extractRawTagBody regexes cached in module-level Map, mirroring the
existing TAG_REGEX_CACHE pattern.
Deferred to follow-up (tracked for PR description / separate issue):
- Promote shared MAX_BODY constant across the 5 clip sites
- Promote shared truncateForDisplay helper across 4 render sites
- Collapse NewsPanel.{currentHeadlines, currentBodies} → Array<{title, snippet}>
- Promote sanitizeStoryForPrompt to shared/brief-llm-core.js
- Split list-feed-digest.ts parser helpers into sibling -utils.ts
- Strengthen audit test: forward-sweep + behavioral gate test
Tests: 6749/6749 pass. Typecheck clean on both configs.
* fix(summarization): thread bodies through browser T5 path (Codex #2)
Addresses the second of two Codex-raised findings on PR #3370:
The PR threaded bodies through the server-side API provider chain
(Ollama → Groq → OpenRouter → /api/news/v1/summarize-article) but the
local browser T5 path at tryBrowserT5 was still summarising from
headlines alone. In BETA_MODE that ungrounded path runs BEFORE the
grounded server providers; in normal mode it remains the last
fallback. Whenever T5-small won, the dashboard summary surface
regressed to the headline-only path — the exact hallucination class
this PR exists to eliminate.
Fix: tryBrowserT5 accepts an optional `bodies` parameter and
interleaves each body with its paired headline via a `headline —
body` separator in the combined text (clipped to 200 chars per body
to stay within T5-small's ~512-token context window). All three call
sites (BETA warm, BETA cold, normal-mode fallback) now pass the
bodies threaded down from generateSummary options.bodies.
When bodies is empty/omitted, the combined text is byte-identical to
pre-fix (R6 preserved).
On Codex finding #1 (story:track:v1 additive-only HSET keeps a body
from an earlier mention of the same normalized title), declining to
change. The current rule — "if this mention has a body, overwrite;
otherwise leave the prior body alone" — is defensible: a body from
mention A is not falsified by mention B being body-less (a wire
reprint doesn't invalidate the original source's body). A feed that
publishes a corrected headline creates a new normalized-title hash,
so no stale body carries forward. The failure window is narrow (live
story evolving while keeping the same title through hours of
body-less wire reprints) and the 7-day STORY_TTL is the backstop.
Opening a follow-up issue to revisit semantics if real-world evidence
surfaces a stale-grounding case.
* fix(story-track): description always-written to overwrite stale bodies (Codex #1)
Revisiting Codex finding #1 on PR #3370 after re-review. The previous
response declined the fix with reasoning; on reflection the argument
was over-defending the current behavior.
Problem: buildStoryTrackHsetFields previously wrote `description` only
when non-empty. Because story:track:v1 rows are collapsed by
normalized-title hash, an earlier mention's body would persist for up
to STORY_TTL (7 days) on subsequent body-less mentions of the same
story. Consumers reading `track.description` via HGETALL could not
distinguish "this mention's body" from "some mention's body from the
last week," silently grounding brief / whyMatters / SummarizeArticle
LLMs on text the current mention never supplied. That violates the
grounding contract advertised to every downstream surface in this PR.
Fix: HSET `description` unconditionally on every mention — empty
string when the current item has no body, real body when it does. An
empty value overwrites any prior mention's body so the row is always
authoritative for the current cycle. Consumers continue to treat
empty description as "fall back to cleaned headline" (R6 preserved).
The 7-day STORY_TTL and normalized-title hash semantics are unchanged.
Trade-off accepted: a valid body from Feed A (NYT) is wiped when Feed
B (AP body-less wire reprint) arrives for the same normalized title,
even though Feed A's body is factually correct. Rationale: the
alternative — keeping Feed A's body indefinitely — means the user
sees Feed A's body attributed (by proximity) to an AP mention at a
later timestamp, which is at minimum misleading and at worst carries
retracted/corrected details. Honest absence beats unlabeled presence.
Tests: new stale-body overwrite sequence test (T0 body → T1 empty →
T2 new body), existing "writes description when non-empty" preserved,
existing "omits when empty" inverted to "writes empty, overwriting."
cache-keys.ts contract comment updated to mark description as
always-written rather than optional.
|
||
|
Elie Habib
|
959086fd45 |
fix(panels): address Greptile P2 review on #3364 (icons + category map) (#3365)
Two P2 findings on the now-merged #3364:
1. Icon encoding inconsistency. The two new pipeline/storage entries
mixed '\u{1F6E2}' + raw VS16 (U+FE0F) while every other panel in
the file uses '\u{1F6E2}️'. Same runtime glyph, but mixed
encoding is lint-noisy. Normalize to the escaped form.
2. PANEL_CATEGORY_MAP gap. pipeline-status, storage-facility-map and
fuel-shortages were registered in FULL_PANELS + CMD+K but absent
from PANEL_CATEGORY_MAP, so users browsing the settings category
picker didn't see them. Add to marketsFinance alongside
energy-complex. While here, close the same pre-existing gap for
hormuz-tracker and energy-crisis — reviewer explicitly called
these out as worth addressing together.
The third P2 (spr keyword collision with oil-inventories) was fixed
in commit
|
||
|
Elie Habib
|
d521924253 |
fix(resilience): fail closed on missing v2 energy seeds + health CRIT on absent inputs (#3363)
* fix(resilience): fail closed on missing v2 energy seeds + health CRIT on absent inputs PR #3289 shipped the v2 energy construct behind RESILIENCE_ENERGY_V2_ENABLED (default false). Audit on 2026-04-24 after the user flagged "AE only moved 1.49 points — we added nuclear credit, we should see more" revealed two safety gaps that made a future flag flip unsafe: 1. scoreEnergyV2 silently fell back to IMPUTE when any of its three required Redis seeds (low-carbon-generation, fossil-electricity-share, power-losses) was null. A future operator flipping the flag with seeds absent would produce fabricated-looking numbers for every country with zero operator signal. 2. api/health.js had those three seed labels in BOTH SEED_META (CRIT on missing) AND ON_DEMAND_KEYS (which demotes CRIT to WARN). The demotion won. Health has been reporting WARNING on a scorer dependency that has been 100% missing since PR #3289 merged — no paging trail existed. Changes: server/worldmonitor/resilience/v1/_dimension-scorers.ts - Add ResilienceConfigurationError with missingKeys[] payload. - scoreEnergy: preflight the three v2 seeds when flag=true. Throw ResilienceConfigurationError listing the specific absent keys. - scoreAllDimensions: wrap per-dimension dispatch in try/catch so a thrown ResilienceConfigurationError routes to the source-failure shape (imputationClass='source-failure', coverage=0) for that ONE dimension — country keeps scoring other dims normally. Log once per country-dimension pair so the gap is audit-traceable. api/health.js - Remove lowCarbonGeneration / fossilElectricityShare / powerLosses from ON_DEMAND_KEYS. They stay in BOOTSTRAP_KEYS + SEED_META. - Replace the transitional comment with a hard "do NOT add these back" note pointing at the scorer's fail-closed gate. tests/resilience-energy-v2.test.mts - New test: flag on + ALL three seeds missing → throws ResilienceConfigurationError naming all three keys. - New test: flag on + only one seed missing → throws naming ONLY the missing key (operator-clarity guard). - New test: flag on + all seeds present → v2 runs normally. - Update the file-level invariant comment to reflect the new fail-closed contract (replacing the prior "degrade gracefully" wording that codified the silent-IMPUTE bug). - Note: fixture's `??` fallbacks coerce null-overrides into real data, so the preflight tests use a direct-reader helper. docs/methodology/country-resilience-index.mdx - New "Fail-closed semantics" paragraph in the v2 Energy section documenting the throw + source-failure + health-CRIT contract. Non-goals (intentional): - This PR does NOT flip RESILIENCE_ENERGY_V2_ENABLED. - This PR does NOT provision seed-bundle-resilience-energy-v2 on Railway. - This PR does NOT touch RESILIENCE_PILLAR_COMBINE_ENABLED. Operational effect post-merge: - /api/health flips from WARNING → CRITICAL on the three v2 seed-meta entries. That is the intended alarm; it reveals that the Railway bundle was never provisioned. - scoreEnergy behavior with flag=false is unchanged (legacy path). - scoreEnergy behavior with flag=true + seeds present is unchanged. - scoreEnergy behavior with flag=true + seeds absent changes from "silently IMPUTE all 217 countries" to "source-failure on the energy dim for every country, visible in widget + API response". Tests: 511/511 resilience-* pass. Biome clean. Lint:md clean. Related plan: docs/plans/2026-04-24-001-fix-resilience-v2-fail-closed-on-missing-seeds-plan.md * docs(resilience): scrub stale ON_DEMAND_KEYS references for v2 energy seeds Greptile P2 on PR #3363: four stale references implied the three v2 energy seeds were still gated as ON_DEMAND_KEYS (WARN-on-missing) even though this PR's api/health.js change removed them (now strict SEED_META = CRIT on missing). Scrubbing each: - api/health.js:196 (BOOTSTRAP_KEYS comment) — was "ON_DEMAND_KEYS until Railway cron provisions; see below." Updated to cite plan 2026-04-24-001 and the strict-SEED_META posture. - api/health.js:398 (SEED_META comment) — was "Listed in ON_DEMAND_KEYS below until Railway cron provisions..." Updated for same reason. - docs/methodology/country-resilience-index.mdx:635 — v2.1 changelog entry said seed keys were ON_DEMAND_KEYS until graduation. Replaced with the fail-closed contract description. - docs/methodology/energy-v2-flag-flip-runbook.md:25 — step 3 said "ON_DEMAND_KEYS graduation" was required at flag-flip time. Rewrote to explain no graduation step is needed because the posture was removed pre-activation. No code change. Tests still 14/14 on the energy-v2 suite, lint:md clean. * fix(docs): escape MDX-unsafe `<=` in energy-v2 runbook to unblock Mintlify Mintlify deploy on PR #3363 failed with `Unexpected character '=' (U+003D) before name` at `docs/methodology/energy-v2-flag-flip-runbook.md`. Two lines had `<=` in plain prose, which MDX tries to parse as a JSX-tag-start. Replaced both with `≤` (U+2264) — and promoted the two existing `>=` on adjacent lines to `≥` for consistency. Prose is clearer and MDX safe. Same pattern as `mdx-unsafe-patterns-in-md` skill; also adjacent to PR #3344's `(<137 countries)` fix. |
||
|
Elie Habib
|
c517b2fb17 |
feat(energy-atlas): expose Atlas panels on FULL variant + CMD+K (#3364)
* feat(energy-atlas): expose Atlas panels on FULL variant + CMD+K Three Atlas panels (PipelineStatusPanel, StorageFacilityMapPanel, FuelShortagePanel) shipped in PR #3294 but were registered only in ENERGY_PANELS — invisible on worldmonitor.app because the energy variant subdomain is not yet wired. Additionally, no CMD+K entries existed for them, so command-palette search for "pipeline" or "storage" returned nothing. Changes: - src/config/commands.ts: add panel:pipeline-status, panel:storage-facility-map, panel:fuel-shortages with relevant keywords (oil/gas/nord stream/druzhba/spr/lng/ugs/rationing/…). - src/config/panels.ts: add the 3 panel keys to FULL_PANELS with priority 2 so they appear in the main worldmonitor.app drawer under the existing energy-crisis block. ENERGY_PANELS keeps its own priority-1 copies so the future energy.worldmonitor.app subdomain still surfaces them top of list. Unblocks the plan's announcement gate item "UI: at least one Atlas panel renders registry data on worldmonitor.app in a browser." Part of docs/internal/energy-atlas-registry-expansion.md follow-up. * fix(cmd-k): resolve spr/lng keyword collisions so Atlas panel wins Reviewer found that the original PR wiring let CMD+K "spr" and "lng" route to the wrong panel because matchCommands() (SearchModal.ts:273) ranks by exact/prefix/substring then keeps array-insertion order on ties. Storage-atlas was declared AFTER the colliding entries. Collisions: - "spr": panel:oil-inventories (line 105) had exact 'spr' → tied with the new storage-facility-map (line 108) → insertion order kept oil-inventories winning. - "lng": panel:hormuz-tracker (line 135) has exact 'lng' → storage-facility-map only had substring 'lng terminals' (score 1) → hormuz won outright. Fix: - Remove 'spr' from oil-inventories keywords. The SPR as a *site list* semantically belongs to Strategic Storage Atlas. Stock-level queries still route to oil-inventories via 'strategic petroleum' (the word 'spr' is not a substring of 'strategic petroleum', so no fallback score leaks). - Add exact 'lng' to storage-facility-map. Both it and hormuz-tracker now score 3 on 'lng'; stable sort preserves declaration order, so storage (line 108) outranks hormuz (line 135). Hormuz still matches via 'hormuz', 'strait of hormuz', 'tanker', 'shipping'. |
||
|
Elie Habib
|
b68d98972a |
fix(unrest): bump GDELT proxy timeout 20s → 45s (#3362)
GDELT's v1 gkg_geojson endpoint is currently responding in ~19s (direct curl test: HTTP 200 at t=19.4s). With the old 20s proxy timeout the Decodo leg hits Cloudflare origin timeout and returns HTTP 522 on nearly every tick, so fetchGdeltEvents throws "both paths failed — proxy: HTTP/1.1 522 Server Error" and runSeed freezes seed-meta fetchedAt. Result: the unrest:events seed-meta stops advancing while Redis still holds the last-good payload — health.js reports STALE_SEED even though the seeder is running on schedule every 45 min. 4.5+ hours of consecutive failures observed in production logs overnight. Direct path has been chronically broken (UND_ERR_CONNECT_TIMEOUT in every tick since PR #3256 added the proxy fallback), so the proxy is the real fetch path. Giving it 45s absorbs GDELT's current degraded response time with headroom, without changing any other behavior. ACLED credentials remain unconfigured in this environment, so GDELT is effectively the single upstream — separate ops task to wire ACLED as a real second source. |
||
|
Elie Habib
|
a409d5f79d |
fix(agent-readiness): WebMCP uses registerTool + static import (#3316) (#3361)
* fix(agent-readiness): WebMCP uses registerTool + static import (#3316)
isitagentready.com reported "No WebMCP tools detected on page load"
on prod. Two compounding bugs in PR #3356:
1) API shape mismatch. Deployed code calls
navigator.modelContext.provideContext({ tools }), but the scanner
SKILL and shipping Chrome implementation use
navigator.modelContext.registerTool(tool, { signal }) per tool with
AbortController-driven teardown. The older provideContext form is
kept as a fallback.
2) Dynamic-import timing. The webmcp module was lazy-loaded from a
deep init phase, so the chunk resolved after the scanner probe
window elapsed.
Fix:
- Rewrite registerWebMcpTools to prefer registerTool with an
AbortController. provideContext becomes a legacy fallback. Returns
the AbortController so teardown paths exist.
- Static-import webmcp in App.ts and call registerWebMcpTools
synchronously at the start of init, before any await. Bindings
close over lazy refs so throw-on-null guards still fire correctly
when a tool is invoked later.
Test additions lock in registerTool-precedes-provideContext ordering,
AbortController pattern, static import, and call-before-first-await.
* fix(agent-readiness): WebMCP readiness wait + teardown on destroy (#3316)
Addresses three findings on PR #3361.
P1 — startup race. Early registration is required for scanner probes,
but a tool invoked during the window between register and Phase-4 UI
init threw "Search modal is not initialised yet." Both scanners and
agents that probe-and-invoke hit this. Bindings now await a uiReady
promise that resolves after searchManager.init and countryIntel.init.
A 10s timeout keeps a broken init from hanging the caller. After
readiness, a still-null target is a real failure and still throws.
Mechanics: App constructor builds uiReady as a Promise with its
resolve stored on the instance; Phase-4 end calls resolveUiReady;
waitForUiReady races uiReady against a timeout; both bindings await it.
P2 — AbortController was returned and dropped. registerWebMcpTools
returns a controller so callers can unregister on teardown, but App
discarded it. Stored on App now and aborted in destroy, so test
harnesses and SPA re-inits don't accumulate stale registrations.
P2 — test coverage. Added assertions for: bindings await
waitForUiReady before accessing state; resolveUiReady fires after
countryIntel.init; waitForUiReady uses Promise.race with a timeout;
destroy aborts the stored controller. Kept silent-success guard
assertions so bindings still throw when state is absent post-readiness.
Tests: 16 webmcp, 6682 full suite, all green.
* test(webmcp): tighten init()/destroy() regex anchoring (#3316)
Addresses P2 from PR #3361 review. The init() and destroy() body
captures used lazy `[\s\S]+?\n }` which stops at the first
2-space-indent close brace. An intermediate `}` inside init (e.g.
some exotic scope block) would truncate the slice; the downstream
`.split(/\n\s+await\s/)` would then operate on a smaller string and
could let a refactor slip by without tripping the assertion.
Both regexes now end with a lookahead for the next class member
(`\n\n (?:public|private) `), so the capture spans the whole method
body regardless of internal braces. If the next-member anchor ever
breaks, the match returns null and the `assert.ok` guard fails
loudly instead of silently accepting a short capture.
P1 (AbortController silently dropped) was already addressed in
|
||
|
Elie Habib
|
38f7002f19 |
fix(checkout): entitlement watchdog unblocks Dodo wallet-return deadlock (#3357)
* fix(checkout): entitlement watchdog unblocks Dodo wallet-return deadlock
Buyers completing a Dodo checkout on the subscription-trial flow get
stranded on Dodo's "Payment successful" page indefinitely. HAR evidence
(session cks_0NdL9xlzrFFNivgTeGFU9 / pay_0NdLA3yIfX3BVDoXrFltx, live):
after 3DS succeeds, Dodo's iframe navigates to
/status/{id}/wallet-return?status=succeeded and then emits nothing --
no checkout.status, no checkout.redirect_requested postMessage. Our
onEvent handler never runs, so onSuccess / banner / redirect never
fire. Prior PRs #3298, #3346, #3354 all depended on Dodo emitting a
terminal event; this path emits none.
Fix: merchant-side entitlement watchdog in both the /pro bundle
(pro-test/src/services/checkout.ts) and the dashboard bundle
(src/services/checkout.ts). When the overlay is open, poll
/api/me/entitlement every 3s with a 10min cap. When the webhook flips
the user to pro, close the stuck overlay and run the post-checkout
side effects -- independent of whatever Dodo's iframe does. Existing
event-driven paths are preserved unchanged (they remain the fast path
for non-wallet-return checkouts); the watchdog is the floor.
Idempotency via a successFired closure flag; both the event handler
and the watchdog route through the same runTerminalSuccessSideEffects
function, making double-fires impossible. checkout.closed stops the
watchdog cleanly on cancel.
Observability: Sentry breadcrumb with reason tag on every terminal
success, plus captureMessage at info level when the watchdog resolves
it -- countable signal for prevalence tracking while Dodo investigates.
Rebuilt public/pro/ bundle (index-CiMZEtgt.js to index-QpSvSkuY.js).
Plan: docs/plans/2026-04-23-002-fix-dodo-checkout-entitlement-watchdog-plan.md
Skill: .claude/skills/dodo-wallet-return-skips-postmessage/SKILL.md
* fix(checkout): stop watchdog on destroyCheckoutOverlay to prevent orphan side effects
Greptile P1 on #3357. destroyCheckoutOverlay cleared initialized and
onSuccessCallback but never called _resetOverlaySession, so if the
dashboard layout unmounted mid-checkout the watchdog setInterval kept
running inside the closed-over scope. On entitlement flip, the orphaned
watchdog would fire clearCheckoutAttempt / clearPendingCheckoutIntent /
markPostCheckout / safeCloseOverlay against whatever session was active
by then -- stepping on a new checkout's state or silently closing a
fresh overlay.
Fix: call _resetOverlaySession before dropping references, and null it
out after. _resetOverlaySession is the only accessor for the closure's
stopWatchdog so it must run before the module-scoped slot is cleared.
* test(checkout): extract testable entitlement watchdog + state-machine tests
Greptile residual risk on #3357: the watchdog state machine had no
targeted automated coverage, especially the wallet-return path where
no terminal Dodo event arrives and success is detected only via
entitlement polling.
Extract the watchdog into src/services/entitlement-watchdog.ts as a
pure DI module (fetch / setInterval / clock / token source / onPro
all injected). Mirror the file at pro-test/src/services/entitlement-
watchdog.ts since the two bundles have no cross-root imports (pro-test
alias '@' resolves to pro-test root only). Both src/services/checkout.ts
and pro-test/src/services/checkout.ts now consume createEntitlement-
Watchdog instead of inlining setInterval.
Tests cover the wallet-return scenario explicitly plus the full state
matrix:
- wallet-return path: isPro flips to true -> onPro fires exactly once
- timeout cap: isPro stays false past timeoutMs -> self-terminate
WITHOUT firing onPro
- missing token: tick no-ops, poller keeps trying
- non-2xx response (401/5xx): tick swallows, poller continues
- fetch rejection: tick swallows, poller continues
- idempotence: onPro never fires twice across consecutive pro ticks
- stop(): clears interval immediately, onPro never called
- double-start while active: second start is a no-op
- start after prior onPro: no-op (post-success reuse guard)
Parity test (tests/entitlement-watchdog-parity.test.mts) asserts the
two mirror files are byte-identical so drift alarms at CI time.
Rebuilt public/pro/ bundle (index-QpSvSkuY.js -> index-C-qy2Yt9.js).
|
||
|
Elie Habib
|
5cec1b8c4c |
fix(insights): trust cluster rank, stop LLM from re-picking top story (#3358)
* fix(insights): trust cluster rank, stop LLM from re-picking top story WORLD BRIEF panel published "Iran's new supreme leader was seriously wounded, leading him to delegate power to the Revolutionary Guards. This development comes amid an ongoing war with Israel." to every visitor for 3h. Payload: openrouter / gemini-2.5-flash. Root cause: callLLM sent all 10 clustered headlines with "pick the ONE most significant and summarize ONLY that story". Clustering ranked Lebanon journalist killing #1 (2 corroborating sources); News24 Iran rumor ranked #3 (1 source). Gemini overrode the rank, picked #3, and embellished with war framing from story #4. Objective rank (sourceCount, velocity, isAlert) lost to model vibe. Shrink the LLM's job to phrasing. Clustering already ranks — pass only topStories[0].primaryTitle and instruct the model to rewrite it using ONLY facts from the headline. No name/place/context invention. Also: - temperature 0.3 -> 0.1 (factual summary, not creative) - CACHE_TTL 3h -> 30m so a bad brief ages out in one cron cycle - Drop dead MAX_HEADLINES const Payload shape unchanged; frontend untouched. * fix(insights): corroboration gate + revert TTL + drop unconditional WHERE Follow-up to review feedback on the ranking contract, TTL, and prompt: 1. Corroboration gate (P1a). scoreImportance() in scripts/_clustering.mjs is keyword-heavy (violence +125 on a single word, flashpoint +75, ^1.5 multiplier when both hit), so a single-source sensational rumor can outrank a 2-source lead purely on lexical signals. Blindly trusting topStories[0] would let the ranker's keyword bias still pick bad stories. Walk topStories for sourceCount >= 2 instead — corroboration becomes a hard requirement, not a tiebreaker. If no cluster qualifies, publish status=degraded with no brief (frontend already handles this). 2. CACHE_TTL back to 10800 (P1b). 30m TTL == one cron cadence means the key expires on any missed or delayed run and /api/bootstrap loses insights entirely (api/bootstrap.js reads news:insights:v1 directly, no LKG across TTL-gap). The short TTL was defense-in-depth for bad content; the real safety is now upstream (corroboration gate + grounded prompt), so the LKG window doesn't need to be sacrificed for it. 3. Prompt: location conditional (P2). "Use ONLY facts present" + "Lead with WHAT happened and WHERE" conflicted for headlines without an explicit location and pushed the model toward inferred-place hallucination. Replaced with "Include a location, person, or organization ONLY if it appears in the headline." * test(insights): lock corroboration gate + grounded-prompt invariants Review P2: the corroboration gate and the prompt's no-invention rules had no tests, so future edits to selectTopStories() ordering or prompt text could silently reintroduce the original hallucination. Extract the brief-selection helper and prompt builders into a pure module (scripts/_insights-brief.mjs) so tests can import them without triggering seed-insights.mjs's top-level runSeed() call: - pickBriefCluster(topStories) returns first sourceCount>=2 cluster - briefSystemPrompt(dateISO) returns the system prompt - briefUserPrompt(headline) returns the user prompt Regression tests (tests/seed-insights-brief.test.mjs, 12 cases) lock: - pickBriefCluster skips single-source rumors even when ranked above a multi-sourced lead (explicit regression: News24 Iran supreme leader 2026-04-23 scenario with realistic scores) - pickBriefCluster tolerates missing/null entries - briefSystemPrompt forbids invented facts and proper nouns - briefSystemPrompt's "location" rule is conditional (no unconditional "Lead with WHAT and WHERE" directive that would push the model toward place-inference when the headline has no location) - briefSystemPrompt does not contain "pick the most important" style language (ranking is done by pickBriefCluster upstream) - briefUserPrompt passes the headline verbatim and instructs "only facts from this headline" Also fix a misleading comment on CACHE_TTL: corroboration is gated at brief-selection time, not on the topStories payload itself (which still includes single-source clusters rendered as the headline list). test:data: 6657/6657 pass (was 6645; +12). |
||
|
Elie Habib
|
efb6037fcc |
feat(agent-readiness): WebMCP in-page tool surface (#3316) (#3356)
* feat(agent-readiness): WebMCP in-page tool surface (#3316) Closes #3316. Exposes two UI tools to in-browser agents via the draft WebMCP spec (webmachinelearning.github.io/webmcp), mirroring the static Agent Skills index (#3310) for consistency: - openCountryBrief({ iso2 }): opens the country deep-dive panel. - openSearch(): opens the global command palette. No bypass: both tools route through the exact methods a click would hit (countryIntel.openCountryBriefByCode, searchModal.open), so auth and Pro-tier gates apply to agent invocations unchanged. Feature-detected: no-ops in Firefox, Safari, and older Chrome without navigator.modelContext. No behavioural change outside WebMCP browsers. Lazy-imported from App.ts so the module only enters the bundle if the dynamic import resolves; keeps the hot-path init synchronous. Each execute is wrapped in a logging shim that emits a typed webmcp-tool-invoked analytics event per call; webmcp-registered fires once at setup so we can distinguish capable-browser share from actual tool usage. v1 tools do not branch on auth state, so a single registration at init is correct. Source-level comment flags that any future Pro-only tool must re-register on sign-in/sign-out per the symmetric-listener rule documented in the memory system. tests/webmcp.test.mjs asserts the contract: feature-detect gate runs before provideContext, two-or-more tools ship, ISO-2 validation lives in the tool execute, every execute is wrapped in logging, and the AppBindings surface stays narrow. * fix(agent-readiness): WebMCP bindings surface missing-target as errors (#3316) Addresses PR #3356 review. P1 — silent-success via optional-chain no-op: The App.ts bindings used this.state.searchModal?.open() and an unchecked call to countryIntel.openCountryBriefByCode(). When the underlying UI state was absent (pre-init, or in a variant that skips the panel), the optional chain and the method's own null guard both returned quietly, but the tool still reported "Opened" with ok:true. Agents relying on that result would be misled. Bindings now throw when the required UI target is missing. The existing withInvocationLogging shim catches the throw, emits ok:false in analytics, and returns isError:true, so agents get an honest failure instead of a fake success. Fixed both bindings. P2: dropped unused beforeEach import in tests/webmcp.test.mjs. Added source-level assertions that both bindings throw when the UI target is absent, so a future refactor that drops the check fails loudly at CI time. |
||
|
Elie Habib
|
6d4c717e75 |
fix(health): treat empty intlDelays as OK, matching faaDelays (#3360)
intlDelays was alarming EMPTY_DATA during calm windows (seedAge 25m, records 0) while its faaDelays sibling — written by the same aviation seeder — was in EMPTY_DATA_OK_KEYS. The seeder itself declares zeroIsValid: true (scripts/seed-aviation.mjs:1171) because 0 airport disruptions is a real steady state, so the health classifier should agree. Stale-seed degradation still kicks in once seedAge > 90min. |
||
|
Elie Habib
|
def94733a8 |
feat(agent-readiness): Agent Skills discovery index (#3310) (#3355)
* feat(agent-readiness): Agent Skills discovery index (#3310) Closes #3310. Ships the Agent Skills Discovery v0.2.0 manifest at /.well-known/agent-skills/index.json plus two real, useful skills. Skills are grounded in real sebuf proto RPCs: - fetch-country-brief → GetCountryIntelBrief (public). - fetch-resilience-score → GetResilienceScore (Pro / API key). Each SKILL.md documents endpoint, auth, parameters, response shape, worked curl, errors, and when not to use the skill. scripts/build-agent-skills-index.mjs walks every public/.well-known/agent-skills/<name>/SKILL.md, sha256s the bytes, and emits index.json. Wired into prebuild + every variant build so a deploy can never ship an index whose digests disagree with served files. tests/agent-skills-index.test.mjs asserts the index is up-to-date via the script's --check mode and recomputes every sha256 against the on-disk SKILL.md bytes. Discovery wiring: - public/.well-known/api-catalog: new anchor entry with the agent-skills-index rel per RFC 9727 linkset shape. - vercel.json: adds agent-skills-index rel to the homepage + /index.html Link headers; deploy-config required-rels list updated. Canonical URLs use the apex (worldmonitor.app) since #3322 fixed the apex redirect that previously hid .well-known paths. * fix(agent-readiness): correct auth header + harden frontmatter parser (#3310) Addresses review findings on #3310. ## P1 — auth header was wrong in both SKILL.md files The published skills documented `Authorization: Bearer wm_live_...`, but WorldMonitor API keys must be sent in `X-WorldMonitor-Key`. `Authorization: Bearer` is for MCP/OAuth or Clerk JWTs — not raw `wm_live_...` keys. Agents that followed the SKILL.md verbatim would have gotten 401s despite holding valid keys. fetch-country-brief also incorrectly claimed the endpoint was "public"; server-to-server callers without a trusted browser origin are rejected by `validateApiKey`, so agents do need a key there too. Fixed both SKILL.md files to document `X-WorldMonitor-Key` and cross-link docs/usage-auth as the canonical auth matrix. ## P2 — frontmatter parser brittleness The hand-rolled parser used `indexOf('\n---', 4)` as the closing fence, which matched any body line that happened to start with `---`. Swapped for a regex that anchors the fence to its own line, and delegated value parsing to js-yaml (already a project dep) so future catalog growth (quoted colons, typed values, arrays) does not trip new edge cases. Added parser-contract tests that lock in the new semantics: body `---` does not terminate the block, values with colons survive intact, non-mapping frontmatter throws, and no-frontmatter files return an empty mapping. Index.json rebuilt against the updated SKILL.md bytes. |
||
|
Elie Habib
|
7cf0c32eaa |
fix(checkout): merchant-side escape hatch for Dodo overlay deadlock (#3354)
Dodo's hosted overlay can deadlock: X-button click fires
GET /api/checkout/sessions/{id}/payment-link, the 404 goes unhandled
inside their React, and the resulting Maximum-update-depth render
loop prevents the checkout.closed postMessage from ever escaping the
iframe. Our onEvent handler never runs, the user is stuck.
Add a merchant-side safety net: Escape-key listener on window that
calls DodoPayments.Checkout.close() (works via the merchant-mounted
iframe node, independent of the frozen inner UI), plus an auto-close
from the checkout.error branch so any surfaced error doesn't leave a
zombie overlay behind. Cleanup is wired into destroyCheckoutOverlay.
SDK 1.8.0 has no onCancel/cancel_url/dismissBehavior option —
close() is the only escape hatch Dodo exposes.
Observed 2026-04-23 session cks_0NdL3CalSpBDR6vrMFIS3 from the
?embed=pro-preview iframe-in-iframe landing flow.
|
||
|
Elie Habib
|
26d426369f |
feat(agent-readiness): RFC 8288 Link headers on homepage (#3353)
* feat(agent-readiness): RFC 8288 Link headers on homepage Closes #3308, part of epic #3306. Emit Link response headers on / and /index.html advertising every live agent-discoverable target. All rels use IANA-registered values (api-catalog, service-desc, service-doc, status) or the full IANA URI form for OAuth metadata rels (per RFC 9728). The mcp-server-card rel carries anchor="/mcp" to scope it to the MCP endpoint rather than the homepage, since the server card describes /mcp specifically. New guardrail block in tests/deploy-config.test.mjs asserts every required rel is present, targets are root-relative, and the MCP anchor remains in place. * test(agent-readiness): lockstep / + /index.html Link + exact target count Adds two test-only guards on the homepage Link-headers suite: - exact-count assertion on link targets (was `>= requiredRels.length`), catches accidental duplicate rels in vercel.json - equality guard between `/` and `/index.html` Link headers, catches silent drift when one entry gets edited and the other doesn't No production behavior change. |
||
|
Elie Habib
|
e9146516a5 |
fix(swf): restore 8/8 fund coverage + explicit per-country observability (#3352)
* fix(swf): restore 8/8 fund coverage — WB bulk mrv=1 silently dropped Gulf countries
The 2026-04-23 post-#3344 Railway run seeded 4/8 funds (NO, SA, SG) and
silently dropped AE/KW/QA. Root cause: WB's `country/all/indicator/…?mrv=1`
returns the SAME year across every country (the most recent year that any
country publishes). KW/QA/AE report NE.IMP.GNFS.CD a year or two behind
NO/SA/SG, so mrv=1 gave them `value: null` and the seeder skipped them
because the rawMonths denominator was missing.
Fix: bump to `mrv=5` and pick the most recent non-null value per country
via a new pure helper `pickLatestPerCountry(records)`. Verified via
6 back-to-back live dry-runs (all 8/8, byte-identical numbers):
NO: GPFG 1/1 effMo=93.05 (2024 imports)
AE: ADIA+Mubadala 2/2 effMo=3.85 (2023 imports)
SA: PIF 1/1 effMo=1.68 (2024 imports)
KW: KIA 1/1 effMo=45.43 (2023 imports)
QA: QIA 1/1 effMo=8.61 (2022 imports)
SG: GIC+Temasek 2/2 effMo=7.11 (2024 imports; Temasek via infobox)
Second fix (observability): every manifest country is now enumerated in
a `summary` block in the payload + logged with an explicit status and
reason. Prod 14:59Z run had logs for KW/QA ("missing WB imports") but AE
was dropped with no log line — the operator has to cross-reference the
manifest to notice. New `buildCoverageSummary(manifest, imports, countries)`
is exported and always emits one row per manifest country: `complete`,
`partial`, or `missing` with `reason ∈ {'missing WB imports', 'no fund
AUM matched'}`. Summary is also embedded in the published payload so
downstream consumers can detect degraded runs without parsing logs.
Tests (48/48 pass, 9 new):
- `pickLatestPerCountry` — 7 cases including the exact prod scenario
(AE-2024-null + AE-2023-non-null → resolves to 2023 row). Guards
against upstream re-order (asserts latest-year wins regardless of
array order), rejects null-only countries, rejects non-positive
values, handles both iso3 and iso2 codes.
- `buildCoverageSummary` — 2 cases covering the regression
(silent-drop of AE) and the reason-string disambiguation (operator
should know whether to investigate WB or Wikipedia).
Validated: 6 live end-to-end dry-runs (all 8/8), full test suite
569/569 pass, biome + lint:md clean.
* fix(swf): address Greptile P2 — uniform reason field + meaningful null-filter test
Two P2 findings on PR #3352:
1. `complete` and `partial` entries in countryStatuses were pushed
without a `reason` key, while `missing` always carried one. The log
path tolerated this (`row.reason ? ... : ''`), but the summary is
now persisted in Redis — any downstream consumer iterating
countryStatuses and reading `.reason` on a `partial` would see
undefined. Added `reason: null` to complete + partial for uniform
persisted shape. Test now asserts the `reason` key is present on
every row regardless of status.
2. The null-only pickLatestPerCountry test used `'XYZ'` as the ISO-3
code, which is filtered at the iso3→iso2 lookup stage BEFORE ever
reaching the null-value guard — a regression that removed null
filtering entirely would leave the test green. Swapped to `'NOR'`
(real ISO-3 with a valid iso2 mapping) so the null-filter is the
actual gate under test. Verified via sanity probe: `NOR + null`
still drops, `NOR + value` still lands.
Tests 48/48 pass; live dry-run still 8/8 byte-identical; biome clean.
|
||
|
Elie Habib
|
d75bde4e03 |
fix(agent-readiness): host-aware oauth-protected-resource endpoint (#3351)
* fix(agent-readiness): host-aware oauth-protected-resource endpoint
isitagentready.com enforces that `authorization_servers[*]` share
origin with `resource` (same-origin rule, matches Cloudflare's
mcp.cloudflare.com reference — RFC 9728 §3 permits split origins
but the scanner is stricter).
A single static file served from 3 hosts (apex/www/api) can only
satisfy one origin at a time. Replacing with an edge function that
derives both `resource` and `authorization_servers` from the
request `Host` header gives each origin self-consistent metadata.
No server-side behavior changes: api/oauth/*.js token issuer
doesn't bind tokens to a specific resource value (verified in
the previous PR's review).
* fix(agent-readiness): host-derive resource_metadata + runtime guardrails
Addresses P1/P2 review on this PR:
- api/mcp.ts (P1): WWW-Authenticate resource_metadata was still
hardcoded to apex even when the client hit api.worldmonitor.app.
Derive from request.headers.get('host') so each client gets a
pointer matching their own origin — consistent with the host-
aware edge function this PR introduces.
- api/oauth-protected-resource.ts (P2): add Vary: Host so any
intermediate cache keys by hostname (belt + suspenders on top of
Vercel's routing).
- tests/deploy-config.test.mjs (P2): replace regex-on-source with
a runtime handler invocation asserting origin-matching metadata
for apex/www/api hosts, and tighten the api/mcp.ts assertion to
require host-derived resource_metadata construction.
---------
Co-authored-by: Elie Habib <elie@worldmonitor.app>
|
||
|
Elie Habib
|
fc94829ce7 |
fix(settings): prevent paying users hitting 409 on stale Upgrade CTA (#3349)
* fix(settings): stop paying users hitting 409 on stale Upgrade CTA
UnifiedSettings.renderUpgradeSection branched on `isEntitled()`, which
returns false while the Convex entitlement snapshot is still null during
a cold WebSocket load. The modal's `onEntitlementChange` listener only
re-rendered the `api-keys` tab, so the stale "Upgrade to Pro" button in
the Settings tab was never replaced once the snapshot arrived.
Paying users who opened settings in that window saw "Upgrade to Pro",
clicked it, hit `/api/create-checkout`, got 409 `duplicate_subscription`,
and cascaded into the billing-portal fallback path (which itself can
dead-end on a generic Dodo login). Same class of bug as the 2026-04-17/18
panel-overlay duplicate-subscription incident called out in
panel-gating.ts:20-22 -- different surface, same race.
Three-part fix in src/components/UnifiedSettings.ts:
- renderUpgradeSection() returns a hidden wrapper for non-Dodo premium
(API key / tester key / Clerk pro role) so those users don't get
stuck on the loading placeholder indefinitely.
- A signed-in user whose Convex entitlement snapshot is still null gets
a neutral loading placeholder, not "Upgrade to Pro". An early click
can no longer submit before Convex hydrates.
- open()'s onEntitlementChange handler now swaps the .upgrade-pro-section
element in place when the snapshot arrives. Click handlers are
delegated at overlay level, so replacing the node needs no rebind.
Observed signal behind this fix:
- WORLDMONITOR-NY (2026-04-23): Checkout error: duplicate_subscription
- WORLDMONITOR-NZ (2026-04-23): getCustomerPortalUrl Server Error, same
session, 4s later, triggered by the duplicate-subscription dialog.
* fix(settings): hide loading placeholder to avoid empty bordered card
The base .upgrade-pro-section CSS (main.css:22833) applies margin, padding,
border, and surface background. Without 'hidden', the empty loading
placeholder paints a visibly blank bordered box during the Convex
cold-load window — swapping one bad state (stale 'Upgrade to Pro') for
another (confusing empty card).
Adding 'hidden' lets the browser's default [hidden] { display: none }
suppress the card entirely. Element stays queryable for the replaceWith
swap in open(), so the onEntitlementChange listener still finds it.
* fix(settings): bounded readiness window + click-time isEntitled guard
Addresses P1 review on PR #3349: the initial fix treated
getEntitlementState() === null as "still loading", but null is ALSO a
terminal state when Convex is disabled (no VITE_CONVEX_URL), auth times
out at waitForConvexAuth (10s), or initEntitlementSubscription throws
(entitlements.ts:41,47,58,78). In those cases a signed-in free user
would have seen a permanently empty placeholder instead of the Upgrade
to Pro CTA — a real regression on the main conversion surface.
Changes in src/components/UnifiedSettings.ts:
- Add `entitlementReady` class flag + `entitlementReadyTimer`. The flag
flips true on first snapshot OR after a 12s fallback timer kicks in.
12s > the 10s waitForConvexAuth timeout so healthy-but-slow paths land
on the real state before the fallback fires.
- Seed `entitlementReady = getEntitlementState() !== null` BEFORE the
first render() so the initial paint branches on the current snapshot,
not the stale value from a prior open/close cycle.
- renderUpgradeSection() now gates the loading placeholder on
`!this.entitlementReady` so the signed-in-free branch eventually
renders the Upgrade CTA even when Convex never hydrates.
- handleUpgradeClick() defensively re-checks isEntitled() at click time:
if the snapshot arrives AFTER the 12s timer but BEFORE the user's
click, route to the billing portal instead of triggering
/api/create-checkout against an active subscription (which would 409
and re-enter the exact duplicate_subscription → getCustomerPortalUrl
cascade this PR is trying to eliminate).
- Extract replaceUpgradeSection() helper so both the listener and the
fallback timer share the same in-place swap path.
- close() clears the timer.
* fix(settings): clear entitlementReadyTimer in destroy()
Mirror the close() cleanup. Without this, if destroy() is called during
the 12s fallback window the timer fires after teardown and invokes
replaceUpgradeSection() against a detached overlay. The early-return
inside replaceUpgradeSection (querySelector returns null) makes the
callback a no-op, but the stray async callback + DOM reference stay
alive until fire — tidy them up at destroy time.
|
||
|
Elie Habib
|
38218db7cd |
fix(energy): strict validation — emptyDataIsFailure on Atlas seeders (#3350)
Adds `emptyDataIsFailure: true` to all 5 curated-registry seeders in the `seed-bundle-energy-sources` Railway service. File-read-and-validate seeders whose validateFn returns false (stale container, missing data file, shape regression, etc.) MUST leave seed-meta stale rather than stamping fresh `recordCount: 0` via the default `publishResult.skipped` branch in `_seed-utils.mjs:906-917`. Why this matters — observed production incident on 2026-04-23 (post PR #3337 merge): - Subset of Atlas seeders hit the validation-skip path (for reasons involving a Railway container stale vs the merged code + a local Option A run during an intermediate-file-state window). - `_seed-utils.mjs:910` `writeFreshnessMetadata(..., 0, ...)` stamped `seed-meta:energy:pipelines-oil` and `seed-meta:energy:storage-facilities` with fresh `fetchedAt + recordCount: 0`. - Bundle runner's interval gate at `_bundle-runner.mjs:210` reads `fetchedAt` only, not `recordCount`. With `elapsed < 0.8 × 10080min = 8064min`, the gate skipped these 2 sections for ~5.5 days. No canonical data was written; health reported EMPTY; bundle never self-healed. With `emptyDataIsFailure: true`, the strict branch at `_seed-utils.mjs:897-905` fires instead: FAILURE: validation failed (empty data) — seed-meta NOT refreshed; bundle will retry next cycle Seed-meta stays stale, bundle counts it as `failed++`, next cron tick retries. Health flips STALE_SEED within max-stale-min. Operator sees it. Loud-failure instead of silent-skip-with-meta-refresh. Pattern previously documented for strict-floor validators (IMF/WEO 180+ country seeders in `feedback_strict_floor_validate_fail_poisons_seed_meta.md`) — now applied to all 5 Energy Atlas curated registries for the same reasons. No functional change in the healthy path — validation-passing runs still publish canonical + fresh seed-meta as before. Verification: typecheck clean, 6618/6618 data tests pass. |
||
|
Elie Habib
|
8278c8e34e |
fix(forecasts): unwrap seed-contract envelope in canonical-key sim patcher (#3348)
* fix(forecasts): unwrap seed-contract envelope in canonical-key sim patcher Production bug observed 2026-04-23 across both forecast worker services (seed-forecasts-simulation + seed-forecasts-deep): every successful run logs `[SimulationDecorations] Cannot patch canonical key — predictions missing or not an array` and silently fails to write simulation adjustments back to forecast:predictions:v2. Root cause: PR #3097 (seed-contract envelope dual-write) wraps canonical seed writes in `{_seed: {...}, data: {predictions: [...]}}` via runSeed. The Lua patcher (_SIM_PATCH_LUA) and its JS test-path mirror both read `payload.predictions` directly with no envelope unwrap, so they always return 'MISSING' against the new shape — meeting the documented pattern in the project's worldmonitor-seed-envelope-consumer-drift learning (91 producers enveloped, private-helper consumers not migrated). User-visible impact: ForecastPanel renders simulation-adjusted scores only when a fast-path seed has touched a forecast since the bug landed; deep-forecast and simulation re-scores never reach the canonical feed. Fix: - _SIM_PATCH_LUA detects envelope shape (`type(payload._seed) == 'table' and type(payload.data) == 'table'`), reads `inner.predictions`, and re-encodes preserving the wrapper so envelope shape persists across patches. Legacy bare values still pass through unchanged. - JS test path mirrors the same unwrap/rewrap. - New test WD-20b locks the regression: enveloped store fixture, asserts `_seed` wrapper preserved on write + inner predictions patched. Also resolves the per-run `[seed-contract] forecast:predictions missing fields: sourceVersion — required in PR 3` warning by passing `sourceVersion: 'detectors+llm-pipeline'` to runSeed (PR 3 of the seed-contract migration will start enforcing this; cheap to fix now). Verified: typecheck (both tsconfigs) clean; lint 0 errors; test:data 6631/6631 green (forecast suite 309/309 incl new WD-20b); edge-functions 176/176 green; markdown + version-check clean. * fix(forecasts): tighten JS envelope guard to match Lua's strict table check PR #3348 review (P2): JS test path used `!!published._seed` (any truthy value) while the Lua script requires `type(payload._seed) == 'table'` (strict object check). Asymmetry: a fixture with `_seed: true`, `_seed: 1`, or `_seed: 'string'` would be treated as enveloped by JS and bare by Lua — meaning the JS test mirror could silently miss real Lua regressions that bisect on fixture shape, defeating the purpose of having a parity test path. Tighten JS to require both `_seed` and `data` be plain objects (rejecting truthy non-objects + arrays), matching Lua's `type() == 'table'` semantics exactly. New test WD-20c locks the parity: fixture with non-table `_seed` (string) + bare-shape `predictions` → must succeed via bare path, identical to what Lua would do. Verified: 6632/6632 tests pass; new WD-20c green. |
||
|
Elie Habib
|
54479feacc |
fix(ci): vercel-ignore prefers merge-base over VERCEL_GIT_PREVIOUS_SHA on previews (#3347)
* fix(ci): vercel-ignore prefers merge-base over VERCEL_GIT_PREVIOUS_SHA on previews PR #3346 was incorrectly skipped with 'Canceled by Ignored Build Step' despite touching src/, pro-test/, and public/. Root cause: on a PR branch's FIRST push, Vercel populates VERCEL_GIT_PREVIOUS_SHA in ways that make the path-diff collapse to empty (e.g., same SHA as HEAD, or a parent commit that sees no net change in the allowed paths). The script preferred PREVIOUS_SHA and only fell back to `git merge-base HEAD origin/main` when PREVIOUS_SHA was literally empty or unresolvable — which misses the 'PREVIOUS_SHA resolves but gives wrong answer' case that Vercel hits on first-push PRs. Fix: flip the priority for PREVIEW deploys. Use merge-base first (the stable truth: 'everything on this PR since it left main'), fall back to PREVIOUS_SHA only for the rare shallow-clone scenario where origin/main isn't in Vercel's clone and merge-base returns empty. Main-branch branch (line 6) is UNCHANGED — it correctly wants PREVIOUS_SHA = the last deployed commit, not merge-base (which would be HEAD itself on main and skip every push). Tested locally: - PR branch + web change + PREVIOUS_SHA=HEAD → exit 1 (build) ✓ - PR branch + scripts-only change + PREVIOUS_SHA=HEAD → exit 0 (skip) ✓ - main + PREVIOUS_SHA=valid previous deploy → exit 1 (build) ✓ Related: PR #3346 needed an empty commit to retrigger the preview deploy. After this fix, first-push PRs should deploy without the dance. * chore: retrigger vercel deploy (previous attempt failed on git-provider transient) |
||
|
Elie Habib
|
64edfffdfc |
fix(checkout): implement checkout.redirect_requested — the Dodo handler we were missing (#3346)
* fix(checkout): implement checkout.redirect_requested — the Dodo handler we were missing (revert #3298) Buyers got stuck on /pro after successful Dodo payment because NEITHER pro-test nor dashboard checkout services handled `checkout.redirect_requested` — the event Dodo's SDK fires under `manualRedirect: true` carrying the URL the MERCHANT must navigate to. We were only listening for `checkout.status`, so navigation never happened for Safari users (saw the orphaned about:blank tab). PR #3298 chased the wrong theory (flipped /pro to `manualRedirect: false`, hoping the SDK would auto-navigate). Dodo docs explicitly say that mode disables BOTH `checkout.status` AND `checkout.redirect_requested` ("only when manualRedirect is enabled"), and the SDK's internal redirect is where Safari breaks. Fix: - Revert /pro to `manualRedirect: true` - Add `checkout.redirect_requested` handler in BOTH /pro and dashboard: `window.location.href = event.data.message.redirect_to` - Align `status` read to docs-documented `event.data.message.status` only (drop legacy top-level `.status` guess) - `checkout.link_expired` logged to Sentry (follow-up if volume warrants UX) - Rebuilt public/pro/ bundle on Node 22 (new hash: index-CiMZEtgt.js) Docs: https://docs.dodopayments.com/developer-resources/overlay-checkout ## Test plan - [ ] Vercel preview: complete Dodo test-mode checkout on /pro with 4242 card. Verify console shows `[checkout] dodo event checkout.status {status: "succeeded"}` followed by `checkout.redirect_requested`, and the tab navigates to worldmonitor.app/?wm_checkout=success&... WITHOUT an about:blank second tab. - [ ] Vercel preview: same flow with a 3DS-required test card. - [ ] Vercel preview: dashboard in-app upgrade click → overlay → success → same-origin navigation lands on worldmonitor.app with Dodo's appended ?payment_id=...&status=succeeded&... - [ ] Post-deploy: Sentry breadcrumbs show full event sequence on every success; no new "stuck after paying" user reports in 24h. ## Rollback Single `git revert` + bundle rebuild. Fallback state is PR #3298's broken-for-Safari `manualRedirect: false`. * chore(ci): retrigger Vercel preview build — initial push skipped via ignore-step |
||
|
Elie Habib
|
53c50f4ba9 |
fix(swf): move manifest next to its loader so Railway ships it (#3344)
PR #3336 fixed the yaml dep but the next Railway tick crashed with `ENOENT: no such file or directory, open '/docs/methodology/swf-classification-manifest.yaml'`. Root cause: the loader at scripts/shared/swf-manifest-loader.mjs resolved `../../docs/methodology/swf-classification-manifest.yaml`, which works in a full repo checkout but lands at `/docs/...` (outside `/app`) in the Railway recovery-bundle container. That service has rootDirectory=scripts/ in the dashboard, so NIXPACKS only copies `scripts/` into the image — `docs/` is never shipped. Fix: move the YAML to scripts/shared/swf-classification-manifest.yaml, alongside its loader. MANIFEST_PATH becomes `./swf-classification-manifest.yaml` so the file is always adjacent to the code that reads it, regardless of rootDirectory. Tests: 53/53 SWF tests still pass; biome clean on changed files. |
||
|
Elie Habib
|
9c3c7e8657 |
fix(agent-readiness): align OAuth resource with public MCP origin (#3345)
* fix(agent-readiness): align OAuth resource with actual public MCP origin isitagentready.com's OAuth Protected Resource check enforces an origin match between the scanned host and the metadata's `resource` field (per the spirit of RFC 9728 §3). Our metadata declared `resource: "https://api.worldmonitor.app"` while the MCP endpoint is publicly served at `https://worldmonitor.app/mcp` (per vercel.json's /mcp → /api/mcp rewrite and the MCP card's transport.endpoint). Flip `resource` to `https://worldmonitor.app` across the three places that declare it: - public/.well-known/oauth-protected-resource - public/.well-known/mcp/server-card.json (authentication block) - api/mcp.ts (two WWW-Authenticate resource_metadata pointers) `authorization_servers` intentionally stays on api.worldmonitor.app — that's where /oauth/{authorize,token,register} actually live. RFC 9728 permits AS and resource to be at different origins. No server-side validation breaks: api/oauth/*.js and api/mcp.ts do not bind tokens to the old resource value. * fix(agent-readiness): align docs/tests + add MCP origin guardrail Addresses P1/P2 review on this PR. The resource-origin flip in the previous commit only moved the mismatch from apex to api unless the repo also documents apex as the canonical MCP origin. - docs/mcp.mdx: swap api.worldmonitor.app/mcp -> worldmonitor.app/mcp (OAuth endpoints stay on api.*, only the resource URL changes) - tests/mcp.test.mjs: same fixture update - tests/deploy-config.test.mjs: new guardrail block asserting that MCP transport.endpoint origin, OAuth metadata resource, MCP card authentication.resource, and api/mcp.ts resource_metadata pointers all share the same origin. Includes a regression guard that authorization_servers stays on api.worldmonitor.app (the intentional resource/AS split). |
||
|
Elie Habib
|
dff14ed344 |
feat(agent-readiness): RFC 9727 API catalog + native openapi.yaml serve (#3343)
* feat(agent-readiness): RFC 9727 API catalog + native openapi.yaml serve Closes #3309, part of epic #3306. - Serves the sebuf-bundled OpenAPI spec natively at https://www.worldmonitor.app/openapi.yaml with correct application/yaml content-type (no Mintlify proxy hop). Build-time copy from docs/api/worldmonitor.openapi.yaml. - Publishes RFC 9727 API catalog at /.well-known/api-catalog with service-desc pointing at the native URL, status rel pointing at /api/health, and a separate anchor for the MCP endpoint referencing its SEP-1649 card (#3311). Refs PR #3341 (sebuf v0.11.1 bundle landed). * test(deploy-config): update SPA catch-all regex assertion The deploy-config guardrail hard-codes the SPA catch-all regex string and asserts its Cache-Control is no-cache. The prior commit added openapi.yaml to the exclusion list; this updates the test to match so the guardrail continues to protect HTML entry caching. * fix(agent-readiness): address Greptile review on PR #3343 - Extract openapi.yaml copy into named script `build:openapi` and prefix every web-variant build (build:full/tech/finance/happy/ commodity). prebuild delegates to the same script so the default `npm run build` path is unchanged. Swap shell `cp` for Node's cpSync for cross-platform safety. - Bump service-desc MIME type in /.well-known/api-catalog from application/yaml to application/vnd.oai.openapi (IANA-registered OpenAPI media type). Endpoint Content-Type stays application/yaml for browser/tool compatibility. * fix(agent-readiness): P1 health href + guardrail tests on PR #3343 - status.href in /.well-known/api-catalog was pointing at https://api.worldmonitor.app/health (which serves the SPA HTML, not a health response). Corrected to /api/health, which returns the real {"status":"HEALTHY",...} JSON from api/health.js. - Extend tests/deploy-config.test.mjs with assertions that would have caught this regression: linkset structure, status/service- desc href shapes, and presence of build:openapi across every web-variant build script. |
||
|
Elie Habib
|
0b7069f5dc | chore(railway): force rebuild of seed bundles after infra-error build failure (#3342) | ||
|
Sebastien Melki
|
fcbb8bc0a1 |
feat(proto): unified OpenAPI bundle via sebuf v0.11.0 (#3341)
* feat(proto): generate unified worldmonitor.openapi.yaml bundle Adds a third protoc-gen-openapiv3 invocation that merges every service into a single docs/api/worldmonitor.openapi.yaml spanning all 68 RPCs, using the new bundle support shipped in sebuf 0.11.0 (SebastienMelki/sebuf#158). Per-service YAML/JSON files are untouched and continue to back the Mintlify docs in docs/docs.json. The bundle runs with strategy: all and bundle_only=true so only the aggregate file is emitted, avoiding duplicate-output conflicts with the existing per-service generator. Requires protoc-gen-openapiv3 >= v0.11.0 locally: go install github.com/SebastienMelki/sebuf/cmd/protoc-gen-openapiv3@v0.11.0 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(proto): bump sebuf to v0.11.0 and document unified OpenAPI bundle - Makefile: SEBUF_VERSION v0.7.0 → v0.11.0 (required for bundle support). - proto/buf.gen.yaml: point bundle_server at https://api.worldmonitor.app. - CONTRIBUTING.md: new "OpenAPI Output" section covering per-service specs vs the unified worldmonitor.openapi.yaml bundle, plus a note that all three sebuf plugins must be installed from the pinned version. - AGENTS.md: clarify that `make generate` also produces the unified spec and requires sebuf v0.11.0. - CHANGELOG.md: Unreleased entry announcing the bundle and version bump. Also regenerates the bundle with the updated server URL. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(codegen): regenerate TS client/server with sebuf v0.11.0 Mechanical output of the bumped protoc-gen-ts-client and protoc-gen-ts-server. Two behavioral improvements roll in from sebuf: - Proto enum fields now use the proper `*_UNSPECIFIED` sentinel in default-value checks instead of the empty string, so generated query-string serializers correctly omit enum params only when they actually equal the proto default. - `repeated string` query params now serialize via `forEach(v => params.append(...))` instead of being coerced through `String(req.field)`, matching the existing `parseStringArray()` contract on the server side. All files also drop the `// @ts-nocheck` header that earlier sebuf versions emitted — 0.11.0 output type-checks cleanly under our tsconfig. No hand edits. Reproduce with `make install-plugins && make generate`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(proto): bump sebuf v0.11.0 → v0.11.1, realign tests with repeated-param wire format - Bump SEBUF_VERSION to v0.11.1, pulling in the OpenAPI fix for repeated scalar query params (SebastienMelki/sebuf#161). `repeated string` fields now emit `type: array` + `items.type: string` + `style: form` + `explode: true` instead of `type: string`, so SDK generators consuming the unified bundle produce correct array clients. - Regenerate all 12 OpenAPI specs (unified bundle + Aviation, Economic, Infrastructure, Market, Trade per-service). TS client/server codegen is byte-identical to v0.11.0 — only the OpenAPI emitter was out of sync. - Update three tests that asserted the pre-v0.11 comma-joined wire format (`symbols=AAPL,MSFT`) to match the current repeated-param form (`symbols=AAPL&symbols=MSFT`) produced by `params.append(...)`: - tests/market-service-symbol-casing.test.mjs (2 cases: getAll) - tests/stock-analysis-history.test.mts - tests/stock-backtest.test.mts Locally: test:data 6619/6619 pass, typecheck clean, lint exit 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Apply suggestions from code review Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> |
||
|
Elie Habib
|
df91e99142 |
feat(energy): expand 5 curated registries to 100% of plan target (#3337)
* feat(energy): expand gas pipeline registry 12 → 28 (phase 1a batch 1)
Data validation after v1 launch showed pipelines shipped at ~16% of the
plan target (12 gas + 12 oil vs. the plan's 75 + 75 critical
pipelines). This commit closes ~20% of the gas gap with 16 hand-curated
global additions, every entry carrying a full evidence bundle matching
the schema enforced by scripts/_pipeline-registry.mjs.
New additions by region:
North Sea / NW Europe (6):
europipe-1, europipe-2, franpipe, zeepipe, interconnector-uk-be, bbl
Mediterranean / North Africa (3):
transmed (Enrico Mattei), greenstream (LY→IT, reduced),
meg-maghreb-europe (DZ→ES via MA, offline since Oct 2021)
Middle East (1):
arab-gas-pipeline (EG→LB via JO/SY, offline under Caesar Act)
Former Soviet / Turkey (1):
blue-stream (RU→TR, carries EU sanctions ref)
Asia (3):
west-east-3 (CN internal, 7378 km), myanmar-china-gas (shwe),
igb (interconnector-greece-bulgaria, 2022)
Africa / LatAm (2):
wagp (west african gas pipeline, 4-country transit),
gasbol (bolivia-brazil, 3150 km)
Badge distribution on new entries:
flowing: 12, reduced: 2, offline: 2
First non-Russia-exposure offline entries (MEG — Morocco-Algeria
diplomatic closure, Arab Gas — Syria sanctions) — broadens the
geographic distribution of evidence-bundle-backed non-positive badges.
Registry tests: 17/17 pass (identity, geometry bounds, ISO2 country
codes, evidence contract, capacity-commodity pairing, validateRegistry
negative cases).
Next batches in this phase: oil pipelines +16, then second batches
each commodity to reach plan target (75+75). Tracked in
docs/internal/energy-atlas-registry-expansion.md.
* feat(energy): expand oil pipeline registry 12 → 28 (phase 1a batch 2)
Mirror of the gas batch — 16 hand-curated global additions with full
evidence bundles. Closes ~20% of the oil gap.
New additions by region:
North America (6):
enbridge-mainline (CA→US 3.15 mbd), enbridge-line-3-replacement (2021),
flanagan-south, seaway (Cushing→Gulf), marketlink (TC, Cushing→Gulf),
spearhead
Middle East (3):
sumed (EG crude bypass of Suez, 2.8 mbd),
east-west-saudi (Petroline, 5 mbd — largest single oil pipeline in
the registry by capacity),
ipsa-2 (IQ→SA, offline since Iraq invasion of Kuwait 1990, later
converted to gas on the western stretch)
Central Asia (1):
kazakhstan-china-crude (KZ→CN Alashankou, 2228 km)
Africa (1):
chad-cameroon-cotco (TD→CM Kribi, 1070 km)
South America (2):
ocp-ecuador (heavy crude, 450 kbd),
sote-ecuador (lighter grades, 360 kbd)
Europe (3):
tal-trieste-ingolstadt (IT→DE via AT, 770 kbd),
janaf-adria (HR→RS→HU, 280 kbd),
norpipe-oil (NO→DE North Sea crude, 900 kbd)
Badge distribution on new entries:
flowing: 15, offline: 1 (IPSA-2, regulator-sourced + nationalisation
statement backing the offline badge per the evidence-contract rules).
Registry totals after this batch:
gas: 12 → 28 (37% of plan target 75)
oil: 12 → 28 (37% of plan target 75)
total: 24 → 56
Registry tests: 17/17 registry + 23/23 evidence-derivation = 40/40 pass.
Typecheck-free (JSON only).
Next batches (per docs/internal/energy-atlas-registry-expansion.md):
gas batch 2: +22 → 50 (North Sea remainder, Caspian, Asia)
oil batch 2: +22 → 50 (North Sea remainder, Russia diversified,
Asia long-haul)
* feat(energy): expand gas pipeline registry 28 → 50 (phase 1a batch 3)
Second gas batch, 22 additions, bringing gas to ~67% of the 75-pipeline
plan target. Geographic distribution deliberately skewed this batch
toward under-represented regions (Middle East, Central Asia, South
America, Africa, Southeast Asia) since the first batch filled Europe
and North America.
New additions (22):
North Sea / UK (2):
vesterled (NO→GB, 13 bcm/yr),
cats (UK, 9.6 bcm/yr)
Iran family (3):
iran-turkey-gas (Tabriz→Ankara, 14 bcm/yr, OFAC sanctions ref),
iran-armenia-gas (2.3 bcm/yr),
iran-iraq-basra-gas (reduced state — waiver-dependent flows)
Central Asia (2):
central-asia-center (TM→RU via UZ/KZ, 44 bcm/yr nominal, reduced),
turkmenistan-iran-korpeje (expired contract, reduced)
Caucasus / Turkey (2):
south-caucasus-scp (BTE predecessor to TANAP, 22 bcm/yr),
sakarya-black-sea-tr (2023 Turkish offshore)
China (2):
west-east-1 (4200 km, 17 bcm/yr),
west-east-2 (8700 km, 30 bcm/yr)
South America (2):
bolivia-argentina-yacuiba (reduced),
antonio-ricaurte (CO→VE, offline since 2015, PDVSA sanctions)
Saudi / Middle East (2):
saudi-master-gas-system (SA internal, 95 bcm/yr — largest capacity
in the registry), egypt-jordan-aqaba (AGP south leg, flowing)
Israel-Egypt (1):
israel-egypt-arish-ashkelon (reverse-flow since 2020, IL→EG export)
Planned / FID-stage (5):
galsi-planned (DZ→IT, consortium paused),
eastmed-planned (IL→CY→GR, US political support withdrawn Jan 2022),
trans-saharan-planned (NG→DZ via NE, insurgency + financing unresolved),
morocco-nigeria-offshore-planned (NG→MA 11-country offshore route),
power-of-siberia-2-planned (RU→CN via MN, no binding CNPC contract),
kirkuk-dohuk-turkey-gas-planned (IQ→TR, Baghdad-Erbil dispute)
Badge distribution on new batch:
flowing: 10 (incl. Sakarya 2023 commissioned)
reduced: 3 (CAC, BO-AR, IR-IQ)
offline: 1 (Antonio Ricaurte, CO-VE, with operator statement + sanction)
unknown: 6 (all planned/FID-stage, classifierConfidence 0.6-0.75)
All non-flowing badges have evidence (sanction refs, operator
statements, or press sourcing) per the evidence-contract validator.
Registry totals after this batch:
gas: 28 → 50 (67% of plan target; gas ≥60 gate threshold not yet
hit but approaching)
oil: 28 (unchanged — batch 4 will target oil to 50)
total: 56 → 78
Registry tests: 17/17 pass. Includes 8 new fully-hedged "unknown" /
planned-status entries; validator accepts them.
Next: oil batch 2 (+22 → 50), then gas batch 3 (+10 → 60), oil batch 3
(+10 → 60). After that the gate criteria on pipelines hit and we can
focus on storage / shortages / disruptions.
* feat(energy): expand oil pipeline registry 28 → 50 (phase 1a batch 4)
Second oil batch, 22 additions, bringing oil to 67% of plan target and
matching gas (50 each, 100 total pipelines).
New additions (22):
Russia Baltic export (2):
bps-1 (Primorsk, 1.3 mbd — largest single line in oil registry),
bps-2 (Ust-Luga, 0.75 mbd). Both carry G7+EU price-cap sanctions ref.
North America diversified (3):
enbridge-line-5 (CA→CA via US Straits of Mackinac, ongoing litigation),
keystone-xl-cancelled (CA→US, permit revoked 2021, Biden; TC
terminated Jun 2021; listed for historical + geopolitical
completeness, physicalState=unknown by deriver rule),
trans-panama-pipeline (PA, 0.9 mbd cross-isthmus)
Europe remaining (3):
rotterdam-rhine-rrp (NL→DE, 275 km),
spse (FR→DE Lyon→Karlsruhe, 769 km),
forties-pipeline (UK North Sea, 0.6 mbd),
brent-pipeline (NO→GB Sullom Voe, reduced — Brent field in
decommissioning)
Middle East (2):
khafji-neutral-zone (SA/KW, reduced post-2015 neutral-zone dispute),
ab-1-bahrain (SA→BH, 2018, 0.35 mbd)
Africa (4):
greater-nile-petroleum (SS→SD Port Sudan, 1610 km),
djeno-congo (CG terminal system),
nigeria-forcados-export (reduced — recurring force-majeure),
nigeria-bonny-export (Trans Niger Pipeline, reduced)
Latin America (2):
pemex-nuevo-cactus (MX, 0.44 mbd),
trans-andino (AR→CL, offline since 2006 export restrictions)
Ukraine (1):
odesa-brody (offline, under EU 2022/879 Russian-crude embargo
framework)
Asia (1):
myanmar-china-crude (MM→CN Kunming, 771 km parallel to
myanmar-china-gas)
Caspian (1):
baku-novorossiysk-northern (AZ→RU historical route, reduced, carries
Russian crude price-cap ref)
Historical / planned (2):
kirkuk-haifa-idle (IQ→IL via JO, closed 1948 — listed for
completeness; periodically floated as reopening proposal),
uganda-tanzania-eacop-planned (UG→TZ, under construction, Western
bank-financing pulled but TotalEnergies continues)
Badge distribution on new batch:
flowing: 10
reduced: 6 (Brent decommissioning, Khafji dispute, Greater Nile,
Forcados, Bonny, Baku-Novorossiysk)
offline: 2 (Odesa-Brody, Trans-Andino, Kirkuk-Haifa)
unknown: 2 (Keystone XL cancelled, EACOP under construction)
Wait, Kirkuk-Haifa is offline not among 2. Corrected count:
flowing: 10, reduced: 6, offline: 3 (Odesa-Brody, Trans-Andino,
Kirkuk-Haifa), unknown: 2, plus 1 flowing Myanmar-China-crude = 22.
All non-flowing badges carry supporting evidence (operator statements,
sanction refs, or press citations) per the evidence-contract validator.
Registry totals after this batch:
gas: 50 (67% of plan target)
oil: 28 → 50 (67% of plan target)
total: 78 → 100
Registry tests: 17/17 + 23/23 evidence-derivation = 40/40 pass.
Next batches to hit the 60-each gate criteria from
docs/internal/energy-atlas-registry-expansion.md:
gas batch 3: +10 → 60 (EastMed details, Galsi alternative routes,
minor EU-interconnectors, Nigeria LNG feeder gas lines)
oil batch 3: +10 → 60 (Pluto crude, Chinese Huabei system, Latam
infill: Brazil Campos, Peru Northern Trunk)
After 60/60: hit gate, move to storage expansion.
* feat(energy): gas registry 50 → 75 — plan target hit
Batch 3 adds 25 more gas pipelines, bringing gas to 100% of the
75-pipeline plan target.
New additions by region (25):
- Norwegian transport spine: statpipe, sleipner-karsto, troll-a,
oseberg-gas-transport, asgard-transport (covers the major offshore
export collectors — the rest of the Gassco system)
- Australia: dampier-bunbury (1594 km), moomba-sydney (1299 km)
- Africa: mozambique-rompco (MZ→ZA), escravos-lagos-gas (NG),
tanzania-mtwara-dar, ghana-gas (atuabo)
- Southeast Asia: thailand-malaysia-cakerawala, indonesia-singapore
west-natuna + grissik-sakra
- German hubs for Nord Stream continuation: nel-pipeline, opal-pipeline,
eugal-pipeline (built but dormant after NS2 halt/destruction),
megal-pipeline, gascade-jagal, zeelink-germany
- Russia/Ukraine/EU transit: progress-urengoy-uzhhorod (halted 1 Jan
2025 when Ukraine did not renew transit agreement), trans-austria-gas
- Iran: kish-iran-gas, iran-pakistan-gas-planned (Pakistani segment
stalled since 2014)
- China/HK: china-hong-kong-gas
Badge distribution on new batch: 15 flowing, 4 reduced (NEL, OPAL,
TAG, Escravos-Lagos), 2 offline (EUGAL dormant post-NS2,
Urengoy-Uzhhorod transit halt), 4 sanction-exposed (NS-continuation
pipelines + TAG + Urengoy), 1 unknown (Iran-Pakistan stalled
completion).
Plan progress: gas 50 → 75 (100% of plan target).
Registry tests: 17/17 pass.
* feat(energy): oil registry 50 → 75 — plan target hit
Batch 4 adds 25 more oil pipelines, bringing oil to 100% of the
75-pipeline plan target. Combined with gas at 75, total registry is
150 pipelines — full plan coverage for Phase 1a.
New additions by region (25):
- Latin America: colombia-cano-limon-covenas (ELN-sabotaged, reduced),
colombia-ocensa (main trunk), peru-norperuano (reduced from jungle
spills + protests), ecuador-lago-agrio-orellana,
venezuela-anzoategui-puerto-la-cruz (under OFAC PDVSA sanctions),
mexico-salina-cruz-minatitlan, mexico-madero-cadereyta,
mexico-gulf-coast-pipeline (Tuxpan-Mexico City)
- Africa: angola-cabinda-offshore, south-sudan-kenya-lamu-planned
(LAPSSET)
- Middle East: iran-abadan-isfahan, iran-neka-tehran (reduced,
Caspian swap arrangements), saudi-abqaiq-yanbu-products,
iraq-strategic-pipeline (1000 km north-south), iraq-bai-hassan,
oman-muscat-export (Fahud-Mina al-Fahal), uae-habshan-ruwais
- Asia-Pacific: india-salaya-mathura (1770 km, largest Indian crude
trunk), india-vadinar-kandla, india-mundra-bhatinda,
china-qinhuangdao-tianjin-huabei, china-yangzi-hefei-hangzhou
- Russia East: russia-sakhalin-2-crude, russia-komsomolsk-perevoznaya,
russia-omsk-pavlodar (cross-border to KZ)
Badge distribution on this batch: 18 flowing, 6 reduced, 1 unknown
(LAPSSET planned). Sanctions-exposure diversified: Iran framework (3),
Venezuela/PDVSA (1), Russian price-cap (3). All non-flowing badges
carry supporting evidence per validator rules.
Phase 1a final state (pipelines):
gas: 12 → 75 (100% of plan target, 6 batches)
oil: 12 → 75 (100% of plan target, 6 batches)
total: 24 → 150
Geographic distribution now global:
- Russia-exposure: ~22 of 150 entries (~15%, down from 50% at v1)
- US-only: ~8 (~5%, down from 33% storage-side skew)
- Six continents represented in active infrastructure
- Historical + planned pipelines (Kirkuk-Haifa, Keystone XL cancelled,
EACOP u/c, EastMed planned, GALSI planned, TSGP planned,
Nigeria-Morocco offshore, Power of Siberia 2, Iran-Pakistan Peace,
LAPSSET) listed with honest 'unknown' physicalState per validator
Registry tests: 17/17 pass.
Phase 1a complete. Next phase (per
docs/internal/energy-atlas-registry-expansion.md):
- Phase 2: storage 21 → ~200 (+179) via curation + GIIGNL/GIE/EIA
- Phase 3: shortages 14 → 28 countries
- Phase 4: disruptions 12 → 50 events
* feat(energy): shortages 15 → 29 entries across 28 countries — plan target hit
+14 country additions matching the 28-country plan target. The
validator's 'confirmed severity requires authoritative source' rule
caught two of my drafts (Myanmar + Sudan) where I had labeled them
confirmed with press-only evidence because regulator/operator sources
under a junta + active civil war are not independently verifiable.
Downgraded both to 'watch' with an inline note explaining the
evidence-quality choice — exactly the validator's intended behavior
(better to under-claim than over-claim severity when the authoritative
channel is broken).
New shortages (14):
- BD diesel: BPC LC delays, regulator-confirmed
- ZA diesel: loadshedding demand spike
- AO diesel: Luanda/Benguela depot delays
- MZ diesel: FX-allocation import constraints
- ZM diesel: mining-sector demand + TAZAMA product tightness
- MW diesel: FX shortfalls + MERA rationing
- GH petrol: Tema port congestion
- MM diesel: post-coup chronic (watch, press-only evidence)
- MN diesel: winter logistics
- CO diesel: trucker strike cycles
- UA diesel: war-driven chronic (confirmed — Ministry of Energy source)
- SY diesel: Caesar Act chronic (confirmed — Syrian Ministry statement)
- SD diesel: civil-war disruption (watch, press-only)
- DE heating_oil: Rhine low-water logistics (watch)
Badge distribution on new batch: 3 confirmed (BD, UA, SY — all with
regulator/operator evidence), 11 watch.
Plan progress:
shortages: 15 → 29 entries (28 unique countries = 100% of plan)
gas: 75 (100%)
oil: 75 (100%)
storage: 21 (unchanged, next batch)
disruptions: 12 (unchanged, next batch)
Registry tests: 19/19 pass.
* feat(energy): disruption event log 12 → 52 events — plan target hit
+40 historical and ongoing events covering the asset registry,
bringing disruptions to 104% of the 50-event plan target. Every event
ties to an assetId now in pipelines/storage registries (following the
75-gas + 75-oil + 21-storage registry expansion in the preceding
commits).
New additions by eventType:
Sabotage / war (7):
- abqaiq-khurais-drone-strike-2019 (Saudi, 5.7 mbd removed 11 days)
- russia-refinery-drone-strikes-2024 (Ukrainian drone strike series)
- houthi-red-sea-attacks-2024 (indirect SuMed demand impact)
- russia-ukraine-oil-depot-strikes-2022 (series)
- nigeria-trans-niger-attacks-2024 (Bonny system)
- bai-hassan-attack-2022 (Iraq Bai Hassan)
- sudan-pipeline-attacks-2023 (Greater Nile disruption)
Sanctions (7):
- russia-price-cap-implementation-2022 (G7+EU $60/bbl cap)
- eu-oil-embargo-2022 (6th package)
- pdvsa-designation-2019 (Venezuela)
- btc-kurdistan-shutdown-2023 (ICC ruling, ongoing)
- ipsa-nationalization-2001 (SA nationalised after Iraq invasion of Kuwait)
- arctic-lng-2-foreign-partner-withdrawal-2024
- yamal-lng-arctic-sanctions-ongoing (Novatek)
- ogm-moldova-transit-2022
Mechanical (4):
- druzhba-contamination-2019 (chlorides, 3-month shut)
- keystone-milepost-14-leak-2022 (Kansas, 22-day shut, 14k bbl spill)
- forties-crack-2017 (Red Moss hairline)
- ocensa-ocp-ecuador-suspensions-2022 (Amazon landslide)
Weather (2):
- hurricane-ida-lng-2021 (Gulf coast LNG shutdown)
- rotterdam-hub-low-water-2022 (Rhine 2.5-month disruption)
Commercial (9):
- cpc-blockage-threat-2022 (Russian court 30-day halt threat)
- gme-closure-2021 (Algeria-Morocco MEG)
- ukraine-transit-end-2025 (Progress pipeline halted 1 Jan 2025)
- eugal-dormant-since-2022 (NS2 knock-on)
- keystone-xl-permit-revoked-2021 (Biden day-1)
- antonio-ricaurte-halt-2015 (CO→VE gas export halt)
- langeled-brent-decommissioning-2020
- eacop-financing-2023 (Western bank withdrawal)
- dolphin-qatar-uae-commercial-2024 (contract renegotiation)
- trans-austria-gas-reduction-2022 (Gazprom volume drops)
- cushing-stocks-tank-bottoms-2022
- spr-drawdown-2022-2023 (largest ever 180 mbbl release)
- zhoushan-storage-expansion-2023
- fujairah-stockbuild-2024
- futtsu-lng-demand-decline-2024
- bolivia-diesel-import-cut-2023 (GASBOL)
- myanmar-china-gas-reduced-2023
- yamal-europe-poland-halt-follow-on-2024
Maintenance (1): gladstone-lng-maintenance-2023
Ongoing events (endAt=null): 31 of 52 (~60%). Reflects the structural
reality that many 2022-era sanctions + war events remain live in 2026.
Plan progress:
gas: 75 (100%)
oil: 75 (100%)
storage: 21 (unchanged, next batch)
shortages: 29 (100% — 28 countries)
disruptions: 12 → 52 events (104% of plan)
Registry tests: 16/16 pass.
* feat(energy): storage registry 21 → 66 (storage batch 1)
+45 facilities, 33% of plan. Focus: European UGS + second LNG wave.
European UGS additions (35 — mostly filling the gap against GIE AGSI+
coverage which has ~140 EU sites; we now register the majority of
operationally significant ones with non-trivial working capacity):
Germany (9): bierwang, etzel-salt-cavern, jemgum, krummhoern,
peckensen, reckrod, uelsen, xanten, epe-salt-cavern
Netherlands (3): alkmaar, norg (largest NL, 59.2 TWh), zuidwending
Austria (3): 7fields-schonkirchen (24.6 TWh), baumgarten-uhs,
puchkirchen
France (7): chemery (38.5 TWh), cerville-velaine, etrez, manosque,
lussagnet (35 TWh), izaute
Italy (4): minerbio (45 TWh, largest IT), ripalta, sergnano,
brugherio
UK (2): rough (reduced, post-2017 partial reopening 2022), hornsea
Central/Eastern Europe (8): damborice (CZ), lobodice (CZ),
lab-slovakia (36 TWh), hajduszoboszlo (HU), mogilno (PL),
lille-torup (DK), incukalns (LV), gaviota (ES)
Russia (1): kasimovskoe (124 TWh — Gazprom UGS flagship; EU sanctions
ref carried as evidence)
LNG terminals (9 additions to round out global coverage):
- US: freeport-lng, cameron-lng, cove-point-lng, elba-island-lng
- Middle East: qalhat-lng (Oman), adgas-das-island (UAE)
- Russia: sakhalin-2-lng (sanctions-exposed)
- Indonesia: tangguh-lng, bontang-lng (reduced — declining upstream)
Badge distribution on this batch: 43 operational, 2 reduced (Rough,
Bontang). Most entries from GIE AGSI+ fill-disclosed data; Russian
site + LNG terminals fill-not-disclosed (operator choice + sanctions).
Plan progress:
gas pipelines: 75 (100%)
oil pipelines: 75 (100%)
fuel shortages: 29 / 28 countries (100%)
disruptions: 52 (104%)
storage: 21 → 66 (33% of ~200 target)
Registry tests: 21/21 pass.
Next storage batches remaining:
batch 2 (+45): more European UGS tail + Asian national reserves
(CN SPR, IN SPR, JP national reserves, KR KNOC)
batch 3 (+45): LNG import terminals + additional US tank farms +
European tank farms (Rotterdam detail, ARA sub-sites)
batch 4 (+45): remainder to ~200
* feat(energy): storage registry 66 → 110 (storage batch 2)
+44 facilities. Focus: Asian national reserves + global LNG coverage
+ Singapore/ARA tank-farm detail.
Asian national reserves (11):
- IN ISPRL: vizag (9.8 Mb), mangalore (11 Mb), padur (17.4 Mb)
- CN: zhanjiang (45 Mb), huangdao (20 Mb) — fill opaque, press-only
- JP JOGMEC: shibushi (31.2 Mb), kiire (22 Mb), mutsu-ogawara (28 Mb)
- KR KNOC: yeosu (42 Mb), ulsan (33 Mb), geoje (47 Mb)
LNG export additions (11):
- Australia: pluto-lng, prelude-flng (reduced), darwin-lng (reduced
upstream)
- Southeast Asia: mlng-bintulu (29.3 Mtpa — largest in registry),
brunei-lng, donggi-senoro-lng
- Africa: angola-lng (reduced), equatorial-guinea-lng, hilli-episeyo-flng
- Pacific: png-lng
- Caribbean: trinidad-atlantic-lng (reduced)
- Mexico: costa-azul-lng (2025 reverse-to-export commissioning)
LNG import (12):
- UK: south-hook-lng (21 Mtpa), dragon-lng
- EU: zeebrugge-lng, dunkerque-lng, fos-cavaou-lng,
montoir-de-bretagne-lng, gate-terminal (Rotterdam),
revithoussa-lng
- Turkey: aliaga-ege-gaz-lng
- Chile: mejillones-lng, quintero-bay-lng
Tank farms (10):
- Africa: saldanha-bay (ZA 45 Mb)
- Norway: mongstad-crude
- ARA: antwerp-petroleum-hub (BE 55 Mb), amsterdam-petroleum-hub
- Asia hubs: singapore-jurong (120 Mb — largest in registry),
singapore-pulau-ayer-chawan, thailand-sriracha, korea-gwangyang-crude
- Russia Baltic: ust-luga-crude-terminal, primorsk-crude-terminal
(both carry Russian price-cap sanction refs)
Badge distribution on this batch: 39 operational, 5 reduced (Prelude,
Darwin, Angola, Bontang — no wait Bontang already in. Correct: Prelude,
Darwin, Angola, Trinidad).
Plan progress:
gas pipelines: 75 (100%)
oil pipelines: 75 (100%)
fuel shortages: 29 / 28 countries (100%)
disruptions: 52 (104%)
storage: 66 → 110 (55% of ~200 target)
Registry tests: 21/21 pass.
Next batches remaining: ~90 more storage to hit ~200
batch 3 (+45): Middle East tank farms, Chinese coastal commercial
storage, EU UGS tail, African LNG import
batch 4 (+45): remainder to 200
* feat(energy): storage registry 110 → 155 (storage batch 3)
Adds 45 facilities toward 200 plan target:
- 7 Middle East export terminals (Kharg, Sidi Kerir, Mina al-Ahmadi,
Mesaieed, Jebel Dhanna, Mina al-Fahal, Bandar Imam Khomeini)
- 10 EU UGS tail (Reitbrook, Empelde, Kirchheilingen, Stockstadt,
Nüttermoor, Grijpskerk, Târgu Mureș, Třanovice, Uhřice, Háje)
- 4 Chinese coastal crude (Yangshan, Qingdao, Rizhao, Maoming)
- 6 EU LNG import tail (La Spezia, Adriatic, OLT Livorno, Klaipeda,
Mugardos, Cartagena)
- 5 Indian LNG import (Hazira, Kochi reduced, Ennore, Mundra, Dabhol)
- 6 Japan/Korea LNG import (Chita, Negishi, Sodegaura, Himeji,
Pyeongtaek, Incheon)
- 5 NA tank farms (Lake Charles, Corpus Christi, Patoka, Edmonton,
Hardisty)
- 2 Asia-Pacific (Kaohsiung, Nghi Son)
Registry validator: 21/21 tests pass.
* feat(energy): storage registry 155 → 200 (storage batch 4 — plan target hit)
Final batch brings storage to the 200-facility plan target with broad
geographic + facility-type coverage.
New entries (45):
- 6 LNG export: NLNG Bonny (NG, reduced), Arzew (DZ), Skikda (DZ),
Perú LNG, Calcasieu Pass (US), North West Shelf Karratha (AU)
- 7 LNG import: Świnoujście (PL), Krk FSRU (HR), Wilhelmshaven FSRU (DE),
Brunsbüttel (DE), Map Ta Phut (TH), Port Qasim (PK), Batangas (PH)
- 6 UGS: Bilche-Volytsko-Uherske (UA, 154 TWh — largest Europe), Banatski
Dvor (RS), Okoli (HR), Yela (ES), Loenhout (BE), Kushchevskoe (RU)
- 26 crude tank farms: José Terminal (VE, sanctioned), Santos (BR),
TEBAR São Sebastião (BR), Dos Bocas (MX), Bonny (NG, reduced), Es
Sider (LY, reduced), Ras Lanuf (LY, reduced), Ceyhan (TR), Puerto
Rosales (AR), Novorossiysk Sheskharis (RU, sanctioned), Kozmino (RU,
sanctioned), Tema (GH, reduced), Mombasa (KE), Abidjan SIR (CI),
Juaymah (SA), Ras Tanura (SA), Yanbu (SA), Kirkuk (IQ, reduced),
Basra Gulf (IQ), Djibouti Horizon (DJ), Yokkaichi (JP), Mailiao
(TW), Ventspils (LV, reduced), Gdańsk Naftoport (PL), Constanța
(RO), Wood River IL (US).
Geographic balance improved: Africa coverage (NG, DZ, LY, GH, KE, CI,
DJ) from 5 to 12 countries; first Iraq + Saudi entries; Balkans +
Ukraine + Romania now covered. Type mix: UGS 56, SPR 15, LNG export 33,
LNG import 38, crude tank farm 58.
Non-operational entries all carry authoritative evidence (press
operator statements + sanctionRefs for Russia/Venezuela).
Registry validator: 21/21 tests pass. Total: 200 facilities across 55
countries. Plan target hit.
* fix(energy): address Greptile review findings on registries
P1 — abqaiq-khurais-drone-strike-2019 (energy-disruptions.json):
capacityOfflineMbd was 5.7 (plant-level Saudi production loss headline)
against assetId east-west-saudi (5.0 mbd pipeline). Capped offline
figure at the linked pipeline's 5.0 mbd ceiling; moved the 5.7 mbd
historical headline into shortDescription with an explanatory note.
Preserves capacity-offline ≤ asset-capacity invariant for downstream
consumers.
P1 — russia-price-cap-implementation-2022 (energy-disruptions.json):
was linked to assetId espo (land pipeline to China — explicitly out of
scope for G7/EU price cap). Relinked to primorsk-crude-terminal
(largest Baltic seaborne crude export terminal, directly affected);
assetType pipeline → storage. Updated shortDescription to clarify
tanker-shipment scope + out-of-scope note for ESPO.
P2 — 13 reduced-state pipelines missing press citation text
(pipelines-gas.json × 8 + pipelines-oil.json × 5):
Added operatorStatement sentences naming the press/regulator sources
backing each reduction claim (Reuters, NNPC/Chevron releases, NIGC,
Pemex annual reports, S&P Platts, IEA Gas Market Report, BBC, etc.).
Clears the evidence-source-type gap flagged by Greptile for entries
that declared physicalStateSource: "press" with a null statement.
All 6583 data tests + 94 registry tests still pass.
* style(energy): restore compact registry formatting (preserve Greptile-fix evidence)
Prior commit
|
||
|
Elie Habib
|
9f208848b6 |
fix(deps): add yaml to scripts/package.json (Railway installs from THIS) (#3336)
PR #3333 added `yaml` to the root package.json, but the Railway seed-bundle-resilience-recovery service builds with rootDirectory pointing at scripts/ and NIXPACKS auto-detects scripts/package.json as the install manifest. Root package.json is never visited during the container build, so yaml stayed missing and the seeder crashed again at 07:39:26 UTC with the identical ERR_MODULE_NOT_FOUND. Adding yaml ^2.8.3 (matching the root promotion) to scripts/package.json so NIXPACKS' `npm install --prefix scripts` lands it in /app/node_modules/yaml. scripts/shared/swf-manifest-loader.mjs can then resolve the bare specifier. Keeping yaml in the root package.json too — it's harmless noise for local dev + validation bundle (which also imports it via tsx), and defensive for any future consumer that runs against the root deps. Future question worth a separate PR: do we want Railway services pointing at `scripts/` as rootDir, or should we move to a proper per-service Dockerfile that makes the dep source explicit? The current state is easy to miss because the Railway dashboard config is invisible from the repo — second seeder to trip this exact hazard. |
||
|
Elie Habib
|
8ea4c8f163 |
feat(digest-dedup): replayable per-story input log (opt-in, no behaviour change) (#3330)
* feat(digest-dedup): replayable per-story input log (opt-in, no behaviour change)
Ship the measurement layer before picking any recall-lift strategy.
Why: the current dedup path embeds titles only, so brief-wire headlines
that share a real event but drop the geographic anchor (e.g. "Alleged
Coup: defendant arrives in court" vs "Trial opens after Nigeria charges
six over 2025 coup plot") can slip past the 0.60 cosine threshold. To
tune recall without regressing precision we need a replayable per-tick
dataset — one record per story with the exact fields any downstream
candidate (title+slug, LLM-canonicalise, text-embedding-3-large, cross-
encoder re-rank, etc.) would need to score.
This PR ships ONLY the log. Zero behaviour change:
- Opt-in via DIGEST_DEDUP_REPLAY_LOG=1 (default OFF).
- Writer is best-effort: all errors swallowed + warned, never affects
digest delivery. No throw path.
- Records include hash, originalIndex, isRep, clusterId, raw +
normalised title, link, severity/score/mentions/phase/sources,
embeddingCacheKey, hasEmbedding sidecar flag, and the tick's config
snapshot (mode, clustering, cosineThreshold, topicThreshold, veto).
- clusterId derives from rep.mergedHashes (already set by
materializeCluster) so the orchestrator is untouched.
- Storage: Upstash list keyed by {variant}:{lang}:{sensitivity}:{date}
with 30-day EXPIRE. Date suffix caps per-key growth; retention
covers the labelling cadence + cross-candidate comparison window.
- Env flag is '1'-only (fail-closed on typos, same pattern as
DIGEST_DEDUP_MODE).
Activation path (post-merge): flip DIGEST_DEDUP_REPLAY_LOG=1 on the
seed-digest-notifications Railway service. Watch one cron tick for the
RPUSH + EXPIRE pair (or a single warn line if creds/upstream flake),
then leave running for at least one week to accumulate calibration data.
Tests: 21 unit tests covering flag parsing, key shape + sanitisation,
record field correctness (isRep, clusterId, embeddingCacheKey,
hasEmbedding, tickConfig), pipeline null/throw handling, malformed
input. Existing 77 dedup tests unchanged and still green.
* fix(digest-dedup): capture topicGroupingEnabled in replay tickConfig
Review catch (PR #3330): the tickConfig snapshot omitted
topicGroupingEnabled even though readOrchestratorConfig returns it and
the digest's post-dedup topic ordering gates on it. A tick run with
DIGEST_DEDUP_TOPIC_GROUPING=0 serialised identically to a default
tick, making those runs non-replayable for the calibration work this
log is meant to enable.
Add topicGroupingEnabled to the recorded tickConfig. One-line schema
fix + regression test asserting topic-grouping-off ticks serialise
distinctly from default.
22/22 tests pass.
* fix(digest-dedup): await replay-log write to survive explicit process.exit
Review catch (PR #3330): the fire-and-forget `void writeReplayLog(...)`
call could be dropped on the explicit-exit paths — the brief-compose
failure gate at line 1539 and main().catch at line 1545 both call
process.exit(1). Unlike natural exit, process.exit does not drain
in-flight promises, so the last N ticks' replay records could be
silently lost on runs where measurement fidelity matters most.
Fix: await the writeReplayLog call. Safe because:
- writeReplayLog returns synchronously when the flag is off
(replayLogEnabled check is the first thing it does)
- It has a top-level try/catch that always returns a result object
- The Upstash pipeline call has a 10s timeout ceiling
- buildDigest already awaits many Upstash calls (dedup, compose,
render) so one more is not a hot-path concern
Comment block added above the call explains why the await is
deliberate — so a future refactor doesn't revert it to void thinking
it's a leftover.
No test change: existing writeReplayLog unit tests already cover the
disabled / empty / success / error paths. The fix is a single-keyword
change in a caller that was already guaranteed-safe by the callee's
contract.
* refactor(digest-dedup): address Greptile P2 review comments on replay log
Three non-blocking polish items from the automated review, bundled
because they all touch the same new module and none change behaviour.
1. tsMs captured BEFORE deduplicateStories (seed-digest-notifications.mjs).
Previously sampled after dedup returned, so briefTickId reflected
dedup-completion time rather than tick-start. For downstream readers
the natural reading of "briefTickId" is when the tick began
processing; moved the Date.now() call to match that expectation.
Drift is maybe 100ms-2s on cold-cache embed calls — small, but
moving it is free.
2. buildReplayLogKey emptiness check now strips ':' and '-' in addition
to '_'. A pathological ruleId of ':::' previously passed through
verbatim, producing keys like `digest:replay-log:v1::::2026-04-23`
that confuse redis-cli's namespace tooling (SCAN / KEYS / tab
completion). The new guard falls back to "unknown" on any input
that's all separators. Added a regression test covering the
':::' / '---' / '___' / mixed cases.
3. tickConfig is now a per-record shallow copy instead of a shared
reference. Storage is unaffected (writeReplayLog serialises each
record via JSON.stringify independently) but an in-memory consumer
that mutated one record's tickConfig for experimentation would have
silently affected all other records in the same batch. Added a
regression test asserting mutation doesn't leak across records.
Tests: 24/24 pass (22 prior + 2 new regression). Typecheck + lint clean.
|
||
|
Elie Habib
|
dd95a4e06d |
fix(idb-cleanup): swallow TransactionInactiveError in idempotent IDB cursor loops (#3335)
Sentry WORLDMONITOR-NX: iOS Safari kills in-flight IDB transactions when the tab backgrounds. Our idle detector fires `[App] User idle - pausing animations to save resources` right before the browser suspends — any `cursor.delete()` / `cursor.continue()` mid-iteration then throws TransactionInactiveError synchronously inside onsuccess. Both affected sites are idempotent cleanup (`cleanOldSnapshots`, `deleteFromIndexedDbByPrefix`); swallowing the throw lets the next run resume from where we left off. `main.ts` beforeSend keeps TransactionInactiveError surfaced for first-party stacks (storage.ts, persistent-cache.ts, vector-db.ts), so this is the correct layer to handle the background-kill case. |
||
|
Elie Habib
|
8b12ecdf43 |
fix(aviation): seeder writes delays-bootstrap aggregate (close EMPTY-on-quiet-traffic alarm) (#3334)
* fix(aviation): seeder writes delays-bootstrap aggregate (close EMPTY-on-quiet-traffic alarm) api/health.js BOOTSTRAP_KEYS.flightDelays points at aviation:delays-bootstrap:v1, but no seeder ever produced it — the key was only written as a 1800s side-effect inside list-airport-delays.ts. Quiet user-traffic windows >30 min let the bootstrap expire, tripping EMPTY (CRIT) even with healthy upstream FAA + intl + NOTAM seeds. PR #3073 (Apr 13) doubled the cron cadence to 30 min, putting the bootstrap TTL right at the failure edge. Make seed-aviation.mjs the canonical writer: - New writeDelaysBootstrap() reads FAA + intl + NOTAM from Redis, applies the same NOTAM merge + Normal-operations filler the RPC builds, writes aviation:delays-bootstrap:v1 with TTL=7200 (~4 missed cron ticks of cushion). - Called pre-runSeed (last-good intl, covers intl-fail tick) AND inside afterPublishIntl (this-tick intl, happy-path overwrite). - Bump RPC's incidental write TTL 1800 → 7200 so a user-triggered RPC doesn't shorten the seeder's expiry and re-create the failure mode. NOTAM merge logic + filler shape are now mirrored in two files (seeder + RPC's _shared.ts). Both carry comments pointing at the other to surface drift risk. Verified: typecheck (both tsconfigs) clean; node --test tests/aviation-*.test.mjs green; full test:data 6590/6590 green. * fix(aviation): seeder writes restrictedIcaos + bootstrap unwraps intl envelope PR #3334 review (P1 + P2): P1 — bootstrap silently dropped NOTAM restrictions seedNotamClosures() only tracked NOTAM_CLOSURE_QCODES; the live RPC's classifier in server/worldmonitor/aviation/v1/_shared.ts also derives restrictions via NOTAM_RESTRICTION_QCODES (RA, RO) + restriction code45s + restriction-text regex. Seeded NOTAM payload only had `closedIcaos`, so restrictedIcaos was always empty in Redis — both the new bootstrap aggregate AND the RPC's seed-read path silently dropped every NOTAM restriction. Mirror the full classifier from _shared.ts:438-452; side-car write now includes restrictedIcaos and seed-meta count reflects closures + restrictions. P2 — pre-runSeed bootstrap built with no intl alerts on intl-fail tick runSeed wraps the canonical INTL_KEY in {_seed, data} when declareRecords is enabled. writeDelaysBootstrap()'s upstashGet only JSON.parsed — no envelope unwrap — so intlPayload.alerts was undefined on the pre-runSeed bootstrap-build path, and an intl-fail tick would publish a bootstrap with all intl alerts dropped instead of preserving the last-good snapshot. Add upstashGetUnwrapped() (delegates to unwrapEnvelope from _seed-envelope-source.mjs); use it for all three reads (FAA/NOTAM bare values pass through unchanged via unwrapEnvelope's permissive path). Verified: typecheck (both tsconfigs) clean; aviation + edge-functions tests green; full test:data 6590/6590 green. * fix(aviation): bootstrap iterates union of seeder + RPC airport registries PR #3334 review (P2 ×2): P2 — AIRPORTS vs MONITORED_AIRPORTS registry drift Today the two diverge by ~45 iata codes (29 RPC-only, 16 seeder-only). Pre-fix the bootstrap iterated the seeder's local AIRPORTS list for Normal-operations filler and NOTAM airport lookup, so 29 monitored airports never appeared in the bootstrap aggregate even though the live RPC included them. Fix: parse src/config/airports.ts as text at startup (regex over the static const), memoise the parse, build a by-iata Map union (seeder wins on conflict for canonical meta), and iterate that for both NOTAM lookup and filler. First-run divergence summary logged to surface future drift in cron logs without blocking writes. Degrades to seeder AIRPORTS only with a warning if parse fails. P2 — afterPublishIntl receives raw pre-transform data runSeed forwards the RAW fetchIntl() result to afterPublish, NOT the publishTransform()'d shape. Today publishTransform is a pass-through wrapper so data.alerts is correct, but coupling is subtle — added an inline CONTRACT comment so a future publishTransform mutation doesn't silently drift bootstrap from INTL_KEY. Verified: typecheck (both tsconfigs) clean; aviation + edge-functions tests green; full test:data 6590/6590 green; standalone parse harness recovers all 111 MONITORED_AIRPORTS rows. |
||
|
Elie Habib
|
a7bd1248ac |
chore(lint): exclude docs/brainstorms and docs/ideation from lint:md (#3332)
Both dirs are in .gitignore (alongside docs/plans which was already excluded from lint:md). Brings the lint:md exclusion list in sync with .gitignore so pre-push hooks don't flag local-only working docs that the repo isn't tracking anyway. Observed failure mode before: writing a brainstorm doc locally under docs/brainstorms/ for context during a session, then pushing any feature branch → markdownlint runs against the untracked doc, blocks the push until the local doc is deleted or its formatting is scrubbed to markdownlint's satisfaction. The doc never enters git, so the lint work is pure friction. No functional change to shipping code. |
||
|
Elie Habib
|
1958b34f55 |
fix(digest-dedup): CLUSTERING typo fallback fails closed to complete-link (#3331)
DIGEST_DEDUP_CLUSTERING previously fell to 'single' on unrecognised values, which silently defeated the documented kill switch. A typo like `DIGEST_DEDUP_CLUSTERING=complet` during an over-merge incident would stick with the aggressive single-link merger instead of rolling back to the conservative complete-link algorithm. Mirror the DIGEST_DEDUP_MODE typo pattern (PR #3247): - Unrecognised value → fall to 'complete' (SAFE / conservative). - Surface the raw value via new `invalidClusteringRaw` config field. - Emit a warn line on the dedup orchestrator's entry path so operators see the typo alongside the kill-switch-took-effect message. Valid values 'single' (default), 'complete', unset, empty, and any case variation all behave unchanged. Only true typos change behaviour — and the new behaviour is the kill-switch-safe one. Tests: updated the existing case that codified the old behaviour plus added coverage for (a) multiple typo variants falling to complete with invalidClusteringRaw set, (b) case-insensitive valid values not triggering the typo path, and (c) the orchestrator emitting the warn line even on the jaccard-kill-switch codepath (since CLUSTERING intent applies to both modes). 81/81 dedup tests pass. |
||
|
Elie Habib
|
58589144a5 |
fix(deps): promote yaml from transitive peer to top-level dependency (#3333)
Railway seed-bundle-resilience-recovery crashed at 06:36:18 UTC on first tick post-#3328 with: Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'yaml' imported from /app/shared/swf-manifest-loader.mjs `scripts/shared/swf-manifest-loader.mjs` (landed in #3319) imports `parse` from `yaml` to read `docs/methodology/swf-classification-manifest.yaml`. The package is present locally (as a peer of other deps), but Railway installs production deps only — transitive peers don't land in /app/node_modules, so the seeder exits 1 before any work. Adding `yaml ^2.8.3` to `dependencies` so `npm ci` in the container installs it. Version matches the already-on-disk resolution in package-lock. No consumer changes needed. Unblocks the first Sovereign-Wealth Railway run on the resilience-recovery bundle. |
||
|
Elie Habib
|
d3d406448a |
feat(resilience): PR 2 §3.4 recovery-domain weight rebalance (#3328)
* feat(resilience): PR 2 §3.4 recovery-domain weight rebalance
Dials the two PR 2 §3.4 recovery dims (liquidReserveAdequacy,
sovereignFiscalBuffer) to ~10% share each of the recovery-domain
score via a new per-dimension weight channel in the coverage-weighted
mean. Matches the plan's direction that the sovereign-wealth signal
complement — rather than dominate — the classical liquid-reserves
and fiscal-space signals.
Implementation
- RESILIENCE_DIMENSION_WEIGHTS: new Record<ResilienceDimensionId, number>
alongside RESILIENCE_DOMAIN_WEIGHTS. Every dim has an explicit entry
(default 1.0) so rebalance decisions stay auditable; the two new
recovery dims carry 0.5 each.
Share math at full coverage (6 active recovery dims):
weight sum = 4 × 1.0 + 2 × 0.5 = 5.0
each new-dim share = 0.5 / 5.0 = 0.10 ✓
each core-dim share = 1.0 / 5.0 = 0.20
Retired dims (reserveAdequacy, fuelStockDays) keep weight 1.0 in
the map; their coverage=0 neutralizes them at the coverage channel
regardless. Explicit entries guard against a future scorer bug
accidentally returning coverage>0 for a retired dim and falling
through the `?? 1.0` default — every retirement decision is now
tied to a single explicit source of truth.
- coverageWeightedMean (_shared.ts): refactored to apply
`coverage × dimWeight` per dim instead of `coverage` alone. Backward-
compatible when all weights default to 1.0 (reduces to the original
mean). All three aggregation callers — buildDomainList, baseline-
Score, stressScore — pick up the weighting transparently.
Test coverage
1. New `tests/resilience-recovery-weight-rebalance.test.mts`:
pins the per-dim weight values, asserts the share math
(0.10 new / 0.20 core), verifies completeness of the weight map,
and documents why retired dims stay in the map at 1.0.
2. New `tests/resilience-recovery-ordering.test.mts`: fixture-based
Spearman-proxy sensitivity check. Asserts NO > US > YE ordering
preserved on both the overall score and the recovery-domain
subscore after the rebalance. (Live post-merge Spearman rerun
against the PR 0 snapshot is tracked as a follow-up commit.)
3. resilience-scorers.test.mts fixture anchors updated in lockstep:
baselineScore: 60.35 → 62.17 (low-scoring liquidReserveAdequacy
+ partial-coverage SWF now contribute ~half the weight)
overallScore: 63.60 → 64.39 (recovery subscore lifts by ~3 pts
from the rebalance, overall by ~0.79)
recovery flat mean: 48.75 (unchanged — flat mean doesn't apply
weights by design; documents the coverage-weighted diff)
Local coverageWeightedMean helper in the test mirrors the
production implementation (weights applied per dim).
Methodology doc
- New "Per-dimension weights in the recovery domain" subsection with
the weight table and a sentence explaining the cap. Cross-references
the source of truth (RESILIENCE_DIMENSION_WEIGHTS).
Deliberate non-goals
- Live post-merge Spearman ≥0.85 check against the PR 0 baseline
snapshot. Fixture ordering is preserved (new ordering test); the
live-data check runs after Railway cron refreshes the rankings on
the new weights and commits docs/snapshots/resilience-ranking-live-
post-pr2-<date>.json. Tracked as the final piece of PR 2 §3.4
alongside the health.js / bootstrap graduation (waiting on the
7-day Railway cron bake-in window).
Tests: 6588/6588 data-tier tests pass. Typecheck clean on both
tsconfig configs. Biome clean on touched files. NO > US > YE
fixture ordering preserved.
* fix(resilience): PR 2 review — thread RESILIENCE_DIMENSION_WEIGHTS through the comparison harness
Greptile P2: the operator comparison harness
(scripts/compare-resilience-current-vs-proposed.mjs) claims its domain
scores "mirror the production scorer's coverage-weighted mean" and is
the artifact generator for Spearman / rank-delta acceptance decisions.
After PR 2 §3.4's weight rebalance, the production mirror diverged —
production now applies RESILIENCE_DIMENSION_WEIGHTS (liquidReserveAdequacy
= 0.5, sovereignFiscalBuffer = 0.5) inside coverageWeightedMean, but
the harness still used equal-weight aggregation.
Left unfixed, post-merge Spearman / rank-delta diagnostics would
compare live API scores (with the 0.5 recovery weights) against
harness predictions that assume equal-share dims — silently biasing
every acceptance decision until someone noticed a country's rank-
delta didn't track.
Fix
- Mirrored coverageWeightedMean now accepts dimensionWeights and
applies `coverage × weight` per dim, matching _shared.ts exactly.
- Mirrored buildDomainList accepts + forwards dimensionWeights.
- main() imports RESILIENCE_DIMENSION_WEIGHTS from the scorer module
and passes it through to buildDomainList at the single call site.
- Missing-entry default = 1.0 (same contract as production) — makes
the harness forward-compatible with any future weight refactor
(adds a new dim without an explicit entry, old production fallback
path still produces the correct number).
Verification
- Harness syntax-check clean (node -c).
- RESILIENCE_DIMENSION_WEIGHTS import resolves correctly from the
harness's import path.
- 509/509 resilience tests still pass (harness isn't in the test
suite; the invariant is that production ↔ harness use the same
math, and the production side is covered by tests/resilience-
recovery-weight-rebalance.test.mts).
* fix(resilience): PR 2 review — bump cache prefixes v10→v11 + document coverage-vs-weight asymmetry
Greptile P1 + P2 on PR #3328.
P1 — cache prefix not bumped after formula change
--------------------------------------------------
The per-dim weight rebalance changes the score formula, but the
`_formula` tag only distinguishes 'd6' vs 'pc' (pillar-combined vs
legacy 6-domain) — it does NOT detect intra-'d6' weight changes. Left
unfixed, scores cached before deploy would be served with the old
equal-weight math for up to the full 6h TTL, and the ranking key for
up to its 12h TTL. Matches the established v9→v10 pattern for every
prior formula-changing deploy.
Bumped in lockstep:
- RESILIENCE_SCORE_CACHE_PREFIX: v10 → v11
- RESILIENCE_RANKING_CACHE_KEY: v10 → v11
- RESILIENCE_HISTORY_KEY_PREFIX: v5 → v6
- scripts/seed-resilience-scores.mjs local mirrors
- api/health.js resilienceRanking literal
- 4 analysis/backtest scripts that read the cached keys directly
- Test fixtures in resilience-{ranking, handlers, scores-seed,
pillar-aggregation}.test.* that assert on literal key values
The v5→v6 history bump is the critical one: without it, pre-rebalance
history points would mix with post-rebalance points inside the 30-day
window, and change30d / trend math would diff values from different
formulas against each other, producing false-negative "falling" trends
for every country across the deploy window.
P2 — coverage-vs-weight asymmetry in computeLowConfidence / computeOverallCoverage
----------------------------------------------------------------------------------
Reviewer flagged that these two functions still average coverage
equally across all non-retired dims, even after the scoring aggregation
started applying RESILIENCE_DIMENSION_WEIGHTS. The asymmetry is
INTENTIONAL — these signals answer a different question from scoring:
scoring aggregation: "how much does each dim matter to the score?"
coverage signal: "how much real data do we have on this country?"
A dim at weight 0.5 still has the same data-availability footprint as
a weight=1.0 dim: its coverage value reflects whether we successfully
fetched the upstream source, not whether the scorer cares about it.
Applying scoring weights to the coverage signal would let a
half-weight dim hide half its sparsity from the overallCoverage pill,
misleading users reading coverage as a data-quality indicator.
Added explicit comments to both functions noting the asymmetry is
deliberate and pointing at the other site for matching rationale.
No code change — just documentation.
Tests: 6588/6588 data-tier tests pass (+511 resilience-specific
including the prefix-literal assertions). Typecheck clean on both
tsconfig configs. Biome clean on touched files.
* docs(resilience): bump methodology doc cache-prefix references to v11/v6
Greptile P2 on PR #3328: Redis keys table in the reproducibility
appendix still published `score:v10` / `ranking:v10` / `history:v5`,
and the rollback instructions told operators to flush those keys.
After the recovery-domain weight rebalance, live cache runs at
`score:v11` / `ranking:v11` / `history:v6`.
- Updated the Redis keys table (line 490-492) to match `_shared.ts`.
- Updated the rollback block to name the current keys.
- Left the historical "Activation sequence" narrative intact (it
accurately describes the pillar-combine PR's v9→v10 / v4→v5 bump)
but added a parenthetical pointing at the current v11/v6 values.
No code change — doc-only correction for operator accuracy.
* fix(docs): escape MDX-unsafe `<137` pattern to unblock Mintlify deploy
Line 643 had `(<137 countries)` — MDX parses `<137` as a JSX tag
starting with digit `1`, which is illegal and breaks the deploy with
"Unexpected character \`1\` (U+0031) before name". Surfaced after the
prior cache-prefix commit forced Mintlify to re-parse this file.
Replaced with "fewer than 137 countries" for unambiguous rendering.
Other `<` occurrences in this doc (lines 34, 642) are followed by
whitespace and don't trip MDX's tag parser.
|
||
|
Elie Habib
|
fe0e13b99e |
feat(agent-readiness): publish MCP Server Card at /.well-known/mcp/server-card.json (#3329)
SEP-1649 discovery manifest for the existing MCP server at api/mcp.ts (public endpoint https://worldmonitor.app/mcp, protocol 2025-03-26). Authentication block mirrors the live /.well-known/oauth-protected-resource doc byte-for-byte on resource, authorization_servers, and scopes. No $schema field — SEP-1649 schema URL not yet finalised. Closes #3311, part of epic #3306. |
||
|
Elie Habib
|
99b536bfb4 |
docs(energy): /corrections revision-log page (Week 4 launch requirement) (#3323)
* docs(energy): /corrections revision-log page (Week 4 launch requirement)
All five methodology pages reference /corrections (the auto-revision-log
URL promised in the Global Energy Flow parity plan §20) but the page
didn't exist — clicks 404'd. This lands the page.
Content:
- Explains the revision-log shape: `{date, assetOrEventId, fieldChanged,
previousValue, newValue, trigger, sourcesUsed, classifierVersion}`.
- Defines the trigger vocabulary (classifier / source / decay / override)
so readers know what kind of change they're seeing.
- States the v1-launch truth honestly: the log is empty at launch and
fills as the post-launch classifier pass (in proactive-intelligence.mjs)
runs on its normal schedule. No fake entries, no placeholder rows.
- Documents the correction-submission path (operators / regulators /
researchers with public source URLs) and the contract that
corrections write `override`-trigger entries citing the submitted
source — not anonymous overrides.
- Cross-links all five methodology pages.
- Explains WHY we publish this: evidence-first classification only
works if the audit trail is public; otherwise "the classifier said
so" has no more authority than any other opaque pipeline.
Also fixes a navigation gap: docs/docs.json was missing both
methodology/disruptions (landed in PR #3294 but never registered in
nav) and the new corrections page. Both now appear in the "Intelligence
& Analysis" group alongside the other methodology pages.
No code changes. MDX lint + docs.json JSON validation pass.
* docs(energy): reframe /corrections as planned-surface spec (P1 review fix)
Greptile P1: the prior /corrections page made live-product claims
("writes an append-only entry here", "expect the first entries within
days", "email corrections@worldmonitor.app") that the code doesn't
back. The revision-log writer ships with the post-launch classifier;
the correction-intake pipeline does not yet exist; and the related
detail handlers still return empty `revisions` arrays with code
comments explicitly marking the surface as future work.
Fix: rewrite the page as a planned-surface specification with a
prominent Status callout. Changes:
- Page title: "Revision Log" → "Revision Log (Planned)"
- Prominent <Note> callout at the top states v1 launch truth: log is
not yet live, RPC `revisions` arrays are empty by design,
corrections are handled manually today.
- "Current state (v1 launch)" section removed; replaced with two
explicit sections: "What IS live today" (evidence bundles,
methodology, versioned classifier output) and "What is NOT live
today" (log entries, automated correction intake, override-writer).
- "Within days" timeline language removed — no false operational SLA.
- Email submission path removed (no automated intake exists). Points
readers to GitHub issues for manual review today.
- Preserves the planned data shape, trigger vocabulary, policy
commitment, and "why we publish this" framing — those are spec, not
claims.
Also softens /corrections references in the four methodology pages
(pipelines, storage, shortages, disruptions) so none of them claim
the revision log is live. Each now says "planned revision-log shape
and submission policy" and points manual corrections at GitHub issues.
MDX lint 122/122 clean. docs.json JSON validation clean. No code
changes; pure reframing to match reality.
* docs(shortages): fix P1 overclaim + wrong RPC name (round-2 review)
Two findings on the same file:
P1 — `energy_asset_overrides` table documented as existing. It doesn't.
The PR's corrections.mdx explicitly lists the override-writer as NOT
live in v1; this section contradicted that. Rewrote as "Break-glass
overrides (planned)" with a clear Status callout matching the pattern
established in docs/corrections.mdx and the other methodology pages.
Points readers at GitHub issues for manual corrections today.
P2 — Wrong RPC name: `listActiveFuelShortages` doesn't exist. The
shipped RPC (in proto/worldmonitor/supply_chain/v1/
list_fuel_shortages.proto + server/worldmonitor/supply-chain/v1/
list-fuel-shortages.ts) is `ListFuelShortages`. Replaced the name +
reframed the sentence to describe what the actual RPC already exposes
(every FuelShortageEntry includes evidence.evidenceSources[]) rather
than projecting a future surface.
Also swept the other methodology pages for the same class of bug:
- grep for _overrides: only the one line above
- grep for listActive/ getActive RPC names: none found
- verified all RPC mentions in docs/methodology + docs/corrections.mdx
match names actually in proto (ListPipelines, ListStorageFacilities,
ListFuelShortages, ListEnergyDisruptions, GetPipelineDetail,
GetStorageFacilityDetail, GetFuelShortageDetail)
MDX lint clean. No code changes.
* docs(methodology): round-3 sibling sweep for revision-log overclaims
Reviewer (Greptile) caught a third round of the same overclaim pattern
I've been trying to stamp out: docs/methodology/shortages.mdx line 46
said "Stale shortages never persist silently. Every demotion writes to
the public revision log." — contradicting the same PR's /corrections
page which explicitly frames the revision log as not-yet-live. Fixed
that one AND did the mechanical sibling sweep the review pattern
clearly called for.
Changes:
- `docs/methodology/shortages.mdx:46` — rewrote the auto-decay footer
to future tense: "When the post-launch classifier ships, stale
shortages will never persist silently — every demotion will write
an entry to the planned public revision log." Points readers at
/corrections for the designed shape. Notes that today the demotion
thresholds ARE the contract; the structured audit trail is what
lands with the classifier.
- `docs/methodology/chokepoints.mdx:64` — sibling sweep caught the
same bug class ("Every badge transition writes to the public
revision log"). Reworded to future tense and pointed manual
corrections at GitHub issues, matching the pattern already applied
to pipelines / storage / shortages in prior commits on this PR.
Final audit of remaining revision-log mentions across all 5
methodology pages + corrections.mdx — every one uses hedged tense now
("planned", "will", "when live", "designed", "not yet live", "once
the classifier ships"). The one remaining present-tense "emit" in
shortages.mdx:77 is inside the "(planned)" break-glass section with
its own Status callout, so it's correctly scoped.
Following the plan-doc-as-docs-source-overclaim skill's step-4
(sibling sweep) explicitly this time — which also retroactively
validates the skill extraction: three review rounds was the cost of
not running the sweep on round 1.
MDX lint clean. No code changes.
* docs(corrections): drop hardcoded launch date (Greptile P2)
Greptile inline P2 at docs/corrections.mdx:60: the phrase
"v1 launch (2026-04-23)" pins a specific calendar date that will read
inaccurately to visitors months later once entries start appearing.
Dropped the parenthetical date. "Status — v1 launch:" keeps the
scoping clear without tying it to a specific day. When live entries
start appearing on this page (or when the page is rewritten to show
real rows), a "last updated" marker will replace the status callout
entirely — no migration churn needed.
MDX lint 122/122 clean.
|
||
|
Elie Habib
|
7f83e1e0c3 |
chore: remove dormant proactive-intelligence agent (superseded by digest) (#3325)
* chore: remove dormant proactive-intelligence agent (superseded by digest) PR #2889 merged a Phase 4 "Proactive Intelligence Agent" in 2026-04 with 588 lines of code and a PR body explicitly requiring a 6h Railway cron service. That service was never provisioned — no Dockerfile, no Railway entry, no health-registry key, all 7 test-plan checkboxes unchecked. In the meantime the daily Intelligence Brief shipped via scripts/seed-digest-notifications.mjs (PR #3321 and earlier), covering the same "personalized editorial brief across all channels" use-case at a different cadence (30m rather than 6h). The proactive agent's landscape-diff trigger was speculative; the digest is the shipped equivalent. This PR retires the dormant code and scrubs the aspirational "post-launch classifier" references that docs + comments have been quietly carrying: - Deleted scripts/proactive-intelligence.mjs (588 lines). - scripts/_energy-disruption-registry.mjs, scripts/seed-fuel-shortages.mjs, scripts/_fuel-shortage-registry.mjs, src/shared/shortage-evidence.ts: dropped "proactive-intelligence.mjs will extend this registry / classifier output" comments. Registries are curated-only; no classifier exists. - docs/methodology/disruptions.mdx: replaced "post-launch classifier" prose with the accurate "curated-only" description of how the event log is maintained. - docs/api-notifications.mdx: envelope version is shared across **two** producers now (notification-relay, seed-digest-notifications), not three. - scripts/notification-relay.cjs: one cross-producer comment updated. - proto/worldmonitor/supply_chain/v1/list_energy_disruptions.proto + list_fuel_shortages.proto: same aspirational wording scrubbed. - docs/api/SupplyChainService.openapi.{yaml,json} auto-regenerated via `make generate` — text-only description updates, no schema changes. Net: -626 lines, +36 lines. No runtime behavior change. 6573/6573 unit tests pass locally. * fix(proto): scrub stale ListFuelShortages RPC comment (PR #3325 review) Reviewer caught a stale "classifier-extended post-launch" comment on the ListFuelShortages RPC method in service.proto that this PR's initial pass missed — I fixed the message-definition comment in list_fuel_shortages.proto but not the RPC-method comment in service.proto, which propagates into the published OpenAPI operation description. - proto/worldmonitor/supply_chain/v1/service.proto: rewrite the ListFuelShortages RPC comment to match the curated-only framing used elsewhere in this PR. - docs/api/SupplyChainService.openapi.{yaml,json}: auto-regenerated via `make generate`. Text-only operation-description update; no schema / contract changes. No runtime impact. Other `classifier` references remaining in the OpenAPI are legitimate schema field names (classifierVersion, classifierConfidence) and an unrelated auto-revision-log trigger enum value, both of which describe real on-row fields that existed before this cleanup. |
||
|
Elie Habib
|
3918cc9ea8 |
feat(agent-readiness): declare Content-Signal in robots.txt (#3327)
Per contentsignals.org draft RFC: declare AI content-usage preferences at the origin level. Closes #3307, part of epic #3306. Values: ai-train=no — no consent for model-training corpora search=yes — allow search indexing for referral traffic ai-input=yes — allow live agent retrieval (Perplexity, ChatGPT browsing, Claude, etc.) |