mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
feat/unified-openapi-bundle
3560 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Sebastien Melki
|
52bd528247 |
Apply suggestions from code review
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> |
||
|
Sebastien Melki
|
203851e161 |
fix(proto): bump sebuf v0.11.0 → v0.11.1, realign tests with repeated-param wire format
- Bump SEBUF_VERSION to v0.11.1, pulling in the OpenAPI fix for repeated scalar query params (SebastienMelki/sebuf#161). `repeated string` fields now emit `type: array` + `items.type: string` + `style: form` + `explode: true` instead of `type: string`, so SDK generators consuming the unified bundle produce correct array clients. - Regenerate all 12 OpenAPI specs (unified bundle + Aviation, Economic, Infrastructure, Market, Trade per-service). TS client/server codegen is byte-identical to v0.11.0 — only the OpenAPI emitter was out of sync. - Update three tests that asserted the pre-v0.11 comma-joined wire format (`symbols=AAPL,MSFT`) to match the current repeated-param form (`symbols=AAPL&symbols=MSFT`) produced by `params.append(...)`: - tests/market-service-symbol-casing.test.mjs (2 cases: getAll) - tests/stock-analysis-history.test.mts - tests/stock-backtest.test.mts Locally: test:data 6619/6619 pass, typecheck clean, lint exit 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Sebastien Melki
|
f2ad6fe665 |
chore(codegen): regenerate TS client/server with sebuf v0.11.0
Mechanical output of the bumped protoc-gen-ts-client and protoc-gen-ts-server. Two behavioral improvements roll in from sebuf: - Proto enum fields now use the proper `*_UNSPECIFIED` sentinel in default-value checks instead of the empty string, so generated query-string serializers correctly omit enum params only when they actually equal the proto default. - `repeated string` query params now serialize via `forEach(v => params.append(...))` instead of being coerced through `String(req.field)`, matching the existing `parseStringArray()` contract on the server side. All files also drop the `// @ts-nocheck` header that earlier sebuf versions emitted — 0.11.0 output type-checks cleanly under our tsconfig. No hand edits. Reproduce with `make install-plugins && make generate`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Sebastien Melki
|
2906526d02 |
docs(proto): bump sebuf to v0.11.0 and document unified OpenAPI bundle
- Makefile: SEBUF_VERSION v0.7.0 → v0.11.0 (required for bundle support). - proto/buf.gen.yaml: point bundle_server at https://api.worldmonitor.app. - CONTRIBUTING.md: new "OpenAPI Output" section covering per-service specs vs the unified worldmonitor.openapi.yaml bundle, plus a note that all three sebuf plugins must be installed from the pinned version. - AGENTS.md: clarify that `make generate` also produces the unified spec and requires sebuf v0.11.0. - CHANGELOG.md: Unreleased entry announcing the bundle and version bump. Also regenerates the bundle with the updated server URL. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Sebastien Melki
|
03d1496ade |
feat(proto): generate unified worldmonitor.openapi.yaml bundle
Adds a third protoc-gen-openapiv3 invocation that merges every service into a single docs/api/worldmonitor.openapi.yaml spanning all 68 RPCs, using the new bundle support shipped in sebuf 0.11.0 (SebastienMelki/sebuf#158). Per-service YAML/JSON files are untouched and continue to back the Mintlify docs in docs/docs.json. The bundle runs with strategy: all and bundle_only=true so only the aggregate file is emitted, avoiding duplicate-output conflicts with the existing per-service generator. Requires protoc-gen-openapiv3 >= v0.11.0 locally: go install github.com/SebastienMelki/sebuf/cmd/protoc-gen-openapiv3@v0.11.0 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Elie Habib
|
df91e99142 |
feat(energy): expand 5 curated registries to 100% of plan target (#3337)
* feat(energy): expand gas pipeline registry 12 → 28 (phase 1a batch 1)
Data validation after v1 launch showed pipelines shipped at ~16% of the
plan target (12 gas + 12 oil vs. the plan's 75 + 75 critical
pipelines). This commit closes ~20% of the gas gap with 16 hand-curated
global additions, every entry carrying a full evidence bundle matching
the schema enforced by scripts/_pipeline-registry.mjs.
New additions by region:
North Sea / NW Europe (6):
europipe-1, europipe-2, franpipe, zeepipe, interconnector-uk-be, bbl
Mediterranean / North Africa (3):
transmed (Enrico Mattei), greenstream (LY→IT, reduced),
meg-maghreb-europe (DZ→ES via MA, offline since Oct 2021)
Middle East (1):
arab-gas-pipeline (EG→LB via JO/SY, offline under Caesar Act)
Former Soviet / Turkey (1):
blue-stream (RU→TR, carries EU sanctions ref)
Asia (3):
west-east-3 (CN internal, 7378 km), myanmar-china-gas (shwe),
igb (interconnector-greece-bulgaria, 2022)
Africa / LatAm (2):
wagp (west african gas pipeline, 4-country transit),
gasbol (bolivia-brazil, 3150 km)
Badge distribution on new entries:
flowing: 12, reduced: 2, offline: 2
First non-Russia-exposure offline entries (MEG — Morocco-Algeria
diplomatic closure, Arab Gas — Syria sanctions) — broadens the
geographic distribution of evidence-bundle-backed non-positive badges.
Registry tests: 17/17 pass (identity, geometry bounds, ISO2 country
codes, evidence contract, capacity-commodity pairing, validateRegistry
negative cases).
Next batches in this phase: oil pipelines +16, then second batches
each commodity to reach plan target (75+75). Tracked in
docs/internal/energy-atlas-registry-expansion.md.
* feat(energy): expand oil pipeline registry 12 → 28 (phase 1a batch 2)
Mirror of the gas batch — 16 hand-curated global additions with full
evidence bundles. Closes ~20% of the oil gap.
New additions by region:
North America (6):
enbridge-mainline (CA→US 3.15 mbd), enbridge-line-3-replacement (2021),
flanagan-south, seaway (Cushing→Gulf), marketlink (TC, Cushing→Gulf),
spearhead
Middle East (3):
sumed (EG crude bypass of Suez, 2.8 mbd),
east-west-saudi (Petroline, 5 mbd — largest single oil pipeline in
the registry by capacity),
ipsa-2 (IQ→SA, offline since Iraq invasion of Kuwait 1990, later
converted to gas on the western stretch)
Central Asia (1):
kazakhstan-china-crude (KZ→CN Alashankou, 2228 km)
Africa (1):
chad-cameroon-cotco (TD→CM Kribi, 1070 km)
South America (2):
ocp-ecuador (heavy crude, 450 kbd),
sote-ecuador (lighter grades, 360 kbd)
Europe (3):
tal-trieste-ingolstadt (IT→DE via AT, 770 kbd),
janaf-adria (HR→RS→HU, 280 kbd),
norpipe-oil (NO→DE North Sea crude, 900 kbd)
Badge distribution on new entries:
flowing: 15, offline: 1 (IPSA-2, regulator-sourced + nationalisation
statement backing the offline badge per the evidence-contract rules).
Registry totals after this batch:
gas: 12 → 28 (37% of plan target 75)
oil: 12 → 28 (37% of plan target 75)
total: 24 → 56
Registry tests: 17/17 registry + 23/23 evidence-derivation = 40/40 pass.
Typecheck-free (JSON only).
Next batches (per docs/internal/energy-atlas-registry-expansion.md):
gas batch 2: +22 → 50 (North Sea remainder, Caspian, Asia)
oil batch 2: +22 → 50 (North Sea remainder, Russia diversified,
Asia long-haul)
* feat(energy): expand gas pipeline registry 28 → 50 (phase 1a batch 3)
Second gas batch, 22 additions, bringing gas to ~67% of the 75-pipeline
plan target. Geographic distribution deliberately skewed this batch
toward under-represented regions (Middle East, Central Asia, South
America, Africa, Southeast Asia) since the first batch filled Europe
and North America.
New additions (22):
North Sea / UK (2):
vesterled (NO→GB, 13 bcm/yr),
cats (UK, 9.6 bcm/yr)
Iran family (3):
iran-turkey-gas (Tabriz→Ankara, 14 bcm/yr, OFAC sanctions ref),
iran-armenia-gas (2.3 bcm/yr),
iran-iraq-basra-gas (reduced state — waiver-dependent flows)
Central Asia (2):
central-asia-center (TM→RU via UZ/KZ, 44 bcm/yr nominal, reduced),
turkmenistan-iran-korpeje (expired contract, reduced)
Caucasus / Turkey (2):
south-caucasus-scp (BTE predecessor to TANAP, 22 bcm/yr),
sakarya-black-sea-tr (2023 Turkish offshore)
China (2):
west-east-1 (4200 km, 17 bcm/yr),
west-east-2 (8700 km, 30 bcm/yr)
South America (2):
bolivia-argentina-yacuiba (reduced),
antonio-ricaurte (CO→VE, offline since 2015, PDVSA sanctions)
Saudi / Middle East (2):
saudi-master-gas-system (SA internal, 95 bcm/yr — largest capacity
in the registry), egypt-jordan-aqaba (AGP south leg, flowing)
Israel-Egypt (1):
israel-egypt-arish-ashkelon (reverse-flow since 2020, IL→EG export)
Planned / FID-stage (5):
galsi-planned (DZ→IT, consortium paused),
eastmed-planned (IL→CY→GR, US political support withdrawn Jan 2022),
trans-saharan-planned (NG→DZ via NE, insurgency + financing unresolved),
morocco-nigeria-offshore-planned (NG→MA 11-country offshore route),
power-of-siberia-2-planned (RU→CN via MN, no binding CNPC contract),
kirkuk-dohuk-turkey-gas-planned (IQ→TR, Baghdad-Erbil dispute)
Badge distribution on new batch:
flowing: 10 (incl. Sakarya 2023 commissioned)
reduced: 3 (CAC, BO-AR, IR-IQ)
offline: 1 (Antonio Ricaurte, CO-VE, with operator statement + sanction)
unknown: 6 (all planned/FID-stage, classifierConfidence 0.6-0.75)
All non-flowing badges have evidence (sanction refs, operator
statements, or press sourcing) per the evidence-contract validator.
Registry totals after this batch:
gas: 28 → 50 (67% of plan target; gas ≥60 gate threshold not yet
hit but approaching)
oil: 28 (unchanged — batch 4 will target oil to 50)
total: 56 → 78
Registry tests: 17/17 pass. Includes 8 new fully-hedged "unknown" /
planned-status entries; validator accepts them.
Next: oil batch 2 (+22 → 50), then gas batch 3 (+10 → 60), oil batch 3
(+10 → 60). After that the gate criteria on pipelines hit and we can
focus on storage / shortages / disruptions.
* feat(energy): expand oil pipeline registry 28 → 50 (phase 1a batch 4)
Second oil batch, 22 additions, bringing oil to 67% of plan target and
matching gas (50 each, 100 total pipelines).
New additions (22):
Russia Baltic export (2):
bps-1 (Primorsk, 1.3 mbd — largest single line in oil registry),
bps-2 (Ust-Luga, 0.75 mbd). Both carry G7+EU price-cap sanctions ref.
North America diversified (3):
enbridge-line-5 (CA→CA via US Straits of Mackinac, ongoing litigation),
keystone-xl-cancelled (CA→US, permit revoked 2021, Biden; TC
terminated Jun 2021; listed for historical + geopolitical
completeness, physicalState=unknown by deriver rule),
trans-panama-pipeline (PA, 0.9 mbd cross-isthmus)
Europe remaining (3):
rotterdam-rhine-rrp (NL→DE, 275 km),
spse (FR→DE Lyon→Karlsruhe, 769 km),
forties-pipeline (UK North Sea, 0.6 mbd),
brent-pipeline (NO→GB Sullom Voe, reduced — Brent field in
decommissioning)
Middle East (2):
khafji-neutral-zone (SA/KW, reduced post-2015 neutral-zone dispute),
ab-1-bahrain (SA→BH, 2018, 0.35 mbd)
Africa (4):
greater-nile-petroleum (SS→SD Port Sudan, 1610 km),
djeno-congo (CG terminal system),
nigeria-forcados-export (reduced — recurring force-majeure),
nigeria-bonny-export (Trans Niger Pipeline, reduced)
Latin America (2):
pemex-nuevo-cactus (MX, 0.44 mbd),
trans-andino (AR→CL, offline since 2006 export restrictions)
Ukraine (1):
odesa-brody (offline, under EU 2022/879 Russian-crude embargo
framework)
Asia (1):
myanmar-china-crude (MM→CN Kunming, 771 km parallel to
myanmar-china-gas)
Caspian (1):
baku-novorossiysk-northern (AZ→RU historical route, reduced, carries
Russian crude price-cap ref)
Historical / planned (2):
kirkuk-haifa-idle (IQ→IL via JO, closed 1948 — listed for
completeness; periodically floated as reopening proposal),
uganda-tanzania-eacop-planned (UG→TZ, under construction, Western
bank-financing pulled but TotalEnergies continues)
Badge distribution on new batch:
flowing: 10
reduced: 6 (Brent decommissioning, Khafji dispute, Greater Nile,
Forcados, Bonny, Baku-Novorossiysk)
offline: 2 (Odesa-Brody, Trans-Andino, Kirkuk-Haifa)
unknown: 2 (Keystone XL cancelled, EACOP under construction)
Wait, Kirkuk-Haifa is offline not among 2. Corrected count:
flowing: 10, reduced: 6, offline: 3 (Odesa-Brody, Trans-Andino,
Kirkuk-Haifa), unknown: 2, plus 1 flowing Myanmar-China-crude = 22.
All non-flowing badges carry supporting evidence (operator statements,
sanction refs, or press citations) per the evidence-contract validator.
Registry totals after this batch:
gas: 50 (67% of plan target)
oil: 28 → 50 (67% of plan target)
total: 78 → 100
Registry tests: 17/17 + 23/23 evidence-derivation = 40/40 pass.
Next batches to hit the 60-each gate criteria from
docs/internal/energy-atlas-registry-expansion.md:
gas batch 3: +10 → 60 (EastMed details, Galsi alternative routes,
minor EU-interconnectors, Nigeria LNG feeder gas lines)
oil batch 3: +10 → 60 (Pluto crude, Chinese Huabei system, Latam
infill: Brazil Campos, Peru Northern Trunk)
After 60/60: hit gate, move to storage expansion.
* feat(energy): gas registry 50 → 75 — plan target hit
Batch 3 adds 25 more gas pipelines, bringing gas to 100% of the
75-pipeline plan target.
New additions by region (25):
- Norwegian transport spine: statpipe, sleipner-karsto, troll-a,
oseberg-gas-transport, asgard-transport (covers the major offshore
export collectors — the rest of the Gassco system)
- Australia: dampier-bunbury (1594 km), moomba-sydney (1299 km)
- Africa: mozambique-rompco (MZ→ZA), escravos-lagos-gas (NG),
tanzania-mtwara-dar, ghana-gas (atuabo)
- Southeast Asia: thailand-malaysia-cakerawala, indonesia-singapore
west-natuna + grissik-sakra
- German hubs for Nord Stream continuation: nel-pipeline, opal-pipeline,
eugal-pipeline (built but dormant after NS2 halt/destruction),
megal-pipeline, gascade-jagal, zeelink-germany
- Russia/Ukraine/EU transit: progress-urengoy-uzhhorod (halted 1 Jan
2025 when Ukraine did not renew transit agreement), trans-austria-gas
- Iran: kish-iran-gas, iran-pakistan-gas-planned (Pakistani segment
stalled since 2014)
- China/HK: china-hong-kong-gas
Badge distribution on new batch: 15 flowing, 4 reduced (NEL, OPAL,
TAG, Escravos-Lagos), 2 offline (EUGAL dormant post-NS2,
Urengoy-Uzhhorod transit halt), 4 sanction-exposed (NS-continuation
pipelines + TAG + Urengoy), 1 unknown (Iran-Pakistan stalled
completion).
Plan progress: gas 50 → 75 (100% of plan target).
Registry tests: 17/17 pass.
* feat(energy): oil registry 50 → 75 — plan target hit
Batch 4 adds 25 more oil pipelines, bringing oil to 100% of the
75-pipeline plan target. Combined with gas at 75, total registry is
150 pipelines — full plan coverage for Phase 1a.
New additions by region (25):
- Latin America: colombia-cano-limon-covenas (ELN-sabotaged, reduced),
colombia-ocensa (main trunk), peru-norperuano (reduced from jungle
spills + protests), ecuador-lago-agrio-orellana,
venezuela-anzoategui-puerto-la-cruz (under OFAC PDVSA sanctions),
mexico-salina-cruz-minatitlan, mexico-madero-cadereyta,
mexico-gulf-coast-pipeline (Tuxpan-Mexico City)
- Africa: angola-cabinda-offshore, south-sudan-kenya-lamu-planned
(LAPSSET)
- Middle East: iran-abadan-isfahan, iran-neka-tehran (reduced,
Caspian swap arrangements), saudi-abqaiq-yanbu-products,
iraq-strategic-pipeline (1000 km north-south), iraq-bai-hassan,
oman-muscat-export (Fahud-Mina al-Fahal), uae-habshan-ruwais
- Asia-Pacific: india-salaya-mathura (1770 km, largest Indian crude
trunk), india-vadinar-kandla, india-mundra-bhatinda,
china-qinhuangdao-tianjin-huabei, china-yangzi-hefei-hangzhou
- Russia East: russia-sakhalin-2-crude, russia-komsomolsk-perevoznaya,
russia-omsk-pavlodar (cross-border to KZ)
Badge distribution on this batch: 18 flowing, 6 reduced, 1 unknown
(LAPSSET planned). Sanctions-exposure diversified: Iran framework (3),
Venezuela/PDVSA (1), Russian price-cap (3). All non-flowing badges
carry supporting evidence per validator rules.
Phase 1a final state (pipelines):
gas: 12 → 75 (100% of plan target, 6 batches)
oil: 12 → 75 (100% of plan target, 6 batches)
total: 24 → 150
Geographic distribution now global:
- Russia-exposure: ~22 of 150 entries (~15%, down from 50% at v1)
- US-only: ~8 (~5%, down from 33% storage-side skew)
- Six continents represented in active infrastructure
- Historical + planned pipelines (Kirkuk-Haifa, Keystone XL cancelled,
EACOP u/c, EastMed planned, GALSI planned, TSGP planned,
Nigeria-Morocco offshore, Power of Siberia 2, Iran-Pakistan Peace,
LAPSSET) listed with honest 'unknown' physicalState per validator
Registry tests: 17/17 pass.
Phase 1a complete. Next phase (per
docs/internal/energy-atlas-registry-expansion.md):
- Phase 2: storage 21 → ~200 (+179) via curation + GIIGNL/GIE/EIA
- Phase 3: shortages 14 → 28 countries
- Phase 4: disruptions 12 → 50 events
* feat(energy): shortages 15 → 29 entries across 28 countries — plan target hit
+14 country additions matching the 28-country plan target. The
validator's 'confirmed severity requires authoritative source' rule
caught two of my drafts (Myanmar + Sudan) where I had labeled them
confirmed with press-only evidence because regulator/operator sources
under a junta + active civil war are not independently verifiable.
Downgraded both to 'watch' with an inline note explaining the
evidence-quality choice — exactly the validator's intended behavior
(better to under-claim than over-claim severity when the authoritative
channel is broken).
New shortages (14):
- BD diesel: BPC LC delays, regulator-confirmed
- ZA diesel: loadshedding demand spike
- AO diesel: Luanda/Benguela depot delays
- MZ diesel: FX-allocation import constraints
- ZM diesel: mining-sector demand + TAZAMA product tightness
- MW diesel: FX shortfalls + MERA rationing
- GH petrol: Tema port congestion
- MM diesel: post-coup chronic (watch, press-only evidence)
- MN diesel: winter logistics
- CO diesel: trucker strike cycles
- UA diesel: war-driven chronic (confirmed — Ministry of Energy source)
- SY diesel: Caesar Act chronic (confirmed — Syrian Ministry statement)
- SD diesel: civil-war disruption (watch, press-only)
- DE heating_oil: Rhine low-water logistics (watch)
Badge distribution on new batch: 3 confirmed (BD, UA, SY — all with
regulator/operator evidence), 11 watch.
Plan progress:
shortages: 15 → 29 entries (28 unique countries = 100% of plan)
gas: 75 (100%)
oil: 75 (100%)
storage: 21 (unchanged, next batch)
disruptions: 12 (unchanged, next batch)
Registry tests: 19/19 pass.
* feat(energy): disruption event log 12 → 52 events — plan target hit
+40 historical and ongoing events covering the asset registry,
bringing disruptions to 104% of the 50-event plan target. Every event
ties to an assetId now in pipelines/storage registries (following the
75-gas + 75-oil + 21-storage registry expansion in the preceding
commits).
New additions by eventType:
Sabotage / war (7):
- abqaiq-khurais-drone-strike-2019 (Saudi, 5.7 mbd removed 11 days)
- russia-refinery-drone-strikes-2024 (Ukrainian drone strike series)
- houthi-red-sea-attacks-2024 (indirect SuMed demand impact)
- russia-ukraine-oil-depot-strikes-2022 (series)
- nigeria-trans-niger-attacks-2024 (Bonny system)
- bai-hassan-attack-2022 (Iraq Bai Hassan)
- sudan-pipeline-attacks-2023 (Greater Nile disruption)
Sanctions (7):
- russia-price-cap-implementation-2022 (G7+EU $60/bbl cap)
- eu-oil-embargo-2022 (6th package)
- pdvsa-designation-2019 (Venezuela)
- btc-kurdistan-shutdown-2023 (ICC ruling, ongoing)
- ipsa-nationalization-2001 (SA nationalised after Iraq invasion of Kuwait)
- arctic-lng-2-foreign-partner-withdrawal-2024
- yamal-lng-arctic-sanctions-ongoing (Novatek)
- ogm-moldova-transit-2022
Mechanical (4):
- druzhba-contamination-2019 (chlorides, 3-month shut)
- keystone-milepost-14-leak-2022 (Kansas, 22-day shut, 14k bbl spill)
- forties-crack-2017 (Red Moss hairline)
- ocensa-ocp-ecuador-suspensions-2022 (Amazon landslide)
Weather (2):
- hurricane-ida-lng-2021 (Gulf coast LNG shutdown)
- rotterdam-hub-low-water-2022 (Rhine 2.5-month disruption)
Commercial (9):
- cpc-blockage-threat-2022 (Russian court 30-day halt threat)
- gme-closure-2021 (Algeria-Morocco MEG)
- ukraine-transit-end-2025 (Progress pipeline halted 1 Jan 2025)
- eugal-dormant-since-2022 (NS2 knock-on)
- keystone-xl-permit-revoked-2021 (Biden day-1)
- antonio-ricaurte-halt-2015 (CO→VE gas export halt)
- langeled-brent-decommissioning-2020
- eacop-financing-2023 (Western bank withdrawal)
- dolphin-qatar-uae-commercial-2024 (contract renegotiation)
- trans-austria-gas-reduction-2022 (Gazprom volume drops)
- cushing-stocks-tank-bottoms-2022
- spr-drawdown-2022-2023 (largest ever 180 mbbl release)
- zhoushan-storage-expansion-2023
- fujairah-stockbuild-2024
- futtsu-lng-demand-decline-2024
- bolivia-diesel-import-cut-2023 (GASBOL)
- myanmar-china-gas-reduced-2023
- yamal-europe-poland-halt-follow-on-2024
Maintenance (1): gladstone-lng-maintenance-2023
Ongoing events (endAt=null): 31 of 52 (~60%). Reflects the structural
reality that many 2022-era sanctions + war events remain live in 2026.
Plan progress:
gas: 75 (100%)
oil: 75 (100%)
storage: 21 (unchanged, next batch)
shortages: 29 (100% — 28 countries)
disruptions: 12 → 52 events (104% of plan)
Registry tests: 16/16 pass.
* feat(energy): storage registry 21 → 66 (storage batch 1)
+45 facilities, 33% of plan. Focus: European UGS + second LNG wave.
European UGS additions (35 — mostly filling the gap against GIE AGSI+
coverage which has ~140 EU sites; we now register the majority of
operationally significant ones with non-trivial working capacity):
Germany (9): bierwang, etzel-salt-cavern, jemgum, krummhoern,
peckensen, reckrod, uelsen, xanten, epe-salt-cavern
Netherlands (3): alkmaar, norg (largest NL, 59.2 TWh), zuidwending
Austria (3): 7fields-schonkirchen (24.6 TWh), baumgarten-uhs,
puchkirchen
France (7): chemery (38.5 TWh), cerville-velaine, etrez, manosque,
lussagnet (35 TWh), izaute
Italy (4): minerbio (45 TWh, largest IT), ripalta, sergnano,
brugherio
UK (2): rough (reduced, post-2017 partial reopening 2022), hornsea
Central/Eastern Europe (8): damborice (CZ), lobodice (CZ),
lab-slovakia (36 TWh), hajduszoboszlo (HU), mogilno (PL),
lille-torup (DK), incukalns (LV), gaviota (ES)
Russia (1): kasimovskoe (124 TWh — Gazprom UGS flagship; EU sanctions
ref carried as evidence)
LNG terminals (9 additions to round out global coverage):
- US: freeport-lng, cameron-lng, cove-point-lng, elba-island-lng
- Middle East: qalhat-lng (Oman), adgas-das-island (UAE)
- Russia: sakhalin-2-lng (sanctions-exposed)
- Indonesia: tangguh-lng, bontang-lng (reduced — declining upstream)
Badge distribution on this batch: 43 operational, 2 reduced (Rough,
Bontang). Most entries from GIE AGSI+ fill-disclosed data; Russian
site + LNG terminals fill-not-disclosed (operator choice + sanctions).
Plan progress:
gas pipelines: 75 (100%)
oil pipelines: 75 (100%)
fuel shortages: 29 / 28 countries (100%)
disruptions: 52 (104%)
storage: 21 → 66 (33% of ~200 target)
Registry tests: 21/21 pass.
Next storage batches remaining:
batch 2 (+45): more European UGS tail + Asian national reserves
(CN SPR, IN SPR, JP national reserves, KR KNOC)
batch 3 (+45): LNG import terminals + additional US tank farms +
European tank farms (Rotterdam detail, ARA sub-sites)
batch 4 (+45): remainder to ~200
* feat(energy): storage registry 66 → 110 (storage batch 2)
+44 facilities. Focus: Asian national reserves + global LNG coverage
+ Singapore/ARA tank-farm detail.
Asian national reserves (11):
- IN ISPRL: vizag (9.8 Mb), mangalore (11 Mb), padur (17.4 Mb)
- CN: zhanjiang (45 Mb), huangdao (20 Mb) — fill opaque, press-only
- JP JOGMEC: shibushi (31.2 Mb), kiire (22 Mb), mutsu-ogawara (28 Mb)
- KR KNOC: yeosu (42 Mb), ulsan (33 Mb), geoje (47 Mb)
LNG export additions (11):
- Australia: pluto-lng, prelude-flng (reduced), darwin-lng (reduced
upstream)
- Southeast Asia: mlng-bintulu (29.3 Mtpa — largest in registry),
brunei-lng, donggi-senoro-lng
- Africa: angola-lng (reduced), equatorial-guinea-lng, hilli-episeyo-flng
- Pacific: png-lng
- Caribbean: trinidad-atlantic-lng (reduced)
- Mexico: costa-azul-lng (2025 reverse-to-export commissioning)
LNG import (12):
- UK: south-hook-lng (21 Mtpa), dragon-lng
- EU: zeebrugge-lng, dunkerque-lng, fos-cavaou-lng,
montoir-de-bretagne-lng, gate-terminal (Rotterdam),
revithoussa-lng
- Turkey: aliaga-ege-gaz-lng
- Chile: mejillones-lng, quintero-bay-lng
Tank farms (10):
- Africa: saldanha-bay (ZA 45 Mb)
- Norway: mongstad-crude
- ARA: antwerp-petroleum-hub (BE 55 Mb), amsterdam-petroleum-hub
- Asia hubs: singapore-jurong (120 Mb — largest in registry),
singapore-pulau-ayer-chawan, thailand-sriracha, korea-gwangyang-crude
- Russia Baltic: ust-luga-crude-terminal, primorsk-crude-terminal
(both carry Russian price-cap sanction refs)
Badge distribution on this batch: 39 operational, 5 reduced (Prelude,
Darwin, Angola, Bontang — no wait Bontang already in. Correct: Prelude,
Darwin, Angola, Trinidad).
Plan progress:
gas pipelines: 75 (100%)
oil pipelines: 75 (100%)
fuel shortages: 29 / 28 countries (100%)
disruptions: 52 (104%)
storage: 66 → 110 (55% of ~200 target)
Registry tests: 21/21 pass.
Next batches remaining: ~90 more storage to hit ~200
batch 3 (+45): Middle East tank farms, Chinese coastal commercial
storage, EU UGS tail, African LNG import
batch 4 (+45): remainder to 200
* feat(energy): storage registry 110 → 155 (storage batch 3)
Adds 45 facilities toward 200 plan target:
- 7 Middle East export terminals (Kharg, Sidi Kerir, Mina al-Ahmadi,
Mesaieed, Jebel Dhanna, Mina al-Fahal, Bandar Imam Khomeini)
- 10 EU UGS tail (Reitbrook, Empelde, Kirchheilingen, Stockstadt,
Nüttermoor, Grijpskerk, Târgu Mureș, Třanovice, Uhřice, Háje)
- 4 Chinese coastal crude (Yangshan, Qingdao, Rizhao, Maoming)
- 6 EU LNG import tail (La Spezia, Adriatic, OLT Livorno, Klaipeda,
Mugardos, Cartagena)
- 5 Indian LNG import (Hazira, Kochi reduced, Ennore, Mundra, Dabhol)
- 6 Japan/Korea LNG import (Chita, Negishi, Sodegaura, Himeji,
Pyeongtaek, Incheon)
- 5 NA tank farms (Lake Charles, Corpus Christi, Patoka, Edmonton,
Hardisty)
- 2 Asia-Pacific (Kaohsiung, Nghi Son)
Registry validator: 21/21 tests pass.
* feat(energy): storage registry 155 → 200 (storage batch 4 — plan target hit)
Final batch brings storage to the 200-facility plan target with broad
geographic + facility-type coverage.
New entries (45):
- 6 LNG export: NLNG Bonny (NG, reduced), Arzew (DZ), Skikda (DZ),
Perú LNG, Calcasieu Pass (US), North West Shelf Karratha (AU)
- 7 LNG import: Świnoujście (PL), Krk FSRU (HR), Wilhelmshaven FSRU (DE),
Brunsbüttel (DE), Map Ta Phut (TH), Port Qasim (PK), Batangas (PH)
- 6 UGS: Bilche-Volytsko-Uherske (UA, 154 TWh — largest Europe), Banatski
Dvor (RS), Okoli (HR), Yela (ES), Loenhout (BE), Kushchevskoe (RU)
- 26 crude tank farms: José Terminal (VE, sanctioned), Santos (BR),
TEBAR São Sebastião (BR), Dos Bocas (MX), Bonny (NG, reduced), Es
Sider (LY, reduced), Ras Lanuf (LY, reduced), Ceyhan (TR), Puerto
Rosales (AR), Novorossiysk Sheskharis (RU, sanctioned), Kozmino (RU,
sanctioned), Tema (GH, reduced), Mombasa (KE), Abidjan SIR (CI),
Juaymah (SA), Ras Tanura (SA), Yanbu (SA), Kirkuk (IQ, reduced),
Basra Gulf (IQ), Djibouti Horizon (DJ), Yokkaichi (JP), Mailiao
(TW), Ventspils (LV, reduced), Gdańsk Naftoport (PL), Constanța
(RO), Wood River IL (US).
Geographic balance improved: Africa coverage (NG, DZ, LY, GH, KE, CI,
DJ) from 5 to 12 countries; first Iraq + Saudi entries; Balkans +
Ukraine + Romania now covered. Type mix: UGS 56, SPR 15, LNG export 33,
LNG import 38, crude tank farm 58.
Non-operational entries all carry authoritative evidence (press
operator statements + sanctionRefs for Russia/Venezuela).
Registry validator: 21/21 tests pass. Total: 200 facilities across 55
countries. Plan target hit.
* fix(energy): address Greptile review findings on registries
P1 — abqaiq-khurais-drone-strike-2019 (energy-disruptions.json):
capacityOfflineMbd was 5.7 (plant-level Saudi production loss headline)
against assetId east-west-saudi (5.0 mbd pipeline). Capped offline
figure at the linked pipeline's 5.0 mbd ceiling; moved the 5.7 mbd
historical headline into shortDescription with an explanatory note.
Preserves capacity-offline ≤ asset-capacity invariant for downstream
consumers.
P1 — russia-price-cap-implementation-2022 (energy-disruptions.json):
was linked to assetId espo (land pipeline to China — explicitly out of
scope for G7/EU price cap). Relinked to primorsk-crude-terminal
(largest Baltic seaborne crude export terminal, directly affected);
assetType pipeline → storage. Updated shortDescription to clarify
tanker-shipment scope + out-of-scope note for ESPO.
P2 — 13 reduced-state pipelines missing press citation text
(pipelines-gas.json × 8 + pipelines-oil.json × 5):
Added operatorStatement sentences naming the press/regulator sources
backing each reduction claim (Reuters, NNPC/Chevron releases, NIGC,
Pemex annual reports, S&P Platts, IEA Gas Market Report, BBC, etc.).
Clears the evidence-source-type gap flagged by Greptile for entries
that declared physicalStateSource: "press" with a null statement.
All 6583 data tests + 94 registry tests still pass.
* style(energy): restore compact registry formatting (preserve Greptile-fix evidence)
Prior commit
|
||
|
Elie Habib
|
9f208848b6 |
fix(deps): add yaml to scripts/package.json (Railway installs from THIS) (#3336)
PR #3333 added `yaml` to the root package.json, but the Railway seed-bundle-resilience-recovery service builds with rootDirectory pointing at scripts/ and NIXPACKS auto-detects scripts/package.json as the install manifest. Root package.json is never visited during the container build, so yaml stayed missing and the seeder crashed again at 07:39:26 UTC with the identical ERR_MODULE_NOT_FOUND. Adding yaml ^2.8.3 (matching the root promotion) to scripts/package.json so NIXPACKS' `npm install --prefix scripts` lands it in /app/node_modules/yaml. scripts/shared/swf-manifest-loader.mjs can then resolve the bare specifier. Keeping yaml in the root package.json too — it's harmless noise for local dev + validation bundle (which also imports it via tsx), and defensive for any future consumer that runs against the root deps. Future question worth a separate PR: do we want Railway services pointing at `scripts/` as rootDir, or should we move to a proper per-service Dockerfile that makes the dep source explicit? The current state is easy to miss because the Railway dashboard config is invisible from the repo — second seeder to trip this exact hazard. |
||
|
Elie Habib
|
8ea4c8f163 |
feat(digest-dedup): replayable per-story input log (opt-in, no behaviour change) (#3330)
* feat(digest-dedup): replayable per-story input log (opt-in, no behaviour change)
Ship the measurement layer before picking any recall-lift strategy.
Why: the current dedup path embeds titles only, so brief-wire headlines
that share a real event but drop the geographic anchor (e.g. "Alleged
Coup: defendant arrives in court" vs "Trial opens after Nigeria charges
six over 2025 coup plot") can slip past the 0.60 cosine threshold. To
tune recall without regressing precision we need a replayable per-tick
dataset — one record per story with the exact fields any downstream
candidate (title+slug, LLM-canonicalise, text-embedding-3-large, cross-
encoder re-rank, etc.) would need to score.
This PR ships ONLY the log. Zero behaviour change:
- Opt-in via DIGEST_DEDUP_REPLAY_LOG=1 (default OFF).
- Writer is best-effort: all errors swallowed + warned, never affects
digest delivery. No throw path.
- Records include hash, originalIndex, isRep, clusterId, raw +
normalised title, link, severity/score/mentions/phase/sources,
embeddingCacheKey, hasEmbedding sidecar flag, and the tick's config
snapshot (mode, clustering, cosineThreshold, topicThreshold, veto).
- clusterId derives from rep.mergedHashes (already set by
materializeCluster) so the orchestrator is untouched.
- Storage: Upstash list keyed by {variant}:{lang}:{sensitivity}:{date}
with 30-day EXPIRE. Date suffix caps per-key growth; retention
covers the labelling cadence + cross-candidate comparison window.
- Env flag is '1'-only (fail-closed on typos, same pattern as
DIGEST_DEDUP_MODE).
Activation path (post-merge): flip DIGEST_DEDUP_REPLAY_LOG=1 on the
seed-digest-notifications Railway service. Watch one cron tick for the
RPUSH + EXPIRE pair (or a single warn line if creds/upstream flake),
then leave running for at least one week to accumulate calibration data.
Tests: 21 unit tests covering flag parsing, key shape + sanitisation,
record field correctness (isRep, clusterId, embeddingCacheKey,
hasEmbedding, tickConfig), pipeline null/throw handling, malformed
input. Existing 77 dedup tests unchanged and still green.
* fix(digest-dedup): capture topicGroupingEnabled in replay tickConfig
Review catch (PR #3330): the tickConfig snapshot omitted
topicGroupingEnabled even though readOrchestratorConfig returns it and
the digest's post-dedup topic ordering gates on it. A tick run with
DIGEST_DEDUP_TOPIC_GROUPING=0 serialised identically to a default
tick, making those runs non-replayable for the calibration work this
log is meant to enable.
Add topicGroupingEnabled to the recorded tickConfig. One-line schema
fix + regression test asserting topic-grouping-off ticks serialise
distinctly from default.
22/22 tests pass.
* fix(digest-dedup): await replay-log write to survive explicit process.exit
Review catch (PR #3330): the fire-and-forget `void writeReplayLog(...)`
call could be dropped on the explicit-exit paths — the brief-compose
failure gate at line 1539 and main().catch at line 1545 both call
process.exit(1). Unlike natural exit, process.exit does not drain
in-flight promises, so the last N ticks' replay records could be
silently lost on runs where measurement fidelity matters most.
Fix: await the writeReplayLog call. Safe because:
- writeReplayLog returns synchronously when the flag is off
(replayLogEnabled check is the first thing it does)
- It has a top-level try/catch that always returns a result object
- The Upstash pipeline call has a 10s timeout ceiling
- buildDigest already awaits many Upstash calls (dedup, compose,
render) so one more is not a hot-path concern
Comment block added above the call explains why the await is
deliberate — so a future refactor doesn't revert it to void thinking
it's a leftover.
No test change: existing writeReplayLog unit tests already cover the
disabled / empty / success / error paths. The fix is a single-keyword
change in a caller that was already guaranteed-safe by the callee's
contract.
* refactor(digest-dedup): address Greptile P2 review comments on replay log
Three non-blocking polish items from the automated review, bundled
because they all touch the same new module and none change behaviour.
1. tsMs captured BEFORE deduplicateStories (seed-digest-notifications.mjs).
Previously sampled after dedup returned, so briefTickId reflected
dedup-completion time rather than tick-start. For downstream readers
the natural reading of "briefTickId" is when the tick began
processing; moved the Date.now() call to match that expectation.
Drift is maybe 100ms-2s on cold-cache embed calls — small, but
moving it is free.
2. buildReplayLogKey emptiness check now strips ':' and '-' in addition
to '_'. A pathological ruleId of ':::' previously passed through
verbatim, producing keys like `digest:replay-log:v1::::2026-04-23`
that confuse redis-cli's namespace tooling (SCAN / KEYS / tab
completion). The new guard falls back to "unknown" on any input
that's all separators. Added a regression test covering the
':::' / '---' / '___' / mixed cases.
3. tickConfig is now a per-record shallow copy instead of a shared
reference. Storage is unaffected (writeReplayLog serialises each
record via JSON.stringify independently) but an in-memory consumer
that mutated one record's tickConfig for experimentation would have
silently affected all other records in the same batch. Added a
regression test asserting mutation doesn't leak across records.
Tests: 24/24 pass (22 prior + 2 new regression). Typecheck + lint clean.
|
||
|
Elie Habib
|
dd95a4e06d |
fix(idb-cleanup): swallow TransactionInactiveError in idempotent IDB cursor loops (#3335)
Sentry WORLDMONITOR-NX: iOS Safari kills in-flight IDB transactions when the tab backgrounds. Our idle detector fires `[App] User idle - pausing animations to save resources` right before the browser suspends — any `cursor.delete()` / `cursor.continue()` mid-iteration then throws TransactionInactiveError synchronously inside onsuccess. Both affected sites are idempotent cleanup (`cleanOldSnapshots`, `deleteFromIndexedDbByPrefix`); swallowing the throw lets the next run resume from where we left off. `main.ts` beforeSend keeps TransactionInactiveError surfaced for first-party stacks (storage.ts, persistent-cache.ts, vector-db.ts), so this is the correct layer to handle the background-kill case. |
||
|
Elie Habib
|
8b12ecdf43 |
fix(aviation): seeder writes delays-bootstrap aggregate (close EMPTY-on-quiet-traffic alarm) (#3334)
* fix(aviation): seeder writes delays-bootstrap aggregate (close EMPTY-on-quiet-traffic alarm) api/health.js BOOTSTRAP_KEYS.flightDelays points at aviation:delays-bootstrap:v1, but no seeder ever produced it — the key was only written as a 1800s side-effect inside list-airport-delays.ts. Quiet user-traffic windows >30 min let the bootstrap expire, tripping EMPTY (CRIT) even with healthy upstream FAA + intl + NOTAM seeds. PR #3073 (Apr 13) doubled the cron cadence to 30 min, putting the bootstrap TTL right at the failure edge. Make seed-aviation.mjs the canonical writer: - New writeDelaysBootstrap() reads FAA + intl + NOTAM from Redis, applies the same NOTAM merge + Normal-operations filler the RPC builds, writes aviation:delays-bootstrap:v1 with TTL=7200 (~4 missed cron ticks of cushion). - Called pre-runSeed (last-good intl, covers intl-fail tick) AND inside afterPublishIntl (this-tick intl, happy-path overwrite). - Bump RPC's incidental write TTL 1800 → 7200 so a user-triggered RPC doesn't shorten the seeder's expiry and re-create the failure mode. NOTAM merge logic + filler shape are now mirrored in two files (seeder + RPC's _shared.ts). Both carry comments pointing at the other to surface drift risk. Verified: typecheck (both tsconfigs) clean; node --test tests/aviation-*.test.mjs green; full test:data 6590/6590 green. * fix(aviation): seeder writes restrictedIcaos + bootstrap unwraps intl envelope PR #3334 review (P1 + P2): P1 — bootstrap silently dropped NOTAM restrictions seedNotamClosures() only tracked NOTAM_CLOSURE_QCODES; the live RPC's classifier in server/worldmonitor/aviation/v1/_shared.ts also derives restrictions via NOTAM_RESTRICTION_QCODES (RA, RO) + restriction code45s + restriction-text regex. Seeded NOTAM payload only had `closedIcaos`, so restrictedIcaos was always empty in Redis — both the new bootstrap aggregate AND the RPC's seed-read path silently dropped every NOTAM restriction. Mirror the full classifier from _shared.ts:438-452; side-car write now includes restrictedIcaos and seed-meta count reflects closures + restrictions. P2 — pre-runSeed bootstrap built with no intl alerts on intl-fail tick runSeed wraps the canonical INTL_KEY in {_seed, data} when declareRecords is enabled. writeDelaysBootstrap()'s upstashGet only JSON.parsed — no envelope unwrap — so intlPayload.alerts was undefined on the pre-runSeed bootstrap-build path, and an intl-fail tick would publish a bootstrap with all intl alerts dropped instead of preserving the last-good snapshot. Add upstashGetUnwrapped() (delegates to unwrapEnvelope from _seed-envelope-source.mjs); use it for all three reads (FAA/NOTAM bare values pass through unchanged via unwrapEnvelope's permissive path). Verified: typecheck (both tsconfigs) clean; aviation + edge-functions tests green; full test:data 6590/6590 green. * fix(aviation): bootstrap iterates union of seeder + RPC airport registries PR #3334 review (P2 ×2): P2 — AIRPORTS vs MONITORED_AIRPORTS registry drift Today the two diverge by ~45 iata codes (29 RPC-only, 16 seeder-only). Pre-fix the bootstrap iterated the seeder's local AIRPORTS list for Normal-operations filler and NOTAM airport lookup, so 29 monitored airports never appeared in the bootstrap aggregate even though the live RPC included them. Fix: parse src/config/airports.ts as text at startup (regex over the static const), memoise the parse, build a by-iata Map union (seeder wins on conflict for canonical meta), and iterate that for both NOTAM lookup and filler. First-run divergence summary logged to surface future drift in cron logs without blocking writes. Degrades to seeder AIRPORTS only with a warning if parse fails. P2 — afterPublishIntl receives raw pre-transform data runSeed forwards the RAW fetchIntl() result to afterPublish, NOT the publishTransform()'d shape. Today publishTransform is a pass-through wrapper so data.alerts is correct, but coupling is subtle — added an inline CONTRACT comment so a future publishTransform mutation doesn't silently drift bootstrap from INTL_KEY. Verified: typecheck (both tsconfigs) clean; aviation + edge-functions tests green; full test:data 6590/6590 green; standalone parse harness recovers all 111 MONITORED_AIRPORTS rows. |
||
|
Elie Habib
|
a7bd1248ac |
chore(lint): exclude docs/brainstorms and docs/ideation from lint:md (#3332)
Both dirs are in .gitignore (alongside docs/plans which was already excluded from lint:md). Brings the lint:md exclusion list in sync with .gitignore so pre-push hooks don't flag local-only working docs that the repo isn't tracking anyway. Observed failure mode before: writing a brainstorm doc locally under docs/brainstorms/ for context during a session, then pushing any feature branch → markdownlint runs against the untracked doc, blocks the push until the local doc is deleted or its formatting is scrubbed to markdownlint's satisfaction. The doc never enters git, so the lint work is pure friction. No functional change to shipping code. |
||
|
Elie Habib
|
1958b34f55 |
fix(digest-dedup): CLUSTERING typo fallback fails closed to complete-link (#3331)
DIGEST_DEDUP_CLUSTERING previously fell to 'single' on unrecognised values, which silently defeated the documented kill switch. A typo like `DIGEST_DEDUP_CLUSTERING=complet` during an over-merge incident would stick with the aggressive single-link merger instead of rolling back to the conservative complete-link algorithm. Mirror the DIGEST_DEDUP_MODE typo pattern (PR #3247): - Unrecognised value → fall to 'complete' (SAFE / conservative). - Surface the raw value via new `invalidClusteringRaw` config field. - Emit a warn line on the dedup orchestrator's entry path so operators see the typo alongside the kill-switch-took-effect message. Valid values 'single' (default), 'complete', unset, empty, and any case variation all behave unchanged. Only true typos change behaviour — and the new behaviour is the kill-switch-safe one. Tests: updated the existing case that codified the old behaviour plus added coverage for (a) multiple typo variants falling to complete with invalidClusteringRaw set, (b) case-insensitive valid values not triggering the typo path, and (c) the orchestrator emitting the warn line even on the jaccard-kill-switch codepath (since CLUSTERING intent applies to both modes). 81/81 dedup tests pass. |
||
|
Elie Habib
|
58589144a5 |
fix(deps): promote yaml from transitive peer to top-level dependency (#3333)
Railway seed-bundle-resilience-recovery crashed at 06:36:18 UTC on first tick post-#3328 with: Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'yaml' imported from /app/shared/swf-manifest-loader.mjs `scripts/shared/swf-manifest-loader.mjs` (landed in #3319) imports `parse` from `yaml` to read `docs/methodology/swf-classification-manifest.yaml`. The package is present locally (as a peer of other deps), but Railway installs production deps only — transitive peers don't land in /app/node_modules, so the seeder exits 1 before any work. Adding `yaml ^2.8.3` to `dependencies` so `npm ci` in the container installs it. Version matches the already-on-disk resolution in package-lock. No consumer changes needed. Unblocks the first Sovereign-Wealth Railway run on the resilience-recovery bundle. |
||
|
Elie Habib
|
d3d406448a |
feat(resilience): PR 2 §3.4 recovery-domain weight rebalance (#3328)
* feat(resilience): PR 2 §3.4 recovery-domain weight rebalance
Dials the two PR 2 §3.4 recovery dims (liquidReserveAdequacy,
sovereignFiscalBuffer) to ~10% share each of the recovery-domain
score via a new per-dimension weight channel in the coverage-weighted
mean. Matches the plan's direction that the sovereign-wealth signal
complement — rather than dominate — the classical liquid-reserves
and fiscal-space signals.
Implementation
- RESILIENCE_DIMENSION_WEIGHTS: new Record<ResilienceDimensionId, number>
alongside RESILIENCE_DOMAIN_WEIGHTS. Every dim has an explicit entry
(default 1.0) so rebalance decisions stay auditable; the two new
recovery dims carry 0.5 each.
Share math at full coverage (6 active recovery dims):
weight sum = 4 × 1.0 + 2 × 0.5 = 5.0
each new-dim share = 0.5 / 5.0 = 0.10 ✓
each core-dim share = 1.0 / 5.0 = 0.20
Retired dims (reserveAdequacy, fuelStockDays) keep weight 1.0 in
the map; their coverage=0 neutralizes them at the coverage channel
regardless. Explicit entries guard against a future scorer bug
accidentally returning coverage>0 for a retired dim and falling
through the `?? 1.0` default — every retirement decision is now
tied to a single explicit source of truth.
- coverageWeightedMean (_shared.ts): refactored to apply
`coverage × dimWeight` per dim instead of `coverage` alone. Backward-
compatible when all weights default to 1.0 (reduces to the original
mean). All three aggregation callers — buildDomainList, baseline-
Score, stressScore — pick up the weighting transparently.
Test coverage
1. New `tests/resilience-recovery-weight-rebalance.test.mts`:
pins the per-dim weight values, asserts the share math
(0.10 new / 0.20 core), verifies completeness of the weight map,
and documents why retired dims stay in the map at 1.0.
2. New `tests/resilience-recovery-ordering.test.mts`: fixture-based
Spearman-proxy sensitivity check. Asserts NO > US > YE ordering
preserved on both the overall score and the recovery-domain
subscore after the rebalance. (Live post-merge Spearman rerun
against the PR 0 snapshot is tracked as a follow-up commit.)
3. resilience-scorers.test.mts fixture anchors updated in lockstep:
baselineScore: 60.35 → 62.17 (low-scoring liquidReserveAdequacy
+ partial-coverage SWF now contribute ~half the weight)
overallScore: 63.60 → 64.39 (recovery subscore lifts by ~3 pts
from the rebalance, overall by ~0.79)
recovery flat mean: 48.75 (unchanged — flat mean doesn't apply
weights by design; documents the coverage-weighted diff)
Local coverageWeightedMean helper in the test mirrors the
production implementation (weights applied per dim).
Methodology doc
- New "Per-dimension weights in the recovery domain" subsection with
the weight table and a sentence explaining the cap. Cross-references
the source of truth (RESILIENCE_DIMENSION_WEIGHTS).
Deliberate non-goals
- Live post-merge Spearman ≥0.85 check against the PR 0 baseline
snapshot. Fixture ordering is preserved (new ordering test); the
live-data check runs after Railway cron refreshes the rankings on
the new weights and commits docs/snapshots/resilience-ranking-live-
post-pr2-<date>.json. Tracked as the final piece of PR 2 §3.4
alongside the health.js / bootstrap graduation (waiting on the
7-day Railway cron bake-in window).
Tests: 6588/6588 data-tier tests pass. Typecheck clean on both
tsconfig configs. Biome clean on touched files. NO > US > YE
fixture ordering preserved.
* fix(resilience): PR 2 review — thread RESILIENCE_DIMENSION_WEIGHTS through the comparison harness
Greptile P2: the operator comparison harness
(scripts/compare-resilience-current-vs-proposed.mjs) claims its domain
scores "mirror the production scorer's coverage-weighted mean" and is
the artifact generator for Spearman / rank-delta acceptance decisions.
After PR 2 §3.4's weight rebalance, the production mirror diverged —
production now applies RESILIENCE_DIMENSION_WEIGHTS (liquidReserveAdequacy
= 0.5, sovereignFiscalBuffer = 0.5) inside coverageWeightedMean, but
the harness still used equal-weight aggregation.
Left unfixed, post-merge Spearman / rank-delta diagnostics would
compare live API scores (with the 0.5 recovery weights) against
harness predictions that assume equal-share dims — silently biasing
every acceptance decision until someone noticed a country's rank-
delta didn't track.
Fix
- Mirrored coverageWeightedMean now accepts dimensionWeights and
applies `coverage × weight` per dim, matching _shared.ts exactly.
- Mirrored buildDomainList accepts + forwards dimensionWeights.
- main() imports RESILIENCE_DIMENSION_WEIGHTS from the scorer module
and passes it through to buildDomainList at the single call site.
- Missing-entry default = 1.0 (same contract as production) — makes
the harness forward-compatible with any future weight refactor
(adds a new dim without an explicit entry, old production fallback
path still produces the correct number).
Verification
- Harness syntax-check clean (node -c).
- RESILIENCE_DIMENSION_WEIGHTS import resolves correctly from the
harness's import path.
- 509/509 resilience tests still pass (harness isn't in the test
suite; the invariant is that production ↔ harness use the same
math, and the production side is covered by tests/resilience-
recovery-weight-rebalance.test.mts).
* fix(resilience): PR 2 review — bump cache prefixes v10→v11 + document coverage-vs-weight asymmetry
Greptile P1 + P2 on PR #3328.
P1 — cache prefix not bumped after formula change
--------------------------------------------------
The per-dim weight rebalance changes the score formula, but the
`_formula` tag only distinguishes 'd6' vs 'pc' (pillar-combined vs
legacy 6-domain) — it does NOT detect intra-'d6' weight changes. Left
unfixed, scores cached before deploy would be served with the old
equal-weight math for up to the full 6h TTL, and the ranking key for
up to its 12h TTL. Matches the established v9→v10 pattern for every
prior formula-changing deploy.
Bumped in lockstep:
- RESILIENCE_SCORE_CACHE_PREFIX: v10 → v11
- RESILIENCE_RANKING_CACHE_KEY: v10 → v11
- RESILIENCE_HISTORY_KEY_PREFIX: v5 → v6
- scripts/seed-resilience-scores.mjs local mirrors
- api/health.js resilienceRanking literal
- 4 analysis/backtest scripts that read the cached keys directly
- Test fixtures in resilience-{ranking, handlers, scores-seed,
pillar-aggregation}.test.* that assert on literal key values
The v5→v6 history bump is the critical one: without it, pre-rebalance
history points would mix with post-rebalance points inside the 30-day
window, and change30d / trend math would diff values from different
formulas against each other, producing false-negative "falling" trends
for every country across the deploy window.
P2 — coverage-vs-weight asymmetry in computeLowConfidence / computeOverallCoverage
----------------------------------------------------------------------------------
Reviewer flagged that these two functions still average coverage
equally across all non-retired dims, even after the scoring aggregation
started applying RESILIENCE_DIMENSION_WEIGHTS. The asymmetry is
INTENTIONAL — these signals answer a different question from scoring:
scoring aggregation: "how much does each dim matter to the score?"
coverage signal: "how much real data do we have on this country?"
A dim at weight 0.5 still has the same data-availability footprint as
a weight=1.0 dim: its coverage value reflects whether we successfully
fetched the upstream source, not whether the scorer cares about it.
Applying scoring weights to the coverage signal would let a
half-weight dim hide half its sparsity from the overallCoverage pill,
misleading users reading coverage as a data-quality indicator.
Added explicit comments to both functions noting the asymmetry is
deliberate and pointing at the other site for matching rationale.
No code change — just documentation.
Tests: 6588/6588 data-tier tests pass (+511 resilience-specific
including the prefix-literal assertions). Typecheck clean on both
tsconfig configs. Biome clean on touched files.
* docs(resilience): bump methodology doc cache-prefix references to v11/v6
Greptile P2 on PR #3328: Redis keys table in the reproducibility
appendix still published `score:v10` / `ranking:v10` / `history:v5`,
and the rollback instructions told operators to flush those keys.
After the recovery-domain weight rebalance, live cache runs at
`score:v11` / `ranking:v11` / `history:v6`.
- Updated the Redis keys table (line 490-492) to match `_shared.ts`.
- Updated the rollback block to name the current keys.
- Left the historical "Activation sequence" narrative intact (it
accurately describes the pillar-combine PR's v9→v10 / v4→v5 bump)
but added a parenthetical pointing at the current v11/v6 values.
No code change — doc-only correction for operator accuracy.
* fix(docs): escape MDX-unsafe `<137` pattern to unblock Mintlify deploy
Line 643 had `(<137 countries)` — MDX parses `<137` as a JSX tag
starting with digit `1`, which is illegal and breaks the deploy with
"Unexpected character \`1\` (U+0031) before name". Surfaced after the
prior cache-prefix commit forced Mintlify to re-parse this file.
Replaced with "fewer than 137 countries" for unambiguous rendering.
Other `<` occurrences in this doc (lines 34, 642) are followed by
whitespace and don't trip MDX's tag parser.
|
||
|
Elie Habib
|
fe0e13b99e |
feat(agent-readiness): publish MCP Server Card at /.well-known/mcp/server-card.json (#3329)
SEP-1649 discovery manifest for the existing MCP server at api/mcp.ts (public endpoint https://worldmonitor.app/mcp, protocol 2025-03-26). Authentication block mirrors the live /.well-known/oauth-protected-resource doc byte-for-byte on resource, authorization_servers, and scopes. No $schema field — SEP-1649 schema URL not yet finalised. Closes #3311, part of epic #3306. |
||
|
Elie Habib
|
99b536bfb4 |
docs(energy): /corrections revision-log page (Week 4 launch requirement) (#3323)
* docs(energy): /corrections revision-log page (Week 4 launch requirement)
All five methodology pages reference /corrections (the auto-revision-log
URL promised in the Global Energy Flow parity plan §20) but the page
didn't exist — clicks 404'd. This lands the page.
Content:
- Explains the revision-log shape: `{date, assetOrEventId, fieldChanged,
previousValue, newValue, trigger, sourcesUsed, classifierVersion}`.
- Defines the trigger vocabulary (classifier / source / decay / override)
so readers know what kind of change they're seeing.
- States the v1-launch truth honestly: the log is empty at launch and
fills as the post-launch classifier pass (in proactive-intelligence.mjs)
runs on its normal schedule. No fake entries, no placeholder rows.
- Documents the correction-submission path (operators / regulators /
researchers with public source URLs) and the contract that
corrections write `override`-trigger entries citing the submitted
source — not anonymous overrides.
- Cross-links all five methodology pages.
- Explains WHY we publish this: evidence-first classification only
works if the audit trail is public; otherwise "the classifier said
so" has no more authority than any other opaque pipeline.
Also fixes a navigation gap: docs/docs.json was missing both
methodology/disruptions (landed in PR #3294 but never registered in
nav) and the new corrections page. Both now appear in the "Intelligence
& Analysis" group alongside the other methodology pages.
No code changes. MDX lint + docs.json JSON validation pass.
* docs(energy): reframe /corrections as planned-surface spec (P1 review fix)
Greptile P1: the prior /corrections page made live-product claims
("writes an append-only entry here", "expect the first entries within
days", "email corrections@worldmonitor.app") that the code doesn't
back. The revision-log writer ships with the post-launch classifier;
the correction-intake pipeline does not yet exist; and the related
detail handlers still return empty `revisions` arrays with code
comments explicitly marking the surface as future work.
Fix: rewrite the page as a planned-surface specification with a
prominent Status callout. Changes:
- Page title: "Revision Log" → "Revision Log (Planned)"
- Prominent <Note> callout at the top states v1 launch truth: log is
not yet live, RPC `revisions` arrays are empty by design,
corrections are handled manually today.
- "Current state (v1 launch)" section removed; replaced with two
explicit sections: "What IS live today" (evidence bundles,
methodology, versioned classifier output) and "What is NOT live
today" (log entries, automated correction intake, override-writer).
- "Within days" timeline language removed — no false operational SLA.
- Email submission path removed (no automated intake exists). Points
readers to GitHub issues for manual review today.
- Preserves the planned data shape, trigger vocabulary, policy
commitment, and "why we publish this" framing — those are spec, not
claims.
Also softens /corrections references in the four methodology pages
(pipelines, storage, shortages, disruptions) so none of them claim
the revision log is live. Each now says "planned revision-log shape
and submission policy" and points manual corrections at GitHub issues.
MDX lint 122/122 clean. docs.json JSON validation clean. No code
changes; pure reframing to match reality.
* docs(shortages): fix P1 overclaim + wrong RPC name (round-2 review)
Two findings on the same file:
P1 — `energy_asset_overrides` table documented as existing. It doesn't.
The PR's corrections.mdx explicitly lists the override-writer as NOT
live in v1; this section contradicted that. Rewrote as "Break-glass
overrides (planned)" with a clear Status callout matching the pattern
established in docs/corrections.mdx and the other methodology pages.
Points readers at GitHub issues for manual corrections today.
P2 — Wrong RPC name: `listActiveFuelShortages` doesn't exist. The
shipped RPC (in proto/worldmonitor/supply_chain/v1/
list_fuel_shortages.proto + server/worldmonitor/supply-chain/v1/
list-fuel-shortages.ts) is `ListFuelShortages`. Replaced the name +
reframed the sentence to describe what the actual RPC already exposes
(every FuelShortageEntry includes evidence.evidenceSources[]) rather
than projecting a future surface.
Also swept the other methodology pages for the same class of bug:
- grep for _overrides: only the one line above
- grep for listActive/ getActive RPC names: none found
- verified all RPC mentions in docs/methodology + docs/corrections.mdx
match names actually in proto (ListPipelines, ListStorageFacilities,
ListFuelShortages, ListEnergyDisruptions, GetPipelineDetail,
GetStorageFacilityDetail, GetFuelShortageDetail)
MDX lint clean. No code changes.
* docs(methodology): round-3 sibling sweep for revision-log overclaims
Reviewer (Greptile) caught a third round of the same overclaim pattern
I've been trying to stamp out: docs/methodology/shortages.mdx line 46
said "Stale shortages never persist silently. Every demotion writes to
the public revision log." — contradicting the same PR's /corrections
page which explicitly frames the revision log as not-yet-live. Fixed
that one AND did the mechanical sibling sweep the review pattern
clearly called for.
Changes:
- `docs/methodology/shortages.mdx:46` — rewrote the auto-decay footer
to future tense: "When the post-launch classifier ships, stale
shortages will never persist silently — every demotion will write
an entry to the planned public revision log." Points readers at
/corrections for the designed shape. Notes that today the demotion
thresholds ARE the contract; the structured audit trail is what
lands with the classifier.
- `docs/methodology/chokepoints.mdx:64` — sibling sweep caught the
same bug class ("Every badge transition writes to the public
revision log"). Reworded to future tense and pointed manual
corrections at GitHub issues, matching the pattern already applied
to pipelines / storage / shortages in prior commits on this PR.
Final audit of remaining revision-log mentions across all 5
methodology pages + corrections.mdx — every one uses hedged tense now
("planned", "will", "when live", "designed", "not yet live", "once
the classifier ships"). The one remaining present-tense "emit" in
shortages.mdx:77 is inside the "(planned)" break-glass section with
its own Status callout, so it's correctly scoped.
Following the plan-doc-as-docs-source-overclaim skill's step-4
(sibling sweep) explicitly this time — which also retroactively
validates the skill extraction: three review rounds was the cost of
not running the sweep on round 1.
MDX lint clean. No code changes.
* docs(corrections): drop hardcoded launch date (Greptile P2)
Greptile inline P2 at docs/corrections.mdx:60: the phrase
"v1 launch (2026-04-23)" pins a specific calendar date that will read
inaccurately to visitors months later once entries start appearing.
Dropped the parenthetical date. "Status — v1 launch:" keeps the
scoping clear without tying it to a specific day. When live entries
start appearing on this page (or when the page is rewritten to show
real rows), a "last updated" marker will replace the status callout
entirely — no migration churn needed.
MDX lint 122/122 clean.
|
||
|
Elie Habib
|
7f83e1e0c3 |
chore: remove dormant proactive-intelligence agent (superseded by digest) (#3325)
* chore: remove dormant proactive-intelligence agent (superseded by digest) PR #2889 merged a Phase 4 "Proactive Intelligence Agent" in 2026-04 with 588 lines of code and a PR body explicitly requiring a 6h Railway cron service. That service was never provisioned — no Dockerfile, no Railway entry, no health-registry key, all 7 test-plan checkboxes unchecked. In the meantime the daily Intelligence Brief shipped via scripts/seed-digest-notifications.mjs (PR #3321 and earlier), covering the same "personalized editorial brief across all channels" use-case at a different cadence (30m rather than 6h). The proactive agent's landscape-diff trigger was speculative; the digest is the shipped equivalent. This PR retires the dormant code and scrubs the aspirational "post-launch classifier" references that docs + comments have been quietly carrying: - Deleted scripts/proactive-intelligence.mjs (588 lines). - scripts/_energy-disruption-registry.mjs, scripts/seed-fuel-shortages.mjs, scripts/_fuel-shortage-registry.mjs, src/shared/shortage-evidence.ts: dropped "proactive-intelligence.mjs will extend this registry / classifier output" comments. Registries are curated-only; no classifier exists. - docs/methodology/disruptions.mdx: replaced "post-launch classifier" prose with the accurate "curated-only" description of how the event log is maintained. - docs/api-notifications.mdx: envelope version is shared across **two** producers now (notification-relay, seed-digest-notifications), not three. - scripts/notification-relay.cjs: one cross-producer comment updated. - proto/worldmonitor/supply_chain/v1/list_energy_disruptions.proto + list_fuel_shortages.proto: same aspirational wording scrubbed. - docs/api/SupplyChainService.openapi.{yaml,json} auto-regenerated via `make generate` — text-only description updates, no schema changes. Net: -626 lines, +36 lines. No runtime behavior change. 6573/6573 unit tests pass locally. * fix(proto): scrub stale ListFuelShortages RPC comment (PR #3325 review) Reviewer caught a stale "classifier-extended post-launch" comment on the ListFuelShortages RPC method in service.proto that this PR's initial pass missed — I fixed the message-definition comment in list_fuel_shortages.proto but not the RPC-method comment in service.proto, which propagates into the published OpenAPI operation description. - proto/worldmonitor/supply_chain/v1/service.proto: rewrite the ListFuelShortages RPC comment to match the curated-only framing used elsewhere in this PR. - docs/api/SupplyChainService.openapi.{yaml,json}: auto-regenerated via `make generate`. Text-only operation-description update; no schema / contract changes. No runtime impact. Other `classifier` references remaining in the OpenAPI are legitimate schema field names (classifierVersion, classifierConfidence) and an unrelated auto-revision-log trigger enum value, both of which describe real on-row fields that existed before this cleanup. |
||
|
Elie Habib
|
3918cc9ea8 |
feat(agent-readiness): declare Content-Signal in robots.txt (#3327)
Per contentsignals.org draft RFC: declare AI content-usage preferences at the origin level. Closes #3307, part of epic #3306. Values: ai-train=no — no consent for model-training corpora search=yes — allow search indexing for referral traffic ai-input=yes — allow live agent retrieval (Perplexity, ChatGPT browsing, Claude, etc.) |
||
|
Elie Habib
|
2789b07095 |
chore(vercel): RFC 9116 canonical /security.txt → /.well-known/security.txt 301 (#3326)
Per agent-readiness epic #3306 follow-up: legacy /security.txt path was returning the SPA shell (text/html) because of the catch-all rewrite. Add a permanent redirect so both apex and www emit 301 → /.well-known/security.txt, matching RFC 9116 §3. No-op for the canonical /.well-known/security.txt path, which already serves correctly. |
||
|
Elie Habib
|
c48ceea463 |
feat(resilience): PR 2 dimension wiring — split reserveAdequacy + add sovereignFiscalBuffer (#3324)
* feat(resilience): PR 2 dimension wiring — split reserveAdequacy + add sovereignFiscalBuffer Plan §3.4 follow-up to #3305 + #3319. Lands the scorer + dimension registration so the SWF seed from the Railway cron feeds a real score once the bake-in window closes. No weight rebalance yet (separate commit with Spearman sensitivity check), no health.js graduation yet (7-day ON_DEMAND window per feedback_health_required_key_needs_ railway_cron_first.md), no bootstrap wiring yet (follow-up PR). Shape of the change Retirement: - reserveAdequacy joins fuelStockDays in RESILIENCE_RETIRED_DIMENSIONS. The legacy scorer now mirrors scoreFuelStockDays: returns coverage=0 / imputationClass=null so the dimension is filtered out of the confidence / coverage averages via the registry filter in computeLowConfidence, computeOverallCoverage, and the widget's formatResilienceConfidence. Kept in RESILIENCE_DIMENSION_ORDER for structural continuity (tests, cached payload shape, registry membership). Indicator registry tier demoted to 'experimental'. Two new active dimensions: - liquidReserveAdequacy (replaces the liquid-reserves half of the retired reserveAdequacy). Same source (WB FI.RES.TOTL.MO, total reserves in months of imports) but re-anchored 1..12 months instead of 1..18. Twelve months ≈ IMF "full reserve adequacy" benchmark for a diversified emerging-market importer — the tighter ceiling prevents wealthy commodity-exporters from claiming outsized credit for on-paper reserve stocks that are not the relevant shock-absorption buffer. - sovereignFiscalBuffer. Reads resilience:recovery:sovereign-wealth:v1 (populated by scripts/seed-sovereign-wealth.mjs, landed in #3305 + wired into Railway cron in #3319). Computes the saturating transform: effectiveMonths = Σ [ aum/annualImports × 12 × access × liquidity × transparency ] score = 100 × (1 − exp(−effectiveMonths / 12)) Exponential saturation prevents Norway-type outliers (effective months in the 100s) from dominating the recovery pillar. Three code paths in scoreSovereignFiscalBuffer: 1. Seed key absent entirely → IMPUTE.recoverySovereignFiscalBuffer (score 50 / coverage 0.3 / unmonitored). Covers the Railway-cron bake-in window before the first successful tick. 2. Seed present, country NOT in manifest → score=0 with FULL coverage. Substantive absence, NOT imputation — per plan §3.4 "What happens to no-SWF countries." 0 × weight = 0 in the numerator, so the country correctly scores lower than SWF-holding peers on this dim. 3. Seed present, country in payload → saturating score, coverage derated by the partial-seed completeness signal (so a Mubadala or Temasek scrape drift on a multi-fund country shows up as lower confidence rather than a silently-understated total). Indicator registry: - Demoted recoveryReserveMonths (tied to retired reserveAdequacy) to tier='experimental'. - Added recoveryLiquidReserveMonths: WB FI.RES.TOTL.MO, anchors 1..12, tier='core', coverage=188. - Added recoverySovereignWealthEffectiveMonths: the new SWF signal, tier='experimental' for now because the manifest only has 8 funds (below the 180-core / 137-§3.6-gate threshold). Graduating to 'core' requires expanding the manifest past ~137 entries — a later PR. Tests updated - resilience-release-gate: 19→21 dim count; RETIRED_DIMENSIONS allow- list now includes reserveAdequacy alongside fuelStockDays. - resilience-dimension-scorers: scoreReserveAdequacy monotonicity + "high reserves score well" tests migrated to scoreLiquidReserve- Adequacy (same source, new 1..12 anchor). New retirement-shape test for scoreReserveAdequacy mirroring the PR 3 fuelStockDays retirement test. Four new scorer tests pin the three code paths of scoreSovereignFiscalBuffer (absent seed / no-SWF country / SWF country / partial-completeness derate). - resilience-scorers fixture: baseline 60.12→60.35, recovery-domain flat mean 47.33→48.75, overall 63.27→63.6. Each number commented with the driver (split adds liquidReserveAdequacy 18@1.0 + sovereign FiscalBuffer 50@0.3 at IMPUTE; retired reserveAdequacy drops out). - resilience-dimension-monotonicity: target scoreLiquidReserveAdequacy instead of scoreReserveAdequacy. - resilience-handlers: response-shape dim count 19→21. - resilience-indicator-registry: coverage 19→21 dimensions. - resilience-dimension-freshness: allowlisted the new sovereign-wealth seed-meta key in KNOWN_SEEDS_NOT_IN_HEALTH for the ON_DEMAND window. - resilience-methodology-lint HEADING_TO_DIMENSION: added the two new heading mappings. Methodology doc gets H4 sections for Liquid Reserve Adequacy and Sovereign Fiscal Buffer; Reserve Adequacy section is annotated as retired. - resilience-retired-dimensions-parity: client-side RESILIENCE_RETIRED_DIMENSION_IDS gets reserveAdequacy. Parser upgraded to strip inline `// …` comments from the array body so a future reviewer can drop a rationale next to an entry without breaking parity. - resilience-confidence-averaging: fixture updated to include both retired dims (reserveAdequacy + fuelStockDays) — confirms the registry filter correctly excludes BOTH from the visible coverage reading. Extraction harness (scripts/compare-resilience-current-vs-proposed.mjs): - recoveryLiquidReserveMonths: reads the same reserve-adequacy seed field as recoveryReserveMonths. - recoverySovereignWealthEffectiveMonths: reads the new SWF seed key on field totalEffectiveMonths. Absent-payload → 0 for correlation math (matches the substantive-no-SWF scorer branch). Out of scope for this commit (follow-ups) - Recovery-domain weight rebalance + Spearman sensitivity rerun against the PR 0 baseline. - health.js graduation (SEED_META entry + ON_DEMAND_KEYS removal) once Railway cron has ~7 days of clean runs. - api/bootstrap.js wiring once an RPC consumer needs the SWF data. - Manifest expansion past 137 countries so sovereignFiscalBuffer can graduate from tier='experimental' to tier='core'. Tests: 6573/6573 data-tier tests pass. Typecheck clean on both tsconfig configs. Biome clean on all touched files. * fix(resilience): PR 2 review — add widget labels for new dimensions P2 review finding on PR #3324. DIMENSION_LABELS in src/components/ resilience-widget-utils.ts covered only the old 19 dimension IDs, so the two new active dims (liquidReserveAdequacy, sovereignFiscalBuffer) would render with their raw internal IDs in the confidence grid for every country once the scorer started emitting them. The widget test at getResilienceDimensionLabel also asserted only the 19-label set, so the gap would have shipped silently. Fix: add user-facing short labels for both new dims. "Reserves" is already claimed by the retired reserveAdequacy, so the replacement disambiguates with "Liquid Reserves"; sovereignFiscalBuffer → "Sovereign Wealth" per the methodology doc H4 heading. Also added a regression guard — new test asserts EVERY id in RESILIENCE_DIMENSION_ORDER resolves to a non-id label. Any future dimension that ships without a matching DIMENSION_LABELS entry now fails CI loudly instead of leaking the ID into the UI. Tests: 502/502 resilience tests pass (+1 new coverage check). Typecheck clean on both configs. * fix(resilience): PR 2 review — remove dead IMPUTE.recoveryReserveAdequacy entry Greptile P2: the retired scoreReserveAdequacy stub no longer reads from IMPUTE (it hardcodes coverage=0 / imputationClass=null per the retirement pattern), making IMPUTE.recoveryReserveAdequacy dead code. Removed the entry + added a breadcrumb comment pointing at the replacement IMPUTE.recoveryLiquidReserveAdequacy. The second P2 (bootstrap.js not wired) is a deliberate non-goal — the reviewer explicitly flags "for visibility" since it's tracked in the PR body. No action this commit; bootstrap wiring lands alongside the SEED_META graduation after the ~7-day Railway-cron bake-in. Tests: 502/502 resilience tests still pass. Typecheck clean. |
||
|
Elie Habib
|
29306008e4 |
fix(email): route Intelligence Brief off the alerts@ mailbox (#3321)
* fix(email): route Intelligence Brief off the alerts@ mailbox The daily "WorldMonitor Intelligence Brief" email was shipping from `alerts@worldmonitor.app` with a display name that — if the Railway env override dropped the `Name <…>` wrapper — Gmail/Outlook fell back to rendering the local-part ("alerts" / "alert") as the sender name. Recipients saw a scary-looking "alert" in their inbox for what is actually a curated editorial read. Split the sender so editorial mail can't share the `alerts@` mailbox with incident pushes: - New env var `RESEND_FROM_BRIEF` (default `WorldMonitor Brief <brief@worldmonitor.app>`) consumed by seed-digest-notifications.mjs. - Falls back to `RESEND_FROM_EMAIL`, then to the built-in default, so existing deploys keep working and the rollout is a single Railway env flip on the digest service. - notification-relay.cjs (realtime push alerts) intentionally keeps `RESEND_FROM_EMAIL` / `alerts@` — accurate for that path. - .env.example documents the display-name rule so the bare-address trap can't re-introduce the bug. Rollout: set `RESEND_FROM_BRIEF=WorldMonitor Brief <brief@worldmonitor.app>` on the `seed-digest-notifications` Railway service. Domain-level Resend verification already covers the new local-part; no DNS change needed. * fix(email): runtime normalize sender to prevent bare-address regression PR review feedback from codex: > P2 — RESEND_FROM_BRIEF is consumed verbatim, so an operator can > still set brief@worldmonitor.app without a display name and > recreate the same Gmail/Outlook rendering bug for the daily brief. > Today that protection is only documentation in .env.example, not > runtime enforcement. Add a small shared helper `scripts/lib/resend-from.cjs` that coerces a bare email address into a "Name <addr>" wrapper with a loud warning log, and wire it into the digest path. - Bare-address input (e.g. `brief@worldmonitor.app`) is rewritten to `WorldMonitor Brief <brief@worldmonitor.app>` so Gmail/Outlook stop falling back to the local-part as the display name. - Coercion emits a single `console.warn` line per boot so operators see the signal in Railway logs and can fix the underlying env. - Fail-safe (not fail-closed) — a misconfigured env does NOT take the cron down. Also resolves the P3 doc-vs-runtime divergence by reverting .env.example's RESEND_FROM_EMAIL default from "WorldMonitor Alerts <...>" back to "WorldMonitor <...>" to match the existing notification-relay.cjs runtime default. The realtime-alert path will get the same normalizer treatment in a follow-up PR that cohesively touches notification-relay.cjs + Dockerfile.relay. tests: 7 new cases in tests/resend-sender-normalize.test.mjs covering empty/null/whitespace input, wrapped passthrough, trim, bare-address coercion, warning emission, no-warning on wrapped, console.warn default sink. Runs under `npm run test:data`. |
||
|
Elie Habib
|
8a988323d2 |
chore(bundle-runner): emit reliable per-section summary line on parent stdout (#3320)
* chore(bundle-runner): emit reliable per-section summary line on parent stdout Fixes observability asymmetry in Railway bundle service logs where some seeders appeared to skip lines like \`Run ID\`, \`Mode\`, or the structured \`seed_complete\` JSON event. Root cause is Railway's log ingestion dropping child-stdout lines when multiple seeders emit at similar timestamps — observed in the PR #3294 launch run where Pipelines-Gas was missing its \`=== Seed ===\` banner, Pipelines-Oil had \`Key:\` emitted BEFORE the banner, Storage-Facilities and Energy- Disruptions were missing Run ID + Mode + seed_complete entirely, despite identical code paths. All child processes emit the same lines; Railway just loses some. Fix is to piggy-back on the observation that bundle-level lines (\`[Bundle:X] Starting\`, \`[Bundle:X] Finished\`) ARE reliably captured — they come from the parent process's single stdout stream. Changes in scripts/_bundle-runner.mjs: - spawnSeed now captures the child's \`{"event":"seed_complete",...}\` JSON line while streaming stdout, parses it, and attaches to the settle result. - Main loop emits one bundle-level summary line per section after child exit: [Bundle:X] section=NAME status=OK durationMs=1237 records=15 state=OK (or \`status=FAILED elapsed=...s reason=...\` for failures). - Summary line survives Railway's log ingestion even when per-section child lines drop, giving monitors a reliable event to key off. Observability consumers (log-based alerts, seed telemetry scrapers) should now key off the bundle-level summary rather than per-section child lines which remain best-effort. The per-section child lines stay as-is for interactive debugging. Verification: parse logic sanity-checked against the exact seed_complete line format. Node syntax check clean. No schema changes. * fix(bundle-runner): emit FAILED summary line to stderr, not stdout The prior commit introduced a bundle-level structured summary line per section. On success that correctly goes to stdout; on FAILED it was also going to stdout — but that broke tests/bundle-runner.test.mjs test 140 ("timeout emits terminal reason BEFORE SIGTERM/SIGKILL grace"). The test concatenates stdout+stderr and asserts that `SIGKILL` appears AFTER `Failed after` in the combined string (verifying the kill-decision log line is emitted BEFORE the 10s SIGTERM→SIGKILL grace window, so it survives container termination). My new FAILED summary line — which includes the reason string `timeout after 1s (signal SIGKILL)` — landed on stdout, which comes first in the concatenation, and its `SIGKILL` substring matched before the stderr-side `Did not exit on SIGTERM...SIGKILL` line. Ordering assertion failed. Fix: route the FAILED summary line through console.error (same stream as the pre-kill `Failed after ... sending SIGTERM` and the grace-window `Did not exit...SIGKILL` lines). Chronological ordering in combined output is preserved; test passes. OK summary lines stay on stdout — they're observability data, not error diagnostics, and belong on the normal stream alongside the bundle Starting/Finished lines. Local: `node --test tests/bundle-runner.test.mjs` — 4/4 pass including the previously-failing ordering test. |
||
|
Elie Habib
|
24786882ae |
chore(railway): wire seed-sovereign-wealth into resilience-recovery bundle (#3319)
* chore(railway): add seed-sovereign-wealth to resilience-recovery bundle Wires the seeder landed in #3305 into the existing Railway cron service `seed-bundle-resilience-recovery`. One-line bundle entry; no new Railway service (the bundle pattern amortizes cron cost across the recovery-domain seeders). Config matches the rest of the bundle: - intervalMs: 30 * DAY (parity with CACHE_TTL_SECONDS=35d in the seeder + the quarterly manifest revision cadence) - timeoutMs: 600_000 (longer than peers because Tier 3b does N per-fund Wikipedia article fetches for any fund missing from the list article; today Temasek is the only miss but leaving headroom) After deploy, the next cron tick populates `resilience:recovery:sovereign-wealth:v1`, which then unblocks the follow-up PR that adds the scorer + dimension wiring. * fix(tests): update resilience-recovery bundle test for 6th entry Static-analysis test in tests/seed-bundle-resilience-recovery.test.mjs was hardcoded to `5 entries` / `all 5 entries use 30 * DAY`. Adding Sovereign-Wealth to the bundle (previous commit) made the count 6, breaking both assertions. Replaced hardcoded `5` with `EXPECTED_ENTRIES.length` so the next addition only requires appending to the allow-list at the top of the file (and the assertion message prompts the author to do that if the count drifts). Also appended the Sovereign-Wealth entry to the EXPECTED_ENTRIES list. 6566/6566 data-tier tests pass locally. |
||
|
Elie Habib
|
8032dc3a04 |
feat(resilience): PR 2 pre-scorer — SWF manifest + seeder (8/8 funds) (#3305)
* feat(resilience): PR 2 scaffolding — SWF classification manifest + seeder skeleton
Plan §3.4. First of multiple commits for PR 2 (fiscal-buffer split
and sovereign-wealth integration). This commit is SCAFFOLDING ONLY:
no dimension wiring, no scorer, no cache-keys entry yet. The goal is
to land the reviewer-facing metadata and the seeder's three-tier
source shape so an external SWF practitioner can critique before we
wire the scorer.
What is in:
1. docs/methodology/swf-classification-manifest.yaml — authoritative
per-fund classification for the `sovereignFiscalBuffer` dimension.
First-pass estimates for the 8 funds named in plan §3.4 table:
Norway GPFG, UAE ADIA + Mubadala, Saudi PIF, Kuwait KIA,
Qatar QIA, Singapore GIC + Temasek. Each fund carries:
- three-component classification (access, liquidity, transparency)
each on [0, 1], with rationale text citing the mandate / fiscal
rule / asset-mix / transparency-index evidence
- source URLs for audit
Fund-candidates deferred for external-reviewer decision are listed
in a trailing comment block (CIC, NWF, SOFAZ, NSIA, Future Fund,
NZ Super, ESSF, etc.).
external_review_status: PENDING — flip to REVIEWED on sign-off.
2. scripts/shared/swf-manifest-loader.mjs — YAML parser + strict schema
validator. Fails loudly on any deviation (out-of-range scores,
non-ISO2 countries, missing rationale, duplicate fund IDs, wrong
manifest version). Single source of truth for the seeder, future
scorer, and methodology-doc linter.
3. scripts/seed-sovereign-wealth.mjs — seeder shell with the three-tier
source priority from plan §3.4:
1. Official fund disclosures (MoF, central-bank, annual reports)
2. IFSWF member filings
3. SWFI public fund-rankings page (license-free fallback, scraped)
Tiers 1-3 are all stubbed (return null) in this commit — the
seeder publishes a well-formed empty payload so the scorer IMPUTE
fallback can be exercised end-to-end without live data.
emptyDataIsFailure: false is set deliberately so pre-wiring cron
runs do not poison seed-meta (see
feedback_strict_floor_validate_fail_poisons_seed_meta.md).
SWFI scrape target is documented in the file header with the
exact URL and a 2.5s inter-request interval. The scraper itself
lands in the next commit after the external reviewer signs off
on the manifest.
4. tests/swf-classification-manifest.test.mjs — 14 tests exercising
both the shipped YAML (plan §3.4 required-fund presence, [0,1]
bounds, rationale length, source citations, multi-fund country
handling) and the validator's schema enforcement (rejects out-
of-range scores, non-ISO2 codes, missing rationale, empty sources,
duplicates, wrong version, invalid review status).
Out of scope for this commit (follow-ups, in order):
- Implement SWFI scrape + IFSWF parse + per-fund official endpoints
- Add `liquidReserveAdequacy` and `sovereignFiscalBuffer` dimensions
to RESILIENCE_DIMENSION_ORDER, registry, and scorers
- Retire `reserveAdequacy` via RESILIENCE_RETIRED_DIMENSIONS
- cache-keys.ts + api/bootstrap.js + api/health.js wiring (new
seed key needs ON_DEMAND_KEYS gating per Railway-cron bake-in rule)
- Recovery-domain weight rebalance + Spearman sensitivity rerun
- Methodology doc: rewrite the reserveAdequacy section
Tests: 508/508 pass (resilience suite + new manifest tests).
Typecheck clean on both tsconfig.json and tsconfig.api.json.
No external-facing behavior change — all files are new + isolated.
* feat(resilience): PR 2 commit 2 — Wikipedia SWF scraper + SWFI pivot
Implements Tier 3 of the sovereignFiscalBuffer seeder. Tier 1 (official
disclosures) and Tier 2 (IFSWF filings) remain stubbed — they require
per-fund bespoke adapters and will land incrementally.
SWFI pivot
----------
The plan's original Tier 3 target was
https://www.swfinstitute.org/fund-rankings/sovereign-wealth-fund. Live
check on 2026-04-23: the page's <tbody> is empty and AUM is gated
behind a lead-capture form (name + company + job title). SWFI per-fund
/profile/<id> pages are similarly barren. The "public fund rankings"
is effectively no longer public; scraping the lead-gated surface would
require submitting fabricated contact info (TOS violation, legally
questionable), so Tier 3 pivots to Wikipedia.
Wikipedia is legally clean (CC-BY-SA 4.0, attribution required — see
WIKIPEDIA_SOURCE_ATTRIBUTION in the seeder) and structurally scrapable.
The SWFI Linaburg-Maduell Transparency Index mentioned in manifest
rationale text is a SEPARATE SWFI publication (public index scores),
not the fund-rankings paywall — those citations stay valid.
What is in
----------
1. scripts/seed-sovereign-wealth.mjs — Wikipedia scraper implementation:
- parseWikipediaRankingsTable(html) — exported pure function so
the parser is unit-testable without a live fetch. Extracts the
wikitable, parses per-fund rows (Country, Abbrev, Fund name,
Assets USD B, Inception, Origin).
- Strip-HTML helper strips <sup> tags to SPACES (not empty) so
`302.0<sup>41</sup>` stays `302.0 41` — otherwise the decimal
value and its trailing footnote ref get welded into `302.041`,
which the Assets regex mis-parses.
- matchWikipediaRecord(fund, cache) — abbrev + fund-name lookup
with country disambiguation: lookup maps are now
Map<key, Record[]> (list) rather than Map<key, Record>, and the
matcher filters the list by manifest country before returning.
This is the exact fix for the PIF collision:
"PIF" resolves to BOTH Saudi Arabia's Public Investment Fund
(~USD 925B) and Palestine's Palestine Investment Fund (~USD 900M)
on the live article. Without country-filtering, Map.set silently
overwrites one with the other, so Saudi PIF would return
Palestine's AUM — three orders of magnitude wrong.
- When the country disambiguator cannot pick, returns null rather
than a best-guess. Seeder logs the unmatched fund; the IMPUTE
path handles it gracefully.
2. docs/methodology/swf-classification-manifest.yaml — added
`wikipedia` hints block to each of the 8 funds (abbrev and/or
fund_name, matching Wikipedia's canonical naming).
3. scripts/shared/swf-manifest-loader.mjs — optional `wikipedia` field
in the schema: `abbrev` and `fund_name` both optional strings, but
at least one must be present if the block is provided.
4. tests/seed-sovereign-wealth.test.mjs — 12 tests exercising:
- fixture-based parser: abbrev/name indexing, HTML + footnote
stripping, decimal AUM, malformed rows skipped, missing-table error
- abbrev-collision handling: both candidates retained in the list
- country-disambiguation matcher: Saudi PIF correctly picked from
a Saudi-vs-Palestine collision fixture (the exact live bug)
- ambiguous lookup with unknown country returns null, not wrong record
Live verification against the shipped Wikipedia article: 7/8 funds
matched with the correct country; Saudi PIF now correctly returns
USD 925B (not Palestine's USD 0.9B) because of the country-
disambiguation fix. Temasek is the one miss — Wikipedia does not
classify it as an SWF (practitioner debate; it lists under "state
holding companies" instead). Falls through to IMPUTE in the scorer
until Tier 1/2 adapters land with an official-disclosure source.
Tests: 522/522 pass (resilience + manifest + scraper).
Typecheck clean on both tsconfig.json and tsconfig.api.json.
Still stubbed for later commits:
- Tier 1 per-fund official-disclosure adapters (incl. Temasek)
- Tier 2 IFSWF secretariat parser
- Dimension wiring (liquidReserveAdequacy, sovereignFiscalBuffer)
- reserveAdequacy retirement via RESILIENCE_RETIRED_DIMENSIONS
- cache-keys / bootstrap / health.js wiring (ON_DEMAND_KEYS until bake-in)
- Recovery-domain weight rebalance + Spearman sensitivity rerun
* feat(resilience): PR 2 commit 3 — Wikipedia infobox fallback + FX → 8/8 match
Closes the Temasek gap. The Wikipedia list article excludes Temasek on
editorial grounds (classified as a "state holding company" rather than
an SWF), so the Tier-3 list-only path topped out at 7/8 funds matched.
This commit adds Tier 3b — per-fund Wikipedia article infobox scrape
— and a baked-in FX table to handle non-USD infobox currencies.
Live verification on the shipped Wikipedia articles: 8/8 funds matched.
Temasek: S$ 434B → US$ 321B via infobox + SGD→USD FX.
Implementation
1. scripts/seed-sovereign-wealth.mjs
- FX_TO_USD table (USD, SGD, NOK, EUR, GBP, AED, SAR, KWD, QAR)
with FX_RATES_REVIEWED_AT='2026-04-23' committed into the seed
payload so stale rates are visible at audit time.
- CURRENCY_SYMBOL_TO_ISO ordered list — US$ tested before S$ before
bare $, and $ / kr require a space + digit neighbor to avoid
false-matches in rich prose.
- detectCurrency(text) exported pure for unit testing.
- parseWikipediaArticleInfobox(html) exported pure — scans rows
for "Total assets" / "Assets under management" / "AUM" / "Net
assets" / "Net portfolio value" labels, extracts "NUMBER (trillion
| billion | million) (YEAR)" values, applies FX conversion.
- fetchWikipediaInfobox(fund) — per-fund article fetch, gated on
the manifest's wikipedia.article_url hint.
- sourceMix split into {official, ifswf, wikipedia_list,
wikipedia_infobox} counters so the seed payload shows which tier
delivered each fund.
- Source priority chain: official → ifswf → wikipedia_list →
wikipedia_infobox. Infobox last because it is N network round-
trips; amortizing over the list article cache first minimizes
live traffic.
2. docs/methodology/swf-classification-manifest.yaml
- Temasek entry gains wikipedia.article_url:
https://en.wikipedia.org/wiki/Temasek_Holdings with an inline
comment explaining why the list-article path misses.
3. scripts/shared/swf-manifest-loader.mjs
- article_url optional field; validator rejects anything that is
not a https://<lang>.wikipedia.org/... URL so a typo cannot
silently wire the seeder to an off-site fetch.
4. tests/seed-sovereign-wealth.test.mjs (10 new tests, 38/38 pass)
- detectCurrency distinguishes US$ vs S$ vs bare $.
- parseWikipediaArticleInfobox extracts Temasek S$ 434B → US$ 321B
with year tag from "(2025)".
- USD-native row pass-through with fxRate=1.0.
- NOK trillion conversion (NOK 18.7T → USD 1.74T).
- Returns null when no AUM row / no infobox at all.
- Documents the unknown-currency → USD fallback contract.
Tests: 532/532 pass (full resilience + manifest + scraper suite).
Typecheck clean on both tsconfig.json and tsconfig.api.json.
Still stubbed for later commits:
- Tier 1 per-fund official-disclosure adapters
- Tier 2 IFSWF secretariat parser
- Dimension wiring (liquidReserveAdequacy, sovereignFiscalBuffer)
- reserveAdequacy retirement via RESILIENCE_RETIRED_DIMENSIONS
- cache-keys / bootstrap / health.js wiring (ON_DEMAND_KEYS)
- Recovery-domain weight rebalance + Spearman sensitivity rerun
* refactor(resilience): reuse project-shared FX infrastructure for SWF seeder
Self-caught duplication from the previous commit (
|
||
|
Elie Habib
|
84ee2beb3e |
feat(energy): Energy Atlas end-to-end — pipelines + storage + shortages + disruptions + country drill-down (#3294)
* feat(energy): pipeline registries (gas + oil) — evidence-based schema
Day 6 of the Energy Atlas Release 1 plan (Week 2). First curated asset
registry for the atlas — the real gap vs GEF.
## Curated data (critical assets only, not global completeness)
scripts/data/pipelines-gas.json — 12 critical gas lines:
Nord Stream 1/2 (offline; Swedish EEZ sabotage 2022; EU sanctions refs),
TurkStream, Yamal–Europe (offline; Polish counter-sanctions),
Brotherhood/Soyuz (offline; Ukraine transit expired 2024-12-31),
Power of Siberia, Dolphin, Medgaz, TAP, TANAP,
Central Asia–China, Langeled.
scripts/data/pipelines-oil.json — 12 critical oil lines:
Druzhba North/South (N offline per EU 2022/879; S under landlocked
derogation), CPC, ESPO (+ price-cap sanction ref), BTC, TAPS,
Habshan–Fujairah (Hormuz bypass), Keystone, Kirkuk–Ceyhan (offline
since 2023 ICC ruling), Baku–Supsa, Trans-Mountain (TMX expansion
May 2024), ESPO spur to Daqing.
Scope note: 75+ each is Week 2b work via GEM bulk import. Today's cut
is curated from first-hand operator disclosures + regulator filings so
I can stand behind every evidence field.
## Evidence-based schema (not conclusion labels)
Per docs/methodology/pipelines.mdx: no bare `sanctions_blocked` field.
Every pipeline carries an evidence bundle with `physicalState`,
`physicalStateSource`, `operatorStatement`, `commercialState`,
`sanctionRefs[]`, `lastEvidenceUpdate`, `classifierVersion`,
`classifierConfidence`. The public badge (`flowing|reduced|offline|
disputed`) is derived server-side from this bundle at read time.
## Seeder
scripts/seed-pipelines.mjs — single process publishes BOTH keys
(energy:pipelines:{gas,oil}:v1) via two runSeed() calls. Tiny datasets
(<20KB each) so co-location is cheap and guarantees classifierVersion
consistency.
Conventions followed (worldmonitor-bootstrap-registration skill):
- TTL 21d = 3× weekly cadence (gold-standard per
feedback_seeder_gold_standard.md)
- maxStaleMin 20_160 = 2× cadence (health-maxstalemin-write-cadence skill)
- sourceVersion + schemaVersion + recordCount + declareRecords wired
(seed-contract-foundation)
- Zero-case explicitly NOT allowed — MIN_PIPELINES_PER_REGISTRY=8 floor
## Health registration (dual, per feedback_two_health_endpoints_must_match)
- api/health.js: BOOTSTRAP_KEYS adds pipelinesGas + pipelinesOil;
SEED_META adds both with maxStaleMin=20_160.
- api/seed-health.js: mirror entries with intervalMin=10_080 (maxStaleMin/2).
## Bundle registration
scripts/seed-bundle-energy-sources.mjs adds a single Pipelines entry
(not two) because seed-pipelines.mjs publishes both keys in one run —
listing oil separately would double-execute. Monitoring of the oil key
staleness happens in api/health.js instead.
## Tests (tests/pipelines-registry.test.mts)
17 passing node:test assertions covering:
- Schema validation (both registries pass validateRegistry)
- Identity resolution (no id collisions, id matches object key)
- Country ISO2 normalization (from/to/transit all match /^[A-Z]{2}$/)
- Endpoint geometry within Earth bounds
- Evidence rigor: non-flowing badges require at least one supporting
evidence source (operator statement / sanctionRefs / ais-relay /
satellite / press)
- ClassifierConfidence in 0..1
- Commodity/capacity pairing (gas uses capacityBcmYr, oil uses
capacityMbd — mixing = test fail)
- validateRegistry rejects: empty object, null, no-evidence fixtures,
below-floor counts
Typecheck clean (both tsconfig.json and tsconfig.api.json).
Next: Day 7 will add list-pipelines / get-pipeline-detail RPCs in
supply-chain/v1. Day 8 ships PipelineStatusPanel with DeckGL PathLayer
consuming the registry.
* fix(energy): split seed-pipelines.mjs into two entry points — runSeed hard-exits
High finding from PR review. scripts/seed-pipelines.mjs called runSeed()
twice in one process and awaited Promise.all. But runSeed() in
scripts/_seed-utils.mjs hard-exits via process.exit on ~9 terminal paths
(lines 816, 820, 839, 888, 917, 989, plus fetch-retry 946, fatal 859,
skipped-lock 81). The first runSeed to reach any terminal path exits the
entire node process, so the second runSeed's resolve never fires — only
one of energy:pipelines:{gas,oil}:v1 would ever be written.
Since the bundle scheduled seed-pipelines.mjs exactly once, and both
api/health.js and api/seed-health.js expect both keys populated, the
other registry would stay permanently EMPTY/STALE after deploy.
Fix: split into two entry-point scripts around a shared utility.
- scripts/_pipeline-registry.mjs (NEW, was seed-pipelines.mjs) — shared
helpers ONLY. Exports GAS_CANONICAL_KEY, OIL_CANONICAL_KEY,
PIPELINES_TTL_SECONDS, MAX_STALE_MIN, buildGasPayload, buildOilPayload,
validateRegistry, recordCount, declareRecords. Underscore prefix marks
it as non-entry-point (matches _seed-utils.mjs / _seed-envelope-source.mjs
convention).
- scripts/seed-pipelines-gas.mjs (NEW) — imports from the shared module,
single runSeed('energy','pipelines-gas',…) call.
- scripts/seed-pipelines-oil.mjs (NEW) — same shape, oil.
- scripts/seed-bundle-energy-sources.mjs — register BOTH seeders (not one).
- scripts/seed-pipelines.mjs — deleted.
- tests/pipelines-registry.test.mts — update import path to the shared
module. All 17 tests still pass.
Typecheck clean (both configs). Tests pass. No other consumers import
from the deleted script.
* fix(energy): complete pipeline bootstrap registration per 4-file checklist
High finding from PR review. My earlier PR description claimed
worldmonitor-bootstrap-registration was complete, but I only touched two
of the four registries (api/health.js + api/seed-health.js). The bootstrap
hydration payload itself (api/bootstrap.js) and the shared cache-keys
registry (server/_shared/cache-keys.ts) still had no entry for either
pipeline key, so any consumer that reads bootstrap data would see
pipelinesGas/pipelinesOil as missing on first load.
Files updated this commit:
- api/bootstrap.js — KEYS map + SLOW_KEYS set both gain pipelinesGas +
pipelinesOil. Placed next to sprPolicies (same curated-registry cadence
and tier). Slow tier is correct: weekly cron, not needed on first paint.
- server/_shared/cache-keys.ts — PIPELINES_GAS_KEY + PIPELINES_OIL_KEY
exported constants (matches SPR_POLICIES_KEY pattern), BOOTSTRAP_KEYS map
entries, and BOOTSTRAP_TIERS entries (both 'slow').
Not touched (intentional):
- server/gateway.ts — pipeline data is free-tier per the Energy Atlas
plan; no PREMIUM_RPC_PATHS entry required. Energy Atlas monetization
hooks (scenario runner, MCP tools, subscriptions) are Release 2.
Full 4-file checklist now complete:
✅ server/_shared/cache-keys.ts (this commit)
✅ api/bootstrap.js (this commit)
✅ api/health.js (earlier in PR)
✅ api/seed-health.js (earlier in PR — dual-registry rule)
Typecheck clean (both configs).
* feat(energy): ListPipelines + GetPipelineDetail RPCs with evidence-derived badges
Day 7 of the Energy Atlas Release 1 plan (Week 2). Exposes the pipeline
registries (shipped in Day 6) via two supply-chain RPCs and ships the
evidence-to-badge derivation server-side.
## Proto
proto/worldmonitor/supply_chain/v1/list_pipelines.proto — new:
- ListPipelinesRequest { commodity_type?: 'gas' | 'oil' }
- ListPipelinesResponse { pipelines[], fetched_at, classifier_version, upstream_unavailable }
- GetPipelineDetailRequest { pipeline_id (required, query-param) }
- GetPipelineDetailResponse { pipeline?, revisions[], fetched_at, unavailable }
- PipelineEntry — wire shape mirroring scripts/data/pipelines-{gas,oil}.json
+ a server-derived public_badge field
- PipelineEvidence, OperatorStatement, SanctionRef, LatLon, PipelineRevisionEntry
service.proto adds both rpc methods with HTTP_METHOD_GET + path bindings:
/api/supply-chain/v1/list-pipelines
/api/supply-chain/v1/get-pipeline-detail
`make generate` regenerated src/generated/{client,server}/… + docs/api/
OpenAPI json/yaml.
## Evidence-derivation
server/worldmonitor/supply-chain/v1/_pipeline-evidence.ts — new.
derivePublicBadge(evidence) → 'flowing' | 'reduced' | 'offline' | 'disputed'
is deterministic + versioned (DERIVER_VERSION='badge-deriver-v1').
Rules (first match wins):
1. offline + sanctionRef OR expired/suspended commercial → offline
2. offline + operator statement → offline
3. offline + only press/ais/satellite → disputed (single-source negative claim)
4. reduced → reduced
5. flowing → flowing
6. unknown / malformed → disputed
Staleness guard: non-flowing badges on >14d-old evidence demote to
disputed. Flowing is the optimistic default — stale "still flowing" is
safer than stale "offline". Matches seed-pipelines-{gas,oil}.mjs maxStaleMin.
Tests (tests/pipeline-evidence-derivation.test.mts) — 15 passing cases
covering happy paths, disputed fallbacks, staleness guard, versioning.
## Handlers
server/worldmonitor/supply-chain/v1/list-pipelines.ts
- Reads energy:pipelines:{gas,oil}:v1 via getCachedJson.
- projectPipeline() narrows the Upstash `unknown` into PipelineEntry
shape + calls derivePublicBadge.
- Honors commodity_type filter (skip the opposite registry's Redis read
when the client pre-filters).
- Returns upstream_unavailable=true when BOTH registries miss.
server/worldmonitor/supply-chain/v1/get-pipeline-detail.ts
- Scans both registries by id (ids are globally unique per
tests/pipelines-registry.test.mts).
- Empty revisions[] for now; auto-revision log wires up in Week 3.
handler.ts registers both into supplyChainHandler.
## Gateway
server/gateway.ts adds 'static' cache-tier for both new RPC paths
(registry is slow-moving; 'static' matches the other read-mostly
supply-chain endpoints).
## Consumer wiring
Not in this commit — PipelineStatusPanel (Day 8) is what will call
listPipelines/getPipelineDetail via the generated client. pipelinesGas
+ pipelinesOil stay in PENDING_CONSUMERS until Day 8.
Typecheck clean (both configs). 15 new tests + 17 registry tests all pass.
* feat(energy): PipelineStatusPanel — evidence-backed status table + drawer
Day 8 of the Energy Atlas Release 1 plan. First consumer of the Day 6–7
registries + RPCs.
## What this PR adds
- src/components/PipelineStatusPanel.ts — new panel (id=pipeline-status).
* Bootstrap-hydrates from pipelinesGas + pipelinesOil for instant first
paint; falls through to listPipelines() RPC if bootstrap misses.
Background re-fetch runs on every render so a classifier-version bump
between bootstrap stamp and first view produces a visible update.
* Table rows sorted non-flowing-first (offline / reduced / disputed
before flowing) — what an atlas reader cares about.
* Click-to-expand drawer calls getPipelineDetail() lazily — operator
statements, sanction refs (with clickable source URLs), commercial
state, classifier version + confidence %, capacity + route metadata.
* publicBadge color-chip palette matches the methodology doc.
* Attribution footer with GEM (CC-BY 4.0) credit + classifier version.
- src/components/index.ts — barrel export.
- src/app/panel-layout.ts — import + createPanel('pipeline-status', …).
- src/config/panels.ts — ENERGY_PANELS adds 'pipeline-status' at priority 1.
## PENDING_CONSUMERS cleanup
tests/bootstrap.test.mjs — removes 'pipelinesGas' + 'pipelinesOil' from
the allowlist. The invariant "every bootstrap key has a getHydratedData
consumer" now enforces real wiring for these keys: the panel literally
calls getHydratedData('pipelinesGas') and getHydratedData('pipelinesOil').
Future regressions that remove the consumer will fail pre-push.
## Consumer contract verified
- 67 tests pass including bootstrap.test.mjs consumer coverage check.
- Typecheck clean.
- No DeckGL PathLayer in this commit — existing 'pipelines-layer' has a
separate data source, so modifying DeckGLMap.ts to overlay evidence-
derived badges on the map is a follow-up commit to avoid clobbering.
## Out of scope for Day 8 (next steps on same PR)
- DeckGL PathLayer integration (color pipelines on the main map by
publicBadge, click-to-open this drawer) — Day 8b commit.
- Storage facility registry + StorageFacilityMapPanel — Days 9-10.
* fix(energy): PipelineStatusPanel bootstrap path — client-side badge derivation
High finding from PR review. The Day-8 panel crashed on first paint
whenever bootstrap hydration succeeded, because:
- Bootstrap hydrates raw scripts/data/pipelines-{gas,oil}.json verbatim.
- That JSON does NOT include publicBadge — that field is only added by
the server handler's projectPipeline() in list-pipelines.ts.
- PipelineStatusPanel passed raw entries into badgeChip(), which called
badgeLabel(undefined).charAt(0) → TypeError.
The background RPC refresh that would have repaired the data never ran
because the panel threw before reaching it. So the exact bootstrap path
newly wired in commit
|
||
|
Elie Habib
|
706e0e3d01 |
chore(sentry): silence Opera extension injection noise (WORLDMONITOR-NS/NT/NV) (#3304)
Three cascading errors fired within ~40 ms from a single Opera 130 / Windows 10 session — all signatures of an extension injecting broken JS: - `oîUpdateObj` contains a non-ASCII `î`; our minifier only emits ASCII. - `Octal literals are not allowed in strict mode` — our TS bundle never uses octals and never runs eval. - `Unexpected identifier 'm'` — a pre-compiled bundle cannot parse-fail at runtime, so this is foreign `<script>` evaluation. Add three narrowly-scoped `ignoreErrors` patterns (the non-ASCII property-name one is specifically gated on the `reading '…'` shape so genuine first-party property accesses remain visible). |
||
|
Elie Habib
|
7cf37c604c |
feat(resilience): PR 3 — dead-signal cleanup (plan §3.5, §3.6) (#3297)
* feat(resilience): PR 3 §3.5 — retire fuelStockDays from core score permanently
First commit in PR 3 of the resilience repair plan. Retires
`fuelStockDays` from the core score with no replacement.
Why permanent, not replaced:
IEA emergency-stockholding rules are defined in days of NET IMPORTS
and do not bind net exporters by design. Norway/Canada/US measured
in days-of-imports are incomparable to Germany/Japan measured the
same way — the construct is fundamentally different across the two
country classes. No globally-comparable recovery-fuel signal can
be built from this source; the pre-repair probe showed 100% imputed
at 50 for every country in the April 2026 freeze.
scoreFuelStockDays:
- Rewritten to return coverage=0 + observedWeight=0 +
imputationClass='source-failure' for every country regardless
of seed content.
- Drops the dimension from the `recovery` domain's coverage-
weighted mean automatically; remaining recovery dimensions
pick up the share via re-normalisation in
`_shared.ts#coverageWeightedMean`.
- No explicit weight transfer needed — the coverage-weighted
blend handles redistribution.
Registry:
- recoveryFuelStockDays re-tagged from tier='enrichment' to
tier='experimental' so the Core coverage gate treats it as
out-of-score.
- Description updated to make the retirement explicit; entry
stays in the registry for structural continuity (the
dimension `fuelStockDays` remains in RESILIENCE_DIMENSION_ORDER
for the 19-dimension tests; removing the dimension entirely is
a PR 4 structural-audit concern).
Housekeeping:
- Removed `RESILIENCE_RECOVERY_FUEL_STOCKS_KEY` constant (no
longer read; noUnusedLocals would reject it).
- Removed `RecoveryFuelStocksCountry` interface for the same
reason. Comment at the removed declaration instructs future
maintainers not to re-add the type as a reservation; when a
new recovery-fuel concept lands, introduce a fresh interface.
Plan reference: §3.5 point 1 of
`docs/plans/2026-04-22-001-fix-resilience-scorer-structural-bias-plan.md`.
51 resilience tests pass, typecheck + biome clean. The
`recovery` domain's published score will shift slightly for every
country because the 0.10 slot that fuelStockDays was imputing to
now redistributes; the compare-harness acceptance-gate rerun at
merge time will quantify the shift per plan §6 gates.
* feat(resilience): PR 3 §3.5 — retire BIS-backed currencyExternal; rebuild on IMF inflation + WB reserves
BIS REER/DSR feeds were load-bearing in currencyExternal (weights 0.35
fxVolatility + 0.35 fxDeviation, ~70% of dimension). They cover ~60
countries max — so every non-BIS country fell through to
curated_list_absent (coverage 0.3) or a thin IMF proxy (coverage 0.45).
Combined with reserveMarginPct already removed in PR 1, currencyExternal
was the clearest "construct absent for most of the world" carrier left
in the scorer.
Changes:
_dimension-scorers.ts
- scoreCurrencyExternal now reads IMF macro (inflationPct) + WB FX
reserves only. Coverage ladder:
inflation + reserves → 0.85 (observed primary + secondary)
inflation only → 0.55
reserves only → 0.40
neither → 0.30 (IMPUTE.bisEer retained for snapshot
continuity; semantics read as
"no IMF + no WB reserves" now)
- Removed dead symbols: RESILIENCE_BIS_EXCHANGE_KEY constant (reserved
via comment only, flagged by noUnusedLocals), stddev() helper,
getCountryBisExchangeRates() loader, BisExchangeRate interface,
dateToSortableNumber() — all were exclusive callers of the retired
BIS path.
_indicator-registry.ts
- New core entry inflationStability (weight 0.60, tier=core,
sourceKey=economic:imf:macro:v2).
- fxReservesAdequacy weight 0.15 → 0.40 (secondary reliability
anchor).
- fxVolatility + fxDeviation demoted tier=enrichment → tier=experimental
(BIS ~60-country coverage; off the core weight sum).
- Non-experimental weights now sum to 1.0 (0.60 + 0.40).
scripts/compare-resilience-current-vs-proposed.mjs
- EXTRACTION_RULES: added inflationStability →
imf-macro-country-field field=inflationPct so the registry-parity
test passes and the correlation harness sees the new construct.
tests/resilience-dimension-scorers.test.mts
- Dropped BIS-era wording ("non-BIS country") and test 266
(BIS-outage coverage 0.35 branch) which collapsed to the inflation-
only path post-retirement.
- Updated coverage assertions: inflation-only 0.45 → 0.55; inflation+
reserves 0.55 → 0.85.
tests/resilience-scorers.test.mts
- domainAverages.economic 68.33 → 66.33 (US currencyExternal score
shifts slightly under IMF+reserves vs old BIS composite).
- stressScore 67.85 → 67.21; stressFactor 0.3215 → 0.3279.
- overallScore 65.82 → 65.52.
- baselineScore unchanged (currencyExternal is stress-only).
All 6324 data-tier tests pass. typecheck:api clean. No change to
seeders or Redis keys; this is a pure scorer + registry rebuild.
* feat(resilience): PR 3 §3.5 point 3 — re-goalpost externalDebtCoverage (0..5 → 0..2)
Plan §2.1 diagnosis table showed externalDebtCoverage saturating at
score=100 across all 9 probe countries — including stressed states.
Signal was collapsed. Root cause: (worst=5, best=0) gave every country
with ratio < 0.5 a score above 90, and mapped Greenspan-Guidotti's
reserve-adequacy threshold (ratio=1.0) to score 80 — well into "no
worry" territory instead of the "mild warning" it should be.
Re-anchored on Greenspan-Guidotti directly: ratio=1.0 now maps to score
50 (mild warning), ratio=2.0 to score 0 (acute rollover-shock exposure).
Ratios above 2.0 clamp to 0, consistent with "beyond this point the
country is already in crisis; exact value stops mattering."
Files changed:
- _indicator-registry.ts: recoveryDebtToReserves goalposts
{worst: 5, best: 0} → {worst: 2, best: 0}. Description updated to
cite Greenspan-Guidotti; inline comment documents anchor + rationale.
- _dimension-scorers.ts: scoreExternalDebtCoverage normalizer bound
changed from (0..5) to (0..2), with inline comment.
- docs/methodology/country-resilience-index.mdx: goalpost table row
5-0 → 2-0, description cites Greenspan-Guidotti.
- docs/methodology/indicator-sources.yaml:
* constructStatus: dead-signal → observed-mechanism (signal is now
discriminating).
* reviewNotes updated to describe the new anchor.
* mechanismTestRationale names the Greenspan-Guidotti rule.
- tests/resilience-dimension-monotonicity.test.mts: updated the
comment + picked values inside the (0..2) discriminating band (0.3
and 1.5). Old values (1 vs 4) had 4 clamping to 0.
- tests/resilience-dimension-scorers.test.mts: NO score threshold
relaxed >90 → >=85 (NO ratio=0.2 now scores 90, was 96).
- tests/resilience-scorers.test.mts: fixture drift:
* domainAverages.recovery 54.83 → 47.33 (US extDebt 70 → 25).
* baselineScore 63.63 → 60.12 (extDebt is baseline type).
* overallScore 65.52 → 63.27.
* stressScore / stressFactor unchanged (extDebt is baseline-only).
All 6324 data-tier tests pass. typecheck:api clean.
* feat(resilience): PR 3 §3.6 — CI gate on indicator coverage and nominal weight
Plan §3.6 adds a new acceptance criterion (also §5 item 5):
> No indicator with observed coverage below 70% may exceed 5% nominal
> weight OR 5% effective influence in the post-change sensitivity run.
This commit enforces the NOMINAL-WEIGHT half as a unit test that runs
on every CI build. The EFFECTIVE-INFLUENCE half is produced by
scripts/validate-resilience-sensitivity.mjs as a committed artifact;
the gate file only asserts that script still exists so a refactor that
removes it breaks the build loudly.
Why the gate exists (plan §3.6):
"A dimension at 30% observed coverage carries the same effective
weight as one at 95%. This contradicts the OECD/JRC handbook on
uncertainty analysis."
Implementation:
tests/resilience-coverage-influence-gate.test.mts — three tests:
1. Nominal-weight gate: for every core indicator with coverage < 137
countries (70% of the ~195-country universe), computes its nominal
overall weight as
indicator.weight × (1/dimensions-in-domain) × domain-weight
and asserts it does not exceed 5%. Equal-share-per-dimension is
the *upper bound* on runtime weight (coverage-weighted mean gives
a lower share when a dimension drops out), so this is a strict
bound: if the nominal number passes, the runtime number also
passes for every country.
2. Effective-influence contract: asserts the sensitivity script
exists at its expected path. Removing it (intentionally or by
refactor) breaks the build.
3. Audit visibility: prints the top 10 core indicators by nominal
overall weight. No assertion beyond "ran" — the list lets
reviewers spot outliers that pass the gate but are near the cap.
Current state (observed from audit output):
recoveryReserveMonths: nominal=4.17% coverage=188
recoveryDebtToReserves: nominal=4.17% coverage=185
recoveryImportHhi: nominal=4.17% coverage=190
inflationStability: nominal=3.40% coverage=185
electricityConsumption: nominal=3.30% coverage=217
ucdpConflict: nominal=3.09% coverage=193
Every core indicator has coverage ≥ 180 (already enforced by the
pre-existing indicator-tiering test), so the nominal-weight gate has
no current violators — its purpose is catching future drift, not
flagging today's state.
All 6327 data-tier tests pass. typecheck:api clean.
* docs(resilience): PR 3 methodology doc — document §3.5 dead-signal retirements + §3.6 coverage gate
Methodology-doc update capturing the three §3.5 landings and the §3.6 CI
gate. Five edits:
1. **Known construct limitations section (#5 and #6):** strikethrough the
original "dead signals" and "no coverage-based weight cap" items,
annotate them with "Landed in PR 3 §3.5"/"Landed in PR 3 §3.6" +
specifics of what shipped.
2. **Currency & External H4 section:** completely rewritten. Old table
(fxVolatility / fxDeviation / fxReservesAdequacy on BIS primary) is
replaced by the two-indicator post-PR-3 table (inflationStability at
0.60 + fxReservesAdequacy at 0.40). Coverage ladder spelled out
(0.85 / 0.55 / 0.40 / 0.30). Legacy BIS indicators named as
experimental-tier drill-downs only.
3. **Fuel Stock Days H4 section:** H4 heading text kept verbatim so the
methodology-lint H4-to-dimension mapping does not break; body
rewritten to explain that the dimension is retired from core but the
seeder still runs for IEA-member drill-downs.
4. **External Debt Coverage table row:** goalpost 5-0 → 2-0, description
cites Greenspan-Guidotti reserve-adequacy rule.
5. **New v2.2 changelog entry** — PR 3 dead-signal cleanup, covering
§3.5 points 1/2/3 + §3.6 + acceptance gates + construct-audit
updates.
No scoring or code changes in this commit. Methodology-lint test passes
(H4 mapping intact). All 6327 data-tier tests pass.
* fix(resilience): PR 3 §3.6 gate — correct share-denominator for coverage-weighted aggregation
Reviewer catch (thanks). The previous gate computed each indicator's
nominal overall weight as
indicator.weight × (1 / N_total_dimensions_in_domain) × domain_weight
and claimed this was an upper bound ("actual runtime weight is ≤ this
when some dimensions drop out on coverage"). That is BACKWARDS for
this scorer.
The domain aggregation is coverage-weighted
(server/worldmonitor/resilience/v1/_shared.ts coverageWeightedMean),
so when a dimension pins at coverage=0 it is EXCLUDED from the
denominator and the surviving dimensions' shares go UP, not down.
PR 3 commit 1 retires fuelStockDays by hard-coding its scorer to
coverage=0 for every country — so in the current live state the
recovery domain has 5 contributing dimensions (not 6), and each core
recovery indicator's nominal share is
1.0 × 1/5 × 0.25 = 5.00% (was mis-reported as 4.17%)
The old gate therefore under-estimated nominal influence and could
silently pass exactly the kind of low-coverage overweight regression
it is meant to block.
Fix:
- Added `coreBearingDimensions(domainId)` helper that counts only
dimensions that have ≥1 core indicator in the registry. A dimension
with only experimental/enrichment entries (post-retirement
fuelStockDays) has no core contribution → does not dilute shares.
- Updated `nominalOverallWeight` to divide by the core-bearing count,
not the raw dimension count.
- Rewrote the helper's doc comment to stop claiming this is a strict
upper bound — explicitly calls out the dynamic case (source failure
raising surviving dim shares further) as the sensitivity script's
responsibility.
- Added a new regression test: asserts (a) at least one recovery
dimension is all-non-core (fuelStockDays post-retirement),
(b) fuelStockDays has zero core indicators, and (c) recoveryDebt
ToReserves nominal = 0.05 exactly (not 0.0417) — any reversion
of the retirement or regression to N_total-denominator will fail
loudly.
Top-10 audit output now correctly shows:
recoveryReserveMonths: nominal=5% coverage=188
recoveryDebtToReserves: nominal=5% coverage=185
recoveryImportHhi: nominal=5% coverage=190
(was 4.17% each under the old math)
All 486 resilience tests pass. typecheck:api clean.
Note: the 5% figure is exactly AT the cap, not over it. "exceed" means
strictly > 5%, so it still passes. But now the reviewer / audit log
reflects reality.
* fix(resilience): PR 3 review — retired-dim confidence drag + false source-failure label
Addresses the Codex review P1 + P2 on PR #3297.
P1 — retired-dim drag on confidence averages
--------------------------------------------
scoreFuelStockDays returns coverage=0 by design (retired construct),
but computeLowConfidence, computeOverallCoverage, and the widget's
formatResilienceConfidence averaged across all 19 dimensions. That
dragged every country's reported averageCoverage down — US went from
0.8556 (active dims only) to 0.8105 (all dims) — enough drift to
misclassify edge countries as lowConfidence and to shift the ranking
widget's overallCoverage pill for every country.
Fix: introduce an authoritative RESILIENCE_RETIRED_DIMENSIONS set in
_dimension-scorers.ts and filter it out of all three averages. The
filter is keyed on the retired-dim REGISTRY, not on coverage === 0,
because a non-retired dim can legitimately emit coverage=0 on a
genuinely sparse-data country via weightedBlend fall-through — those
entries MUST keep dragging confidence down (that is the sparse-data
signal lowConfidence exists to surface). Verified: sparse-country
release-gate test (marks sparse WHO/FAO countries as low confidence)
still passes with the registry-keyed filter; would have failed with
a naive coverage=0 filter.
Server-client parity: widget-utils cannot import server code, so
RESILIENCE_RETIRED_DIMENSION_IDS is a hand-mirrored constant, kept
in lockstep by tests/resilience-retired-dimensions-parity.test.mts
(parses the widget file as text, same pattern as existing widget-util
tests that can't import the widget module directly).
P2 — false "Source down" label on retired dim
---------------------------------------------
scoreFuelStockDays hard-coded imputationClass: 'source-failure',
which the widget maps to "Source down: upstream seeder failed" with
a `!` icon for every country. That is semantically wrong for an
intentional retirement. Flipped to null so the widget's absent-path
renders a neutral cell without a false outage label. null is already
a legal value of ResilienceDimensionScore.imputationClass; no type
change needed.
Tests
-----
- tests/resilience-confidence-averaging.test.mts (new): pins the
registry-keyed filter semantic for computeOverallCoverage +
computeLowConfidence. Includes a negative-control test proving
non-retired coverage=0 dims still flip lowConfidence.
- tests/resilience-retired-dimensions-parity.test.mts (new):
lockstep gate between server and client retired-dim lists.
- Widget test adds a registry-keyed exclusion test with a non-retired
coverage=0 dim in the fixture to lock in the correct semantic.
- Existing tests asserting imputationClass: 'source-failure' for
fuelStockDays flipped to null.
All 494 resilience tests + full 6336/6336 data-tier suite pass.
Typecheck clean for both tsconfig.json and tsconfig.api.json.
* docs(resilience): align methodology + registry metadata with shipped imputationClass=null
Follow-up to the previous PR 3 review commit that flipped
scoreFuelStockDays's imputationClass from 'source-failure' to null to
avoid a false "Source down" widget label on every country. The code
changed; the doc and registry metadata did not, leaving three sites
in the methodology mdx and two comment/description sites in the
registry still claiming imputationClass='source-failure'. Any future
reviewer (or tooling that treats the registry description as
authoritative) would be misled.
This commit rewrites those sites to describe the shipped behavior:
- imputationClass=null (not 'source-failure'), with the rationale
- exclusion from confidence/coverage averages via the
RESILIENCE_RETIRED_DIMENSIONS registry filter
- the distinction between structural retirement (filtered) and
runtime coverage=0 (kept so sparse-data countries still flag
lowConfidence)
Touched:
- docs/methodology/country-resilience-index.mdx (lines ~33, ~268, ~590)
- server/worldmonitor/resilience/v1/_indicator-registry.ts
(recoveryFuelStockDays comment block + description field)
No code-behavior change. Docs-only.
Tests: 157 targeted resilience tests pass (incl. methodology-lint +
widget + release-gate + confidence-averaging). Typecheck clean on
both tsconfig.json and tsconfig.api.json.
|
||
|
Elie Habib
|
c067a7dd63 |
fix(resilience): include hydroelectric in lowCarbonGenerationShare (PR #3289 follow-up) (#3293)
Greptile P1 review on the merged PR #3289: World Bank EG.ELC.RNEW.ZS explicitly excludes hydroelectric. The v2 lowCarbonGenerationShare composite was summing only nuclear + renew-ex-hydro, which would collapse to ~0 for hydro-dominant economies the moment the RESILIENCE_ENERGY_V2_ENABLED flag flipped: Norway ~95% hydro → score near 0 on a 0.20-weight indicator Paraguay ~99% hydro → same Brazil ~65% hydro → same Canada ~60% hydro → same Directly contradicts the plan §3.3 intent of crediting "firm low-carbon generation" and would produce rankings that contradict the power-system security framing. PR #3289 merged before the review landed. This branch applies the fix against main. Fix: add EG.ELC.HYRO.ZS as a third series in the composite. seed-low-carbon-generation.mjs: - INDICATORS: ['EG.ELC.NUCL.ZS', 'EG.ELC.RNEW.ZS'] + 'EG.ELC.HYRO.ZS' - fetchLowCarbonGeneration(): sum three series, track latest year across all three, same cap-at-100 guard - File header comment names the three-series sum with the hydro- exclusion rationale + the country list that would break. _indicator-registry.ts lowCarbonGenerationShare.description: rewritten to name all three WB codes + explain the hydro exclusion. country-resilience-index.mdx: - Known-limitations item 3 names all three WB codes + country list - Energy domain v2 table row names all three WB codes - v2.1 changelog Indicators-added bullet names all three WB codes - v2.1 changelog New-seeders bullet names all three WB codes on seed-low-carbon-generation No scorer code change (composite lives in the seeder; scorer reads the pre-summed value from resilience:low-carbon-generation:v1). No weight change. Flag-off path remains byte-identical. 25 resilience tests pass, typecheck + typecheck:api clean. |
||
|
Elie Habib
|
c489aa6dab |
fix(pro-marketing): /pro reflects entitlement — swap upgrade CTAs to dashboard link for Pro users (#3301)
* fix(pro-marketing): /pro reflects entitlement — swap upgrade CTAs for dashboard link when user is already Pro
Reported: a Pro subscriber visiting /pro was pitched "UPGRADE TO PRO"
in the nav + "CHOOSE YOUR PLAN" in the hero, even though the dashboard
correctly recognized them as Pro. The avatar bubble was visible (per
PR-3250) but the upgrade CTAs were unconditional — none of the 14 prior
rollout PRs added an entitlement check to the /pro bundle.
Root cause: pro-test (/pro) is a separate React bundle with no Convex
client and no entitlement awareness. PR-3250 made the nav auth-aware
(signed-in vs anonymous) but not entitlement-aware (pro vs free). A
paying Dodo subscriber whose Clerk `publicMetadata.plan` isn't written
(our webhook pipeline doesn't set it — documented in panel-gating.ts)
still sees the upgrade pitch.
## Changes
### api/me/entitlement.ts (new)
Tiny edge endpoint returning `{ isPro: boolean }` via the existing
`isCallerPremium` helper, which does the canonical two-signal check
(Clerk pro role OR Convex Dodo entitlement tier >= 1). `Cache-Control:
private, no-store` — entitlement flips when Dodo webhooks fire, and
/pro reads it on every load.
### pro-test/src/App.tsx — useProEntitlement hook + conditional CTAs
- New `useProEntitlement(signedIn)` hook. When signed in, fetches
`/api/me/entitlement` with the Clerk bearer token. Falls back to
`isPro: false` on any error — /pro stays in upgrade-pitch mode
rather than silently hiding the purchase path on a flaky network.
- Navbar: "UPGRADE TO PRO" → "GO TO DASHBOARD" (→ worldmonitor.app)
when `isLoaded && user && isChecked && isPro`. Free/anonymous users
see the original upgrade CTA unchanged.
- Hero: "CHOOSE YOUR PLAN" → "GO TO DASHBOARD" under the same condition.
Also removes the #pricing anchor jump which is actively misleading
for a paying customer.
- Deliberately delays the swap until the entitlement check resolves —
a one-frame flash of "Upgrade" for a free signed-in user is better
than a flash of "Go to Dashboard" for an unpaid visitor.
### Locale: en.json adds `nav.goToDashboard` + `hero.goToDashboard`
Other locales fall back to English via i18next's `fallbackLng` — no
translation files need updating for this change to work everywhere.
Bundle rebuilt on Node 22 to match CI.
## Post-Deploy Monitoring & Validation
- Test: sign in on /pro as an existing Pro user → nav shows
"GO TO DASHBOARD", hero CTA shows "GO TO DASHBOARD".
- Test: sign in on /pro as a free user → original "UPGRADE TO PRO" /
"CHOOSE YOUR PLAN" CTAs remain unchanged.
- Test: anonymous visitor → identical to pre-change behavior.
- Failure signal: any user report of "went to /pro as Pro user, still
saw upgrade" within 48h → rollback trigger. Check Sentry for
`surface: pro-marketing` + action `load-clerk-for-nav` or similar.
* fix(pro-marketing): address PR review — sebuf exception + Sentry on entitlement check
- api/api-route-exceptions.json: register /api/me/entitlement.ts as an
internal-helper exception. It's a thin wrapper over the canonical
isCallerPremium helper; the authoritative gates remain in panel-gating,
isCallerPremium, and gateway.ts PREMIUM_RPC_PATHS. This endpoint exists
only so the separate pro-test bundle (no Convex client) can ask the
same question without reimplementing the two-signal check. Unblocks
the sebuf API contract lint.
- Greptile P2: capture entitlement-check failures to Sentry to match
the useClerkUser catch-block pattern. Tag surface=pro-marketing,
action=check-entitlement.
* fix(pro-marketing): address PR review — retry-on-null-token + share entitlement state via context
Addresses reviewer P1 + P2 on PR #3301:
P1 — useProEntitlement treated a first null token as a final "not Pro"
result. Clerk can expose `user` before the session-token endpoint is
ready (same reason services/checkout.ts:getAuthToken retries once after
2s). Without retry, a real Pro user hitting /pro on a cold Clerk load
got a permanent isPro=false for the whole session, so the upgrade CTAs
stayed visible even after Clerk finished warming up. Fix: mirror the
checkout.ts retry pattern — try, sleep 2s, try again.
P2 — Navbar and Hero each called useProEntitlement(!!user), producing
two independent /api/me/entitlement fetches AND two independent state
machines that could disagree on transient failure (one 200, one 500 →
nav and hero showing different CTAs). Fix: hoist the effect into a
ProEntitlementProvider at the App root; Navbar and Hero now both read
from the same Context. One fetch per page load, one source of truth.
No behavior change for anonymous users or for successful Pro checks.
* fix(api/me/entitlement): distinguish auth failure from free-tier
Reviewer P2: returning 200 { isPro: false } for both "free user" and
"bearer missing/invalid" collapses the two states, making a /pro auth
regression read like normal free-tier traffic in edge logs / monitoring.
Fix: validate the bearer with validateBearerToken BEFORE delegating to
isCallerPremium. On missing/malformed/invalid bearer return 401
{ error: "unauthenticated" }; on valid bearer return 200 { isPro } as
before. /pro's client already treats any non-200 as isPro:false (safe
default), so no behavior change for callers — only observability
improves.
P1 (reviewer claim): PR-3298's wm_checkout=success bridge is not wired
end-to-end. NOT reproducible — src/services/checkout-return.ts lines
35-36, 52, and 100 already recognize the marker and return
{ kind: 'success' }, which src/app/panel-layout.ts:190 consumes via
`returnResult.kind === 'success'` to trigger showCheckoutSuccess. No
code change needed; the wiring landed in PR-3274 before PR-3298.
|
||
|
Elie Habib
|
d8aee050cb |
fix(sentry): triage 4 inbox issues — silence no-user checkout + deck.gl/Safari noise (#3303)
* fix(sentry): triage 4 inbox issues — silence no-user checkout + deck.gl/Safari noise
- checkout.ts: skip Sentry emit when action='no-user' (pre-auth redirect UX,
not an error; Clerk analytics already tracks this funnel). session_expired
and other codes keep emitting so mid-flight auth drops stay visible.
- main.ts beforeSend: gate short-var Safari ReferenceError
(/^Can't find variable: \w{1,2}$/) with empty stack + !hasFirstParty —
WORLDMONITOR-NQ ('ss' injection).
- main.ts beforeSend: extend deck.gl internal null-access gate to cover
\w{1,3}\.isHidden when !hasFirstParty — WORLDMONITOR-NR (Safari 26.4 beta
MVTLayer crash preceded by DeckGLMap map-error breadcrumbs). First-party
SmartPollContext.isHidden regressions still surface.
Resolves WORLDMONITOR-NN / NQ / NR in Sentry (inNextRelease). NP
(Convex funrun permit) resolved w/o code change — emitted by Convex→Sentry
SDK, not our bundle; recurrence = real concurrency signal.
* test(sentry): add regression coverage for checkout-skip policy + deck.gl/short-var filters
Addresses P2 review findings on PR #3303:
- Extract skip-Sentry predicate to src/services/checkout-sentry-policy.ts
(mirrors checkout-no-user-policy.ts). Pins the contract: 'no-user' skips;
everything else (no-token, http-error, missing-checkout-url, exception,
entitlement-timeout) emits. A static-source assertion walks every
reportCheckoutError call site in checkout.ts and asserts its policy
decision — forces explicit declaration on any new action string.
- Add 8 tests to tests/sentry-beforesend.test.mjs for the two new filters:
* \w{1,3}\.isHidden deck.gl gate: suppresses empty-stack + vendor-only,
surfaces first-party (SmartPollContext.isHidden) regressions, and
surfaces 4+ char symbol accesses.
* ^Can't find variable: \w{1,2}$ gate: suppresses empty-stack short-var,
surfaces first-party frames on same message, surfaces 3+ char names.
test:data: 6358 pass (was 6340 → +18 new regression tests).
|
||
|
Elie Habib
|
858f03c754 |
fix(gating): silence anonymous 401s on trade-policy + regional-intelligence (#3295)
* fix(gating): silence anonymous 401s on trade-policy + regional-intelligence + country-intel New-user trace on worldmonitor.app showed 3 PRO-gated RPCs firing on every page load for anonymous visitors, producing red console errors and Sentry noise: GET /api/intelligence/v1/get-regional-snapshot?region_id=mena 401 GET /api/trade/v1/get-tariff-trends?... 401 GET /api/trade/v1/list-comtrade-flows 401 Fix follows the canonical 4-layer PRO-panel-gating pattern: trade-policy panel (was missing ALL four layers): - Layer 1: add premium:locked to config across full/finance/commodity - Layer 2: add 'trade-policy' to apiKeyPanels so PRO web users stay entitled - Layer 3: short-circuit loadTradePolicy() in data-loader with if !hasPremiumAccess return. Was firing 6 WTO RPCs unconditionally. - Layer 4: move primeTask into the _wmAccess block; add hasPremiumAccess to the scheduleRefresh viewport gate so scroll-into-view doesn't re-trigger 401s for free users. regional-intelligence panel (had Layers 1+2, missing per-component guard): - RegionalIntelligenceBoard.loadCurrent short-circuits with renderEmpty when !hasPremiumAccess, mirroring the IS_EMBEDDED_PREVIEW pattern. Constructor's void loadCurrent was the source of the 401. country-intel per-country drill-down: - Wrap listComtradeFlows + getTariffTrends in hasPremiumAccess (mirrors the guard already used for multi-sector cost shock nearby). Tests: 6326 pass. Typecheck + biome lint clean on changed files. No bootstrap/cache-keys changes so sync test stays green. * fix(gating): re-fire PRO loaders on false-to-true auth transition (PR #3295 review) Reviewer flagged two auth-resolution races introduced by PR #3295's hasPremiumAccess() short-circuits: P1 — RegionalIntelligenceBoard.loadCurrent returns early if auth is not ready at construction time, but the panel never re-fires when entitlement later resolves. panel-layout.ts already re-applies the visual lock state reactively via subscribeAuthState, so the user would end up with an unlocked PRO panel whose content stays empty until they manually change region or reload the page. P2 — Same class of race for trade-policy. Initial priming is now gated behind hasPremiumAccess at boot, and the fallback refresh path is scheduled with runImmediately:false on a 10-minute poll. A PRO user whose Clerk session resolves after boot would see the panel unlock immediately but stay empty for up to 10 minutes. Fix: - src/components/RegionalIntelligenceBoard.ts: subscribe to subscribeAuthState in the constructor, track lastHadPremium on the instance, and re-fire loadCurrent on a false to true transition. Also blank the panel on a true to false transition so a signed-out user does not keep seeing stale PRO data until their next route change. Subscription teardown is intentionally discarded since Panel has no destroy hook and the panel lives for the app lifetime. - src/App.ts: in the existing unsubFreeTier auth subscription, track prevHadPremium and fire dataLoader.loadTradePolicy once on a false to true transition. The loader internally re-checks hasPremiumAccess and the panel existence, so the call is idempotent and safe even if the transition inverts during the microtask. Both paths use false to true TRANSITION gating, not every-auth-event re-fires, so unrelated auth updates (cloud prefs sync, user metadata, session refresh) do not hammer the RPCs. Tests: 6326 pass. Typecheck + lint clean. * fix(gating): re-fire country-brief PRO sections on auth transition (PR #3295 review) Third P1 from reviewer, same class as the RegionalIntelligenceBoard + trade-policy fixes already in this PR. country-intel.ts:490 runs fetchProSections(code) only if hasPremiumAccess is true at the moment the country opens. A user whose Clerk / API-key entitlement resolves AFTER they open a country would see empty nationalDebt, sanctionsPressure, comtradeFlows, and tariffTrends cards with no retry path — panel lock state updates reactively elsewhere, but the data never arrives until the user reselects the country or reloads. Fix mirrors the RegionalIntelligenceBoard shape: - CountryIntelManager.init() subscribes to subscribeAuthState and tracks lastHadPremium on the instance. On a false to true transition, call fetchProSections(openCode) for whatever country the user is currently viewing (if any). No open country = nothing to retry; the next country open picks up the new entitlement via the existing hasPremiumAccess check at line 490. - destroy() unsubscribes the auth listener (this module HAS a destroy hook, unlike Panel, so clean teardown matters here). - fetchProSections already covers all four gated cards (nationalDebt, sanctionsPressure, comtradeFlows, tariffTrends — the comtrade/tariff calls were added to fetchProSections in the earlier commit on this branch), so a single call covers everything. Transition-based gating prevents unrelated auth events (prefs sync, session refresh) from re-hammering RPCs. Only the specific "entitlement just became available" edge triggers a refetch. Tests: 6326 pass. Typecheck + lint clean. * fix(gating): unsubscribe auth listener in RegionalIntelligenceBoard.destroy (PR #3295 review P2) Reviewer correctly flagged that my previous "Panel has no destroy hook" comment was wrong. Panel DOES expose destroy() (Panel.ts:1134), and it IS called from the panel-layout teardown path (panel-layout.ts:293, App.ts:1156). Dropping the subscribeAuthState unsubscribe handle meant every recreated board would leave behind a live subscriber that still captured `this` of the destroyed instance, ready to call loadCurrent (or renderEmpty) on stale DOM/state on every future auth event. Fix: - Store the subscribeAuthState return as this.authUnsubscribe rather than discarding it. - Override destroy() to call this.authUnsubscribe?.() and null it, then super.destroy(). - Also bump this.latestSequence in destroy() so any in-flight getRegionalSnapshot whose .then resolves after destroy fails the existing isLatestSequence guard and doesn't try to render into a detached DOM tree. Belt-and-braces: even if a listener fires between destroy and GC, render won't happen. Rewrote the private-field doc comment to reflect the actual teardown path now that we have one. Tests: 6326 pass. Typecheck + lint clean. |
||
|
Elie Habib
|
3e7bf49393 |
refactor(emails): refresh Pro welcome email — surface WM Analyst, Widgets, MCP (#3300)
* refactor(emails): refresh Pro welcome email — surface WM Analyst, Widgets, MCP
The Pro welcome email rendered a stale 4-card 2×2 grid (Near-Real-Time,
AI Analyst, Multi-Channel Alerts, "10 Dashboards") that missed three of
the release's signature differentiators and carried a wrong stat ("10
Dashboards" — the actual Pro surface is 50+ panels per the pricing
table). Net effect: paying users got a welcome email that undersold what
they bought and didn't point at the highest-retention action (the brief).
This commit rewrites the Pro path of userWelcomeHtml + featureCardsHtml
to the "signature-first" 2×3 grid designed via the playground at
docs/plans/pro-welcome-email-playground.html:
WM Analyst 🤖 Create Custom Widgets 🧩
MCP Integration 🔌 Daily AI Brief ☀️
Multi-Channel 50+ Pro Panels 📐
Delivery 📬
Other shifts on the Pro path:
- Headline: "Welcome to {planName}!" → "Welcome to {planName} — your
intel, delivered." — parameterized so pro_annual/pro_monthly both read
correctly.
- CTA: "Open Dashboard" (→ /) → "Open My Brief" (→ /brief). The brief
is the single highest-retention action for a new Pro.
- New "Invite your team" referral block above the CTA.
- New support contact line under the CTA, pointing at ADMIN_EMAIL so
replies route correctly.
API-plan path (api_starter/api_starter_annual/api_business/enterprise)
preserved byte-for-byte — same 4 cards, same "Welcome to {planName}!"
headline, same "Open Dashboard" CTA, no referral, no support line. A
follow-up will refresh that variant with its own MCP / Webhooks / API
lead after this ships.
Size: 6.4 KB rendered (was ~4 KB; Gmail clips at ~102 KB).
Playground (for future refreshes): docs/plans/pro-welcome-email-playground.html
* fix(emails): allowlist Pro plans + drop referral block pending Phase 9
Addresses two P1 review comments from @greptile-apps on #3300.
1. `isPro` was a deny-list (`!API_PLANS.has(planKey)`) — every plan key
outside API_PLANS fell into the Pro branch, including `free` (already
in PLAN_DISPLAY) and any future tier (e.g. `pro_team`) added to
PLAN_DISPLAY without a matching API_PLANS update. Switched to an
explicit allowlist: `PRO_PLANS = Set(['pro_monthly','pro_annual'])`.
Anything outside falls back to the neutral "Welcome to {planName}!"
shell + "Open Dashboard" CTA.
2. The "Invite your team — earn a month free" referral block linked to
`https://worldmonitor.app/referrals`, but that page isn't mounted and
`src/services/referral.ts` is still flagged "Phase 9 / Todo #223".
Shipping the block today would send paying users to a 404 and promise
a credit the backend can't grant. Removed the block entirely;
reinstate in a follow-up PR once the referral endpoint + credit logic
ship.
Playground (`docs/plans/pro-welcome-email-playground.html`) remains the
source of truth for future refreshes.
* fix(emails): align card gate with shell gate + wire reply_to for support line
Addresses two more review findings on #3300.
1. `featureCardsHtml()` still branched on `API_PLANS.has(planKey)`, so
every non-API plan (including `free` and any future tier added to
PLAN_DISPLAY without a matching PRO_PLANS update) got the 6-card Pro
marketing grid even though the shell gate (`userWelcomeHtml`) now
allowlists Pro correctly. Result: `free` or unknown-tier users saw
"Welcome to Free!" + "Open Dashboard" but still received "WM Analyst",
"Create Custom Widgets", "MCP Integration", etc. card content.
Fixed by parallel-allowlisting on `!PRO_PLANS.has(planKey)` — falls
through to the 4-card generic grid (same cards the API variant shows)
for anyone who isn't on pro_monthly / pro_annual. The generic grid is
safe for unknown tiers: no Pro-only claims, no specific promises.
2. The Pro support line reads "Questions? Reply to this email or ping
elie@worldmonitor.app" but FROM is `noreply@worldmonitor.app` and the
Resend payload omitted reply_to — replies would bounce.
Added an optional `replyTo` param to `sendEmail()` and thread
ADMIN_EMAIL through on the user welcome call. Gmail and every other
major client honour Reply-To over From when both are present.
The admin notification intentionally passes no replyTo to avoid
routing replies back to ADMIN_EMAIL (self-loop).
|
||
|
Elie Habib
|
b8615dd106 |
fix(intelligence): read regional-snapshot latestKey as raw string (#3302)
* fix(intelligence): read regional-snapshot latestKey as raw string
Regional Intelligence panel rendered "No snapshot available yet" for every
region despite the 6h cron writing per-region snapshots successfully. Root
cause: writer/reader encoding mismatch.
Writer (scripts/regional-snapshot/persist-snapshot.mjs:60) stores the
snapshot_id pointer via `['SET', latestKey, snapshotId]` — a BARE string,
not JSON.stringify'd. The seeder's own reader (line 97) reads it as-is
and works.
Vercel RPC handler used `getCachedJson(latestKey, true)`, which internally
does `JSON.parse(data.result)`. `JSON.parse('mena-20260421T000000-steady')`
throws; the try/catch silently returns null; handler returns {}; panel
renders empty.
Fix: new `getCachedRawString()` helper in server/_shared/redis.ts that
reads a Redis key as-is with no JSON.parse. Handler uses it for the
latestKey read (while still using getCachedJson for the snapshot-by-id
payload, which IS JSON.stringify'd). No writer or backfill change needed.
Regression guard: new structural test asserts the handler reads latestKey
specifically via getCachedRawString so a future refactor can't silently
revert to getCachedJson and re-break every region.
Health.js monitors the summary key (intelligence:regional-snapshots:
summary:v1), which stays green because summary writes succeed. Per-region
health probes would be worth adding as a follow-up.
* fix(redis): detect AbortSignal.timeout() as TimeoutError too
Greptile P2 on PR #3302: AbortSignal.timeout() throws a DOMException with
name='TimeoutError' on V8 runtimes (Vercel Edge included). The existing
isTimeout check only matched name==='AbortError' — what you'd get from a
manual controller.abort() — so the [REDIS-TIMEOUT] structured log never
fired. Every redis fetch timeout silently fell through to the generic
console.warn branch, eroding the Sentry drain we added specifically to
catch these per docs/plans/chokepoint-rpc-payload-split.md.
Fix both getCachedJson (pre-existing) and the new getCachedRawString by
matching TimeoutError OR AbortError — covers both the current
AbortSignal.timeout() path and any future switch to manual AbortController.
Pre-existing copy in getCachedJson fixed in the same edit since it's the
same file and the same observability hole.
* test(redis): update isTimeout regex to match new TimeoutError|AbortError check
Pre-push hook caught the brittle static-analysis test in
tests/get-chokepoint-history.test.mjs:83 that asserted the exact old
single-name pattern. Update the regex (and description) to cover both
TimeoutError and AbortError, matching the observability fix in the
previous commit.
|
||
|
Elie Habib
|
32049c07ca | feat(portwatch): H+F — cache by upstream maxDate + parallel window split (#3299) | ||
|
Elie Habib
|
12365129c0 |
fix(pro-marketing): hotfix — Dodo auto-redirect on success (paid users stuck on /pro) (#3298)
* fix(pro-marketing): hotfix — let Dodo auto-redirect on success (manualRedirect=false) Production bug: user paid via /pro overlay, payment succeeded (wallet-return?status=succeeded&payment_id=...) but the parent window stayed at /pro#pricing with spinner visible. No success banner, no dashboard redirect. Root cause: with `manualRedirect: true`, the Dodo SDK defers redirect responsibility to the parent's onEvent handler waiting on `checkout.status === 'succeeded'`. In SDK 0.109.2 the iframe appears to navigate internally to its `wallet-return` page (two 404s on `/api/checkout/sessions/*/payment-link` visible in the user's console) without reliably postMessaging the success event to the parent — so our handler never fires, onSuccess never runs. Fix: set `manualRedirect: false` and include `?wm_checkout=success` in the returnUrl sent to /api/create-checkout. Dodo's SDK then performs the parent-window redirect itself via its built-in `checkout.redirect` → `window.location.href = redirect_to` path, landing on https://worldmonitor.app/?wm_checkout=success. The existing dashboard bridge (src/services/checkout-return.ts) already handles that marker and shows the success banner. Also: add `console.info('[checkout] dodo event', ...)` to every onEvent callback so a future regression in Dodo's event flow is diagnosable from Sentry breadcrumbs instead of a user-reported "stuck on spinner." ## Post-Deploy Monitoring & Validation - Logs: filter Sentry breadcrumbs for `[checkout] dodo event` — should see the full event sequence on every checkout completion. - Validation: complete a test live-mode purchase at /pro, confirm the parent window auto-navigates to worldmonitor.app/?wm_checkout=success and the success banner renders. - Failure signal: any user report of "stuck on /pro after paying" within 24h. Rollback trigger: same report. * fix(pro-marketing): rebuild bundle on Node 22 + address PR review - Rebuild public/pro/ with Node 22 (matches CI) — my Node 24 build produced different content-hashes, breaking the Pro bundle freshness check. - Greptile P2: scrub onEvent breadcrumb — only log event_type + status, not raw event.data (can contain PII: email, billing address, payment_id) which would leak via Sentry's console integration. - Greptile P2: add comment noting onSuccess is best-effort — with manualRedirect=false the SDK's `checkout.redirect` can race our `checkout.status=succeeded` handler. Authoritative success path is the `?wm_checkout=success` bridge. |
||
|
Elie Habib
|
a17a3383d9 |
feat(variant): Energy Atlas — Release 1 Day 1 (variant scaffolding) (#3291)
* feat(variant): add energy variant scaffolding for energy.worldmonitor.app Release 1 Day 1 of the Energy Atlas plan — introduces src/config/variants/energy.ts modeled on the commodity variant. No new panels or RPCs yet; the variant reuses existing energy-related panels (energy-complex, oil-inventories, hormuz, energy-crisis, fuel-prices, renewable-energy) + supply-chain/sanctions context. Map layers enable pipelines, waterways, AIS, commodityPorts, minerals, climate, outages, natural, weather. All geopolitical/military/tech/finance/happy variant layers explicitly disabled per variant isolation conventions. Next PRs on feat/energy-atlas-release-1 add: - Pipeline & storage registries (curated critical assets, ~75 gas / ~75 oil / ~125 storage) - Global fuel-shortage registry with automated evidence-threshold promotion - Pipeline/storage disruption event log - Country drill-down Energy section - Atlas landing composition at variant root * feat(variant): wire energy variant into runtime + atlas landing composition Day 2 of the Energy Atlas Release 1 plan. The Day-1 commit added a canonical variants/energy.ts but discovery during Day 2 showed the app's runtime variant resolution lives in src/config/panels.ts (ENERGY_PANELS/ENERGY_MAP_LAYERS/etc.), not in variants/*.ts (which are orphans). This commit does the real wiring. Changes: - src/config/panels.ts — ENERGY_PANELS, ENERGY_MAP_LAYERS, ENERGY_MOBILE_MAP_LAYERS; registered in ALL_PANELS, VARIANT_DEFAULTS, VARIANT_PANEL_OVERRIDES; wired into DEFAULT_MAP_LAYERS + MOBILE_DEFAULT_MAP_LAYERS ternaries. Panels at launch: map, live-news, insights, energy-complex, oil-inventories, hormuz, energy-crisis, fuel-prices, renewable-energy, commodities, energy (news), macro-signals, supply-chain, sanctions-pressure, gulf-economies, gcc-investments, climate, monitors, world-clock, latest-brief. - src/config/variant.ts — recognize 'energy' as allowed SITE_VARIANT; resolve energy.worldmonitor.app subdomain to 'energy'; honor localStorage override. - src/config/variant-meta.ts — SEO entry for energy.worldmonitor.app (title, description, keywords targeting 'oil pipeline tracker', 'gas storage map', 'fuel shortage tracker', 'chokepoint monitor', etc.). - src/app/panel-layout.ts — desktop variant switcher + mobile menu both list energy with ⚡ icon and t('header.energy') label. - src/App.ts + src/app/data-loader.ts — energy variant enables trade-policy and supply-chain data loads (chokepoint exposure is a core Atlas surface). - src/app/data-loader.ts — daily-brief newsCategories override for energy variant (energy, energy-markets, oil-gas-news, pipeline-news, lng-news). - src/locales/en.json — 'header.energy' translation key. - src/config/variants/energy.ts — add clarifying comment that real wiring lives in panels.ts (same orphan pattern as commodity.ts/finance.ts/etc.). Atlas landing composition: the variant now renders its energy panel set with energy-specific names (Energy Atlas Map, Energy Headlines, AI Energy Insights) when SITE_VARIANT === 'energy'. Pipeline and commodity-port map layers enabled so Week 2's pipeline registry + storage-facility registry drop in with layers already toggled on. Typecheck clean; 175 pre-push tests expected to remain green. Subsequent PRs on feat/energy-atlas-release-1: - Week 2: pipeline registry + storage facility registry (evidence-based) - Week 3: fuel-shortage classifier + disruption log + country drill-down - Week 4: automated revision log, SEO polish, launch * feat(energy): chokepoint strip at top of atlas (7 chokepoints) Day 3 of the Energy Atlas Release 1 plan. Adds ChokepointStripPanel — a compact horizontal strip of chip-style cards, one per chokepoint, showing name + status color + flow-as-%-of-baseline + active-warnings badge. Ordered by volume: Hormuz, Malacca, Suez, Bab el-Mandeb, Turkish Straits, Danish Straits, Panama. GEF covers 5 chokepoints (Hormuz, Malacca, Suez, Bab el-Mandeb, Panama). We cover 7 — adds Turkish Straits + Danish Straits. One of the surpass vectors enumerated in §5.7 of the plan doc. Data: reuses the existing fetchChokepointStatus() RPC backed by supply_chain:chokepoints:v4 (Portwatch DWT + AIS calibration). No new backend work; this is pure UI composition. Changes: - src/components/ChokepointStripPanel.ts — new Panel subclass with in-line CSS for the chip strip; falls back gracefully when a chokepoint is missing from the response or FlowEstimate is absent. - src/components/index.ts — barrel export. - src/app/panel-layout.ts — import + createPanel registration near existing energy panels. - src/config/panels.ts — ENERGY_PANELS adds 'chokepoint-strip' at priority 1 (renders near top of atlas). Also fixes two panel-ID mismatches caught while wiring: 'hormuz' → 'hormuz-tracker' and 'renewable-energy' → 'renewable' (matches HormuzPanel.ts and RenewableEnergyPanel registration). Typecheck clean. No new tests required — panel renders real data. * feat(energy): attribution footer utility + methodology page stubs Days 4 & 5 of the Energy Atlas Release 1 plan. ## Day 4 — Attribution footer (src/utils/attribution-footer.ts) A reusable string-builder that stamps every energy-atlas number with its provenance. Design intent per plan §5.6 (quantitative rigour moat): "every flow number carries {value, baseline, n_vessels, methodology, confidence}". Input schema: - sourceType: operator | regulator | ais | satellite | press | classifier | derived - method: short free-text ("AIS-DWT calibrated", "GIE AGSI+ daily") - sampleSize + sampleLabel: observation count and unit - updatedAt: ISO8601 / Date / number — rendered as "Xm/h/d ago" - confidence: 0..1 — bucketed to high/medium/low - classifierVersion: surfaced when evidence-derived badges ship in Week 2+ - creditName / creditUrl: CC-BY / dataset credit (OWID, GEM pattern) Every field also writes data-attributes (data-attr-source, data-attr-n, data-attr-confidence, data-attr-classifier) so MCP / scraper / analyst agents can extract the same provenance the reader sees. Agent-native by default. Applied to ChokepointStripPanel — the panel now shows its evidence footer ("AIS calibration · Portwatch DWT + AIS · N AIS disruption signals · updated Xh ago · EIA World Oil Transit Chokepoints"). Future pipeline / storage / shortage panels drop the same helper in and hit the same rigour bar automatically. 7 unit tests (tests/attribution-footer.test.mts, node:test via tsx): minimal footer, method + sample size + credit, "X ago" formatting, confidence band mapping, full data-attribute emission, credit omission, HTML escaping. ## Day 5 — Public methodology page stubs (docs/methodology/) Four new MDX pages surfaced in docs/docs.json navigation under "Intelligence & Analysis": - chokepoints.mdx — 7 chokepoints, Portwatch+AIS calibration method, status badge derivation, known limits, revision-log link. - pipelines.mdx — curated critical-asset scope, GEM CC-BY attribution, evidence-schema (NOT conclusion labels), freshness SLA, corrections. - storage.mdx — curated ~125 facilities scope, "published not synthesized" fill % policy, country-aggregate fallback, attribution. - shortages.mdx — automated tiered evidence threshold, LLM second-pass gating, auto-decay cadence, evidence-source transparency, break-glass override policy (admin-only, off critical path). All four explicitly document WorldMonitor's automated-data-quality posture: no human review queues, quality via classifier rigour + evidence transparency + auto-decay + public revision log. Typecheck clean. attribution-footer.test.mts passes all 7 tests. * fix(variant): close three energy-variant isolation leaks from review Addresses three High findings from PR review: 1. Map-layer isolation (src/config/map-layer-definitions.ts) - Add 'energy' to the MapVariant type union. - Add energy entry to VARIANT_LAYER_ORDER with the curated energy subset (pipelines, waterways, commodityPorts, commodityHubs, ais, tradeRoutes, minerals, sanctions, fires, climate, weather, outages, natural, resilienceScore, dayNight). Without this, getLayersForVariant() and sanitizeLayersForVariant() (called from DeckGLMap and App.ts) fell back to VARIANT_LAYER_ORDER.full, letting the full geopolitical palette (military, nuclear, iranAttacks, conflicts, hotspots, bases, protests, flights, ucdpEvents, displacement, gpsJamming, satellites, ciiChoropleth, cables, datacenters, economic, cyberThreats, spaceports, irradiators, radiationWatch) into the desktop map tray and saved/URL layer sanitization — breaking the PR's stated no-geopolitical-bleed goal and violating multi-variant-site-data-isolation. 2. News feeds (src/config/feeds.ts + src/app/data-loader.ts) - Add ENERGY_FEEDS with three keys matching ENERGY_PANELS: live-news (broad energy headlines from OilPrice, Rigzone, Reuters/Bloomberg/FT energy), energy (OPEC + crude + NatGas/LNG + pipelines/chokepoints + crisis/shortages + refineries), supply-chain (tanker/shipping, chokepoints, energy sanctions, ports/terminals). - Add SITE_VARIANT === 'energy' branch to the FEEDS variant selector. - Correct newsCategories override in data-loader.ts — my earlier speculative values ['energy','energy-markets','oil-gas-news', 'pipeline-news','lng-news'] included keys that did not exist in any feed map. Replaced with real ENERGY_FEEDS keys ['live-news', 'energy', 'supply-chain']. Without this, FEEDS resolved to FULL_FEEDS for the energy variant — live-news + daily-brief both ingested the world/geopolitical feed set. 3. Insights / AI brief framing (src/components/InsightsPanel.ts) - Add SITE_VARIANT === 'energy' branch to geoContext: dedicated energy prompt focused on physical supply (pipelines, chokepoints, storage, days-of-cover, refineries, LNG, sanctions, shortages) with evidence-grounded attribution, no bare conclusions. - Add '⚡ ENERGY BRIEF' heading branch in renderWorldBrief(). Without this, the renamed 'AI Energy Insights' panel fell through to the empty default prompt and rendered 'WORLD BRIEF'. Typecheck clean. attribution-footer tests still pass (no coupling changed). * fix(variant): close energy-variant leak in SVG/mobile fallback map Fifth High finding from PR review: src/components/Map.ts createLayerToggles() (line 381-409) has no 'energy' branch in its variant ternary, so energy-variant users whose MapContainer routes to the SVG/mobile fallback (no WebGL, mobile with deviceMemory < 3, or DeckGL init throws) see the full geopolitical toggle set — iranAttacks, conflicts, hotspots, bases, nuclear, irradiators, military, protests, flights, gpsJamming, ciiChoropleth, cables, datacenters. Clicking any toggle flips the layer via toggleLayer() which is variant-blind (Map.ts:3383) — so users could enable military / nuclear layers on the energy variant despite the rest of the isolation work in panels.ts, map-layer-definitions.ts, feeds.ts, and InsightsPanel.ts. Fix: add energyLayers array with the SVG-capable subset of ENERGY_MAP_LAYERS — pipelines, waterways, ais, commodityHubs, minerals, sanctions, outages, natural, weather, fires, economic. Intentionally omitted: commodityPorts, climate, tradeRoutes, resilienceScore, dayNight — none of these have render handlers in Map.ts's SVG path, so including them would create toggles that do nothing. Extended the ternary with 'energy' → energyLayers between 'happy' and the 'full' fallback. Note (preexisting, NOT fixed here): the same ternary has no 'commodity' branch either, so commodity.worldmonitor.app also gets the full geopolitical toggle set on the SVG fallback. Out of scope for this PR; flagged for a separate fix. Defence-in-depth: sanitizeLayersForVariant() (now fixed in map-layer-definitions.ts) strips saved-URL layers to the energy subset before the SVG map sees them, so even if a user arrives with ?layers=military in the URL, it's gone by the time initialState reaches MapComponent. The toggle-list fix closes the UI-path leak; the sanitize fix closes the URL-path leak. Typecheck clean. |
||
|
Elie Habib
|
8172ea5660 |
feat(auth): expose Sign Up CTA + adjacent Settings button + signup analytics (#3258)
* feat(auth): expose Sign Up CTA + adjacent Settings button + signup analytics
The header auth widget had three latent gaps that this PR closes in one
cohesive change — all touch the same ~30 lines of widget code and the
same Clerk service module:
1. No explicit Sign Up entry point anywhere in the app. Clerk's sign-up
modal was only reachable via the footer link inside the sign-in
modal. Add `openSignUp()` to `clerk.ts` + `AuthLauncher` for symmetry
with `openSignIn`, and render a secondary "Create account" CTA in
the signed-out header.
2. `AuthHeaderWidget` constructor accepted `_onSignInClick` /
`_onSettingsClick` callbacks but the underscore prefix was literal —
the callbacks were never invoked. `event-handlers.ts:1121` passed
`modal.open()` and `unifiedSettings?.open()` that went nowhere. The
signed-in state now renders the Clerk UserButton + an adjacent
Settings button (gear icon) wired to the previously-dead callback,
so `UserButton → Settings → Manage Billing` is 2 clicks from the
avatar area. Sign-in path also now honors `onSignInClick` when
provided, falling back to direct `openSignIn()`.
3. `trackSignUp` was exported but never called. Wiring it to a button
click would conflate "opened the modal" with "completed sign-up".
Instead, detect the genuine null→non-null user-id transition inside
`initAuthAnalytics` and gate on `user.createdAt` being within a 60s
fresh-signup window. Factored the predicate into
`isLikelyFreshSignup()` for unit testing.
Changes:
- `src/services/clerk.ts` — add `openSignUp()` + `getClerkUserCreatedAt()`.
- `src/components/AuthHeaderWidget.ts` — consume callbacks; render
Create Account link (signed-out) and Settings button (signed-in).
- `src/components/AuthLauncher.ts` — add `.openSignUp()` method.
- `src/services/analytics.ts` — detect fresh signup inside auth-state
listener; export `isLikelyFreshSignup` + `FRESH_SIGNUP_WINDOW_MS`.
- `src/locales/en.json` — new `auth.signIn`, `auth.createAccount`,
`auth.settings` keys.
- `src/styles/main.css` — `.auth-signup-link` and `.auth-settings-btn`.
- `tests/signup-analytics-gate.test.mts` — 8 unit tests for the
fresh-signup predicate including boundary, clock-skew, and sign-in
vs sign-up discrimination.
PR-1 of the 14-PR rollout at
docs/plans/2026-04-21-002-feat-harden-auth-checkout-flow-ux-plan.md.
Pre-merge: verify sign-up is enabled in the Clerk dashboard (if not,
`openSignUp` is a no-op but does not crash).
Typecheck + lint clean. test:data 6036/6036 passing.
* fix(analytics): bound clock-skew tolerance + fix contradictory test
isLikelyFreshSignup previously accepted ANY future createdAt as fresh —
a 10-minute-future timestamp (buggy client clock, spoofed user input)
would fire trackSignUp('clerk'). Now tolerates up to 5s of forward skew
(real server/client drift) and rejects anything further out as malformed.
Test 'returns false when createdAt is in the future (clock skew guard)'
actually asserted true, making the name lie about the behaviour. Split
into two tests with accurate names: one for tiny accepted skew, one for
unrealistic rejected skew.
* fix(analytics): persist signup-tracked marker in sessionStorage
_lastAuth resets to null on every page load, so the null→userId transition
looks identical on the actual signup-completion reload AND on any tab
reload within the 60s createdAt freshness window. Without a durable
marker, trackSignUp fired on every reload until createdAt aged out —
inflating the signup metric.
Add sessionStorage-keyed per-user fire-once guard (`wm-signup-tracked:<uid>`).
Tab-scoped so a fresh session correctly re-counts, and keyed per user so
account switches within the same tab still register if the new user just
signed up. Fails open on storage-unavailable (private mode / quota).
* fix(analytics): promote signup-tracked marker to localStorage (cross-tab)
Reviewer caught a per-tab scope hole: sessionStorage is per-tab, so a
user who signs up in tab A and opens the app in tab B within the 60s
createdAt window would fire a second trackSignUp from B's fresh
sessionStorage. Promote to localStorage (shared across all tabs in the
same browser profile) — once any tab marks the user as tracked, no
other tab for the same user re-fires.
Clerk user ids are effectively unique forever, so no cleanup is needed
and the cross-tab scope is safe (no risk of id reuse).
|
||
|
Elie Habib
|
52659ce192 |
feat(resilience): PR 1 — energy construct repair (flag-gated) (#3289)
* docs(resilience): PR 1 foundation — Option B framing + v2 energy construct spec
First commit in PR 1 of the resilience repair plan. Zero scoring-behaviour
change; sets up the construct contract that the code changes will implement.
Declares the framing decision required by plan section 3.2 before any
scorer code lands: Option B (power-system security) is adopted. Electricity
grids are the dominant short-horizon shock-transmission channel, and the
choice lets the v2 energy indicator set share one denominator (percent of
electricity generation) instead of mixing primary-energy and power-system
measures in a composite.
Methodology doc changes:
- Energy Domain section now documents both the legacy indicator set
(still the default) and the v2 indicator set (flag-gated), under a
single #### Energy H4 heading so the methodology-doc linter still
asserts dimension-id parity with the registry.
- v2 indicators: importedFossilDependence (EG.ELC.FOSL.ZS x
max(EG.IMP.CONS.ZS, 0)), lowCarbonGenerationShare (EG.ELC.NUCL.ZS +
EG.ELC.RNEW.ZS), powerLossesPct (EG.ELC.LOSS.ZS), reserveMarginPct
(IEA), euGasStorageStress (renamed + scoped to EU), energyPriceStress
(retained at 0.15 weight).
- Retired under v2: electricityConsumption, gasShare, coalShare,
dependency (all into importedFossilDependence), renewShare.
- electricityAccess moves from energy to infrastructure under v2.
- Added a v2.1 changelog section documenting the flag-gated rollout,
acceptance gates (per plan section 6), and snapshot filenames for
the post-flag-flip captures.
- Known-limitations items 1-3 updated to note PR 1 lands the v2
construct behind RESILIENCE_ENERGY_V2_ENABLED (default off).
Methodology-doc linter + mdx-lint + typecheck all clean. Indicator
registry, seeders, and scorer rewrite land in subsequent commits on
this same branch.
* feat(resilience): PR 1 — RESILIENCE_ENERGY_V2_ENABLED flag + scoreEnergy v2 + registry entries
Second commit in PR 1 of the resilience repair plan. Lands the flag,
the v2 scorer code path, and the registry entries the methodology
doc referenced. Default is flag off; published rankings are unchanged
until the flag flips in a later commit (after seeders land and the
acceptance-gate rerun produces a fresh post-flip snapshot).
Changes:
- _shared.ts: isEnergyV2Enabled() function reader on the canonical
RESILIENCE_ENERGY_V2_ENABLED env var. Dynamic read (like
isPillarCombineEnabled) so tests can flip per-case.
- _dimension-scorers.ts:
- New Redis key constants for the three v2 seed keys plus the
reserved reserveMargin key (seeder deferred per plan §3.1
open-question).
- EU_GAS_STORAGE_COUNTRIES set (EU + EFTA + UK) for the renamed
euGasStorageStress signal per plan §3.5 point 2.
- isEnergyV2EnabledLocal() — private duplicate of the flag reader
to avoid a circular import (_shared.ts already imports from
this module). Same env-var contract.
- scoreEnergy split into scoreEnergyLegacy() + scoreEnergyV2().
Public scoreEnergy() branches on the flag. Legacy path is
byte-identical to the pre-commit behaviour.
- scoreEnergyV2() reads four new bulk payloads, composes
importedFossilDependence = fossilElectricityShare × max(netImports, 0)/100
per plan §3.2, collapses net exporters to 0, and gates
euGasStorageStress on EU membership so non-EU countries
re-normalise rather than getting penalised for a regional
signal.
- _indicator-registry.ts: four new entries under `dimension: 'energy'`
with `tier: 'experimental'` — importedFossilDependence (0.35),
lowCarbonGenerationShare (0.20), powerLossesPct (0.10),
reserveMarginPct (0.10). Experimental tier keeps them out of the
Core coverage gate until seed coverage is confirmed.
- compare-resilience-current-vs-proposed.mjs: new
'bulk-v1-country-value' shape family in the extraction dispatcher.
EXTRACTION_RULES now covers the four v2 registry indicators so
the per-indicator influence harness tracks them from day one.
When the seeders are absent, pairedSampleSize = 0 and Pearson = 0
— the harness output surfaces the "no influence yet" state rather
than silently dropping the indicators.
- tests/resilience-energy-v2.test.mts: 11 new tests pinning:
- flag-off = legacy behaviour preserved (v2 seed keys have no
effect when flag is off — catches accidental cross-path reads)
- flag-on = v2 composite behaves correctly:
- lower fossilElectricityShare raises score
- net exporter with 90% fossil > net importer with 90% fossil
(max(·, 0) collapse verified)
- higher lowCarbonGenerationShare raises score (nuclear credit)
- higher powerLossesPct lowers score
- euGasStorageStress is invariant for non-EU, responds for DE
- all v2 inputs absent = graceful degradation, coverage < 1.0
106 resilience tests pass (existing + 11 new). Typecheck clean. Biome
clean. No production behaviour change with flag off (default).
Next commits on this branch: three World Bank seeders for the v2 keys,
health.js + SEED_META registration (gated ON_DEMAND_KEYS until Railway
cron provisions), acceptance-gate rerun at flag-flip time.
* feat(resilience): PR 1 — three WB seeders + health registration for v2 energy construct
Third commit in PR 1. Lands the seed scripts for the three v2 energy
indicator source keys, registered in api/health.js with ON_DEMAND_KEYS
gating until Railway cron provisions.
New seeders (weekly cron cadence, 8d maxStaleMin = 2x interval):
- scripts/seed-low-carbon-generation.mjs
Pulls EG.ELC.NUCL.ZS + EG.ELC.RNEW.ZS from World Bank, sums per
country into `resilience:low-carbon-generation:v1`. Partial
coverage (one series missing) still emits a value using the
observed half — the scorer's 0-80 saturating goalpost tolerates
it and the underlying construct is "firm low-carbon share".
- scripts/seed-fossil-electricity-share.mjs
Pulls EG.ELC.FOSL.ZS into `resilience:fossil-electricity-share:v1`.
Feeds the importedFossilDependence composite at score time
(composite = fossilShare × max(netImports, 0) / 100 per plan §3.2).
- scripts/seed-power-reliability.mjs
Pulls EG.ELC.LOSS.ZS into `resilience:power-losses:v1`. Direct
grid-integrity signal replacing the retired electricityConsumption
wealth proxy.
All three follow the existing seed-recovery-*.mjs template:
- Shape: { countries: { [ISO2]: { value, year } }, seededAt }
- runSeed() from _seed-utils.mjs with schemaVersion=1, ttl=35d
- validateFn floor of 150 countries (WB coverage is 150-180 for
the three indicators; below 150 = transient fetch failure)
- ISO3 → ISO2 mapping via scripts/shared/iso3-to-iso2.json
No reserveMargin seeder is shipped in this commit per plan §3.1 open
question: IEA electricity-balance coverage is sparse outside OECD+G20,
and the indicator will likely ship as 'unmonitored' with weight 0.05
if it lands at all. The Redis key (`resilience:reserve-margin:v1`) is
reserved in _dimension-scorers.ts so the v2 scorer shape is stable.
api/health.js:
- SEED_DOMAINS: add `lowCarbonGeneration`, `fossilElectricityShare`,
`powerLosses` → their Redis keys.
- SEED_META: same three, pointing at `seed-meta:resilience:*` meta
keys with maxStaleMin=11520 (8d, per the worldmonitor
health-maxstalemin-write-cadence pattern: 2x weekly cron).
- ON_DEMAND_KEYS: three new entries gated as TRANSITIONAL until
Railway cron provisions and the first clean run completes. Remove
from this set after ~7 days of green production runs.
Typecheck clean; existing 106 resilience tests pass (seeders have no
in-repo callers yet, so nothing depends on them executing). Real-API
integration tests land when Railway cron is provisioned.
Next commit: Railway cron configuration + bundle-runner wiring.
* feat(resilience): PR 1 — bundle-runner + acceptance-gate verdict + flag-flip runbook
Final commit in the PR 1 tranche. Lands the three remaining pieces so
the flag-flip is fully operable once Railway cron provisions.
- scripts/seed-bundle-resilience-energy-v2.mjs
Railway cron bundle wrapping the three v2 energy seeders
(low-carbon-generation, fossil-electricity-share, power-losses).
Weekly cadence (7-day intervalMs); the underlying data is annual
at source so polling more frequently just hammers the World Bank
API. 5-minute per-script timeout. Mirrors the existing
seed-bundle-resilience-recovery.mjs pattern.
- scripts/compare-resilience-current-vs-proposed.mjs: acceptanceGates
block. Programmatic evaluation of plan §6 gates using the inputs
the harness already computes:
gate-1-spearman Spearman vs baseline >= 0.85
gate-2-country-drift Max country drift vs baseline <= 15
gate-6-cohort-median Cohort median shift vs baseline <= 10
gate-7-matched-pair Every pair holds expected direction
gate-9-effective-influence >= 80% Core indicators measurable
gate-universe-integrity No cohort/pair endpoint missing from scorable
Thresholds are encoded in a const so they can't silently soften.
Output verdict is PASS / CONDITIONAL / BLOCK. Emitted in
summary.acceptanceVerdict for at-a-glance PR comment pasting, with
full per-gate detail in acceptanceGates.results.
- docs/methodology/energy-v2-flag-flip-runbook.md
Operator runbook for the flag flip. Pre-flip checklist (seeders
green, health endpoint green, ON_DEMAND_KEYS graduation, Spearman
verification), flip procedure (pre-flip snapshot, dry-run, cache
prefix bump, Vercel env flip, post-flip snapshot, methodology
doc reclassification), rollback procedure, and a reference table
for the three possible verdict states.
PR 1 is now code-complete pending:
1. Railway cron provisioning (ops, not code)
2. Flag flip + acceptance-gate rerun (follows runbook, not code)
3. Reserve-margin seeder (deferred per plan §3.1 open-question)
Zero scoring-behaviour change in this commit. 121 resilience tests
pass, typecheck clean.
* fix(resilience): PR 1 — drop unseeded reserveMargin from scorer + fix composite extractor
Addresses two P1 review findings on PR #3289.
Finding 1: scoreEnergyV2 read resilience:reserve-margin:v1 at weight
0.10 but no seeder ships in this PR (indicator deferred per plan
§3.1 open-question). On flag flip that slot would be permanently
null, silently renormalizing the remaining 90% of weight and
producing a construct different from what the methodology doc
describes. Fix: remove reserve-margin from the v2 reader +
blend entirely. Redistribute its 0.10 weight to powerLossesPct
(now 0.20); both are grid-integrity signals per plan §3.1, and
the original plan split electricityConsumption's 0.30 weight
across powerLossesPct + reserveMarginPct + importedFossilDependence
— without reserveMarginPct, powerLossesPct carries the shared
grid-integrity load until the IEA seeder ships.
v2 weights now: 0.35 + 0.20 + 0.20 + 0.10 + 0.15 = 1.00
(importedFossilDependence + lowCarbonGenerationShare +
powerLossesPct + euGasStorageStress + energyPriceStress)
Reserve-margin Redis key constant stays reserved so the v2
scorer shape is stable when a future commit lands the seeder;
split 0.10 back out of powerLossesPct at that point.
Methodology doc, _shared.ts flag comment, and v2 test suite all
updated to the 5-indicator shape. New regression test asserts
that changing reserve-margin Redis content has zero effect on
the v2 score — guards against a future commit accidentally
wiring the reader back in without its seeder.
Finding 2: scripts/compare-resilience-current-vs-proposed.mjs
measured importedFossilDependence by reading fossilElectricityShare
alone. The scorer defines it as fossilShare × max(netImports, 0)
/ 100, so the extractor zeroed out net exporters and
under-reported net importers — making gate-9 effective-influence
wrong for the centrepiece construct change of PR 1.
Fix: new 'imported-fossil-dependence-composite' extractor type
in applyExtractionRule that recomputes the same composite from
both inputs (fossilShare bulk payload + staticRecord.iea.
energyImportDependency.value). Stays in lockstep with the
scorer — drift between the two would break gate-9's
interpretation.
New unit tests pin:
- net importer: 80% × max(60, 0) / 100 = 48 ✓
- net exporter: 80% × max(-40, 0) / 100 = 0 ✓
- missing either input → null
64 resilience tests pass; typecheck clean. Flag-off path is
still byte-identical to pre-PR behaviour.
* docs(resilience): PR 1 — align methodology doc with actual shipped indicators and seeders
Addresses P1 review on docs/methodology/country-resilience-index.mdx
lines 29 and 574-575. The doc still described reserveMarginPct as a
shipped v2 indicator and listed seed-net-energy-imports.mjs in the
new-seeders list, neither of which the branch actually ships.
Doc changes to match the code in this branch:
Known-limitations item 1: restated to describe the actual v2
replacement footprint — powerLossesPct at 0.20 (temporarily
absorbing reserveMarginPct's 0.10) plus accessToElectricityPct
moved to infrastructure. reserveMarginPct is named as a deferred
companion with the split-out instructions for when its seeder
lands.
v2.1 changelog (Indicators added): split into "live in PR 1" and
"deferred in PR 1" so the reader can distinguish which entries
match real code. importedFossilDependence's composite formula
now written out and the net-imports source attributed to the
existing resilience:static.iea path (not a new seeder).
v2.1 changelog (New seeders): lists the three actual files that
ship in this branch (seed-low-carbon-generation, seed-fossil-
electricity-share, seed-power-reliability) and explicitly notes
seed-net-energy-imports.mjs is NOT a new seeder — the
EG.IMP.CONS.ZS series is already fetched by seed-resilience-
static.mjs. Adds the bundle-runner reference.
Methodology-doc linter + mdx-lint both pass (125/125). Typecheck
clean. Doc is now the source of truth for what PR 1 actually ships.
* fix(resilience): PR 1 — sync powerLossesPct registry weight with scorer (0.10 → 0.20)
Reviewer-caught mismatch between INDICATOR_REGISTRY and scoreEnergyV2.
The previous commit redistributed the deferred reserveMarginPct's 0.10
weight into powerLossesPct in the SCORER but left the REGISTRY entry
unchanged at 0.10. Two downstream effects:
1. scripts/compare-resilience-current-vs-proposed.mjs copies
`spec.weight` into `nominalWeight` for gate-9 reporting, so
powerLossesPct's nominal influence would be under-reported by
half in every post-flip acceptance run — exactly the harness PR 1
relies on for merge evidence.
2. Methodology doc vs registry vs scorer drift is the pattern the
methodology-doc linter is supposed to catch; it passes here
because the linter only checks dimension-id parity, not weights.
Registry is now the only remaining source of truth to keep in
lockstep with the scorer.
Change:
- `_indicator-registry.ts` powerLossesPct.weight: 0.1 → 0.2
- Inline comment names the deferral and instructs: "when the IEA
electricity-balance seeder lands, split 0.10 back out and restore
reserveMarginPct at 0.10. Keep this field in lockstep with
scoreEnergyV2 ... because the PR 0 compare harness copies
spec.weight into nominalWeight for gate-9 reporting."
Experimental weights per dimension invariant still holds (0.35 + 0.20
+ 0.20 = 0.75 for energy, well under the 1.0 ceiling). 64 resilience
tests pass, typecheck clean.
|
||
|
Elie Habib
|
da0f26a3cf |
feat(resilience): PR 0 diagnostic freeze + fairness-audit harness (no scoring changes) (#3284)
* feat(resilience): PR 0 diagnostic freeze + fairness-audit harness
Lands the before-state and measurement apparatus every subsequent
resilience-scorer PR validates against. Zero scoring changes. Per the
v3 plan at docs/plans/2026-04-22-001-fix-resilience-scorer-structural-
bias-plan.md this is tranche 0 of five.
What lands:
- Construct contract published in the methodology doc: absolute
resilience not development-adjusted, mechanism test for every
indicator, peer-relative views published separately from the core.
- Known construct limitations section: six construct errors scheduled
for PR 1-3 repair with explicit mapping to plan tranches.
- Indicator-source manifest at docs/methodology/indicator-sources.yaml
with source, seriesId, seriesUrl, coveragePct, lastObservedYear,
license, mechanismTestRationale, and a constructStatus classification.
- Pre-repair ranking snapshot at
docs/snapshots/resilience-ranking-live-pre-repair-2026-04-22.json
(217 items + 5 greyedOut, captured 2026-04-22 08:38 UTC at commit
|
||
|
Elie Habib
|
2765b46dad |
fix(pro-marketing): honor catalog CTA labels + same-tab for in-product hrefs (#3268)
Squashed rebase onto current main (which now includes #3263's CheckoutPhase machine + URL-intent resume). After #3263 merged, this branch's conflicts with main were mostly cosmetic comment drift on the phase machine docs (same logic, slightly different wording). Took main's versions for shared checkout.ts / App.tsx changes (already-merged and authoritative). Preserved this PR's unique contributions in PricingSection.tsx: 1. `isInProductHref`: catch relative paths without leading slash (`pricing`, `./dashboard`) — resolve against window.location so same-origin relative links don't get misclassified as external 2. CTA label override: honor per-tier `cta` from the catalog (e.g. "Start Pro", "Subscribe") instead of hardcoded "Get Started" Pro bundle rebuilt fresh. |
||
|
Elie Habib
|
cfedcc3ea3 |
feat(pro-marketing): per-tier loading state on pricing checkout buttons (#3263)
Squashed rebase onto current main (which now includes #3262's interstitial + #3273's duplicate dialog + #3270's referral + #3274's overlay marker). Source-only diff extracted from merge-base; 2 conflicts in pro-test/src/services/checkout.ts resolved additively: 1. doCheckout entry: both interstitial mount (PR-5, from main) and setPhase(creating_checkout, productId) (PR-6) needed; kept both. 2. doCheckout finally: both unmountCheckoutInterstitial (PR-5, main) and setPhase(idle) (PR-6) needed; kept both. Changes: - pro-test/src/services/checkout-intent-url.ts (NEW): pure URL-param intent helpers (parseCheckoutIntentFromSearch, stripCheckoutIntent- FromSearch, buildCheckoutReturnUrl) - pro-test/src/services/checkout.ts: CheckoutPhase state machine, subscribeCheckoutPhase; bind intent to sign-in via afterSignInUrl - pro-test/src/components/PricingSection.tsx: subscribe to phase, per-tier loading + billing toggle lock - pro-test/src/App.tsx: tryResumeCheckoutFromUrl on mount - tests/pro-checkout-intent-url.test.mts: 18-test coverage including 3 reviewer scenario regression guards Pro bundle rebuilt fresh. |
||
|
Elie Habib
|
e9d07949a9 |
feat(pro-marketing): "Opening checkout…" interstitial + 10s safety toast (#3262)
Squashed rebase onto current main (which now includes #3273 and #3270's pro-test changes). Source diff extracted from merge-base `git diff $(git merge-base origin/main HEAD) HEAD -- ':!public/pro/**'` applied cleanly — PR-5's interstitial mount/unmount sits in a different function location than PR-7's duplicate-subscription dialog, so no source conflicts. Changes: - pro-test/src/services/checkout.ts: mountCheckoutInterstitial at doCheckout entry (inside try/finally), auto-unmount on settle, 10s safety toast fallback if SDK lazy-load or network hangs Pro bundle rebuilt fresh against current source (which includes PR-7's duplicate-dialog + PR-9's overlay-success marker + PR-14's referral). |
||
|
Elie Habib
|
6ccf474d0f |
fix(clerk): pin afterSignOutUrl to origin root, not window.location.href (#3266)
`mountUserButton` was passing `window.location.href` as the after-
sign-out destination, which can include stale checkout params
(subscription_id / status / payment_id that handleCheckoutReturn
hasn't cleaned yet at sign-out time) or transient fragments that
shouldn't persist into a signed-out state. A user who signed out
from a post-purchase URL could land on a signed-out page with
their purchase identifiers still in the query string.
Fix: `new URL('/', window.location.origin).toString()` — origin-
root, unambiguous, parameter-free. Identical behavior on Tauri
desktop (WKWebView resolves the absolute URL correctly).
PR-12 of the 14-PR rollout at
docs/plans/2026-04-21-002-feat-harden-auth-checkout-flow-ux-plan.md.
Typecheck + scoped lint clean. test:data 6049/6049 passing.
|
||
|
Elie Habib
|
156cc4b86b |
refactor(checkout): rollout follow-up — P1 correctness + cleanup (#3276)
Squashed rebase onto current main (which now includes #3259, #3260, #3261, #3270, #3273, #3274, #3265). PR-3276 was originally written against PR-11's tip WITHOUT PR-14 (#3270) in its ancestry; now that #3270 is merged, this rebase reconciles PR-3276's refactors with the referral + nested-event-shape fixes that landed in main independently. Conflicts resolved (5 regions in src/services/checkout.ts): 1. event shape read: kept main's nested event.data.message.status check + renamed _successFired → successFired (PR-3276's closure refactor) 2. startCheckout session reset: applied PR-3276's _resetOverlaySession hook AND kept main's effectiveReferral / loadActiveReferral from #3270 3. already-entitled banner branch: kept main's auto-dismiss fix (from #3261/#3265). PR-3276 was written without this fix; not regressing it. Used PR-3276's inlined isEntitled() check (computeInitialBannerState deletion per its P1 #251). 4+5. banner timeout / active branches: kept main's stopEmailWatchers + currentState + currentMaskedEmail AND applied PR-3276's _currentBannerCleanup = null cleanup hook (per its P1 #254 re-mount leak fix) Also removed stale `origin: 'dashboard'` field from saveCheckoutAttempt call (PR-3276 deleted that field from CheckoutAttempt interface per its P1 #251 — dead write-only field). Net refactor delivers all of PR-3276's intended fixes: - #247 Module-scoped _successFired → per-session closure - #249 #3163 hard-dep code markers - #251 Delete over-engineered primitives (-65 LOC) - #254 Banner re-mount listener leak cleanup Tests pass (checkout-attempt-lifecycle + checkout-banner-initial-state). Typecheck clean. |
||
|
Elie Habib
|
7bc2dc03f8 |
feat(checkout): show masked receipt email in success banner (#3265)
Squashed rebase onto current main (which now includes #3259, #3260,
#3261, #3270, #3273, #3274). Source-only diff extracted via
`git diff
|
||
|
Elie Habib
|
dee3b97cfd |
feat(pro-marketing): /pro overlay success bridges to dashboard via ?wm_checkout=success (#3274)
Squashed rebase onto current main (which now includes #3259, #3260,
#3261, #3270, #3273). Source-only diff extracted via `git diff
|
||
|
Elie Habib
|
f2ea87d1f1 |
feat(checkout): duplicate-subscription dialog + unified new-tab portal (#3273)
Squashed rebase of PR-7's 5 commits onto current main, which now
includes #3259/#3260/#3261/#3270. Source changes extracted via
`git diff
|
||
|
Elie Habib
|
89d5dc6f59 |
feat(referral): propagate ref code /pro → dashboard via localStorage (#3270)
Squashed rebase of PR-14's 3 commits onto current main. Source changes extracted via `git diff` (excluding stale pro-bundle artifacts) and reapplied cleanly on main (which now includes #3259/#3260/#3261). Only import-section conflict in src/services/checkout.ts — resolved additively (kept all four parents' imports: checkout-attempt, error taxonomy, reload-unify, referral-capture). Changes: - src/services/referral-capture.ts (NEW): localStorage primitives (captureActiveReferral, loadActiveReferral, clearReferralOnAttribution) - src/App.ts: capture ref on boot - src/services/checkout.ts: read loadActiveReferral in startCheckout - src/services/checkout-attempt.ts: clearCheckoutAttempt also clears referral on success / signout (cross-user leak guard) - pro-test/src/App.tsx: validate ref code before appendRefToUrl - tests/referral-capture.test.mts: 11-test suite Pro bundle rebuilt fresh to match current source. |
||
|
Elie Habib
|
c2c6ca355b |
feat(checkout): unify reload ownership + extended "still unlocking" banner (#3261)
* feat(checkout): unify reload ownership + extended "still unlocking" banner
Before: two independent reload sources competed after a successful
Dodo checkout:
1. `setTimeout(reload, 3000)` inside the overlay `checkout.status`
handler (checkout.ts).
2. `window.location.reload()` inside the entitlement watcher on a
free→pro transition (panel-layout.ts).
The 3s timer fired unconditionally, which meant the success banner
was guaranteed to be wiped at 3s regardless of webhook latency —
making a "still unlocking" UX impossible. If the webhook was slower
than 3s, the user saw locked panels for a beat before the watcher's
second reload eventually landed.
This PR makes the entitlement watcher the SINGLE reload source
(Primitive C) and extends the banner to stay mounted across the
reload via a three-state machine driven by entitlement events:
pending → "Payment received! Unlocking your premium features…"
active → "Premium activated — reloading…" (watcher takes over)
timeout → 30s elapsed with no transition; swap to "Refresh if
features haven't unlocked" + manual Refresh button +
Sentry warning. Never silently disappears.
Hard-dep on #3163 (fix(pro): reliable post-payment activation) —
shipped 2026-04-18. That PR fixed the `skipInitialSnapshot` guard
that used to swallow the post-payment activation event; without it,
removing the 3s reload would leave some users stranded in pending.
Changes:
- `src/services/checkout.ts`:
- Remove the 3s `setTimeout(reload)` from the overlay `succeeded`
handler. Keep `markPostCheckout()` and `onSuccessCallback()` so
the post-reload consume path still seeds the transition detector.
- Rewrite `showCheckoutSuccess()` to accept `{ waitForEntitlement }`.
Classic path (no option) keeps the 5s auto-dismiss. Extended path
subscribes to `onEntitlementChange`, walks the three-state
machine, and exposes `data-entitlement-state` for e2e selectors.
- If already entitled at mount (e.g., post-reload with fast
webhook), skip straight to "active" so the banner doesn't lie
about a webhook still being in flight.
- `src/services/checkout-banner-state.ts` (new): pure helpers
(`computeInitialBannerState`, `EXTENDED_UNLOCK_TIMEOUT_MS`,
`CLASSIC_AUTO_DISMISS_MS`) extracted so they're testable without
pulling in the Dodo SDK through checkout.ts (same pattern as
PR-2's checkout-attempt.ts).
- `src/app/panel-layout.ts`: both success-banner call sites pass
`{ waitForEntitlement: true }` — one on the checkout-return path
and one as the `initCheckoutOverlay` onSuccess callback.
Tests: `tests/checkout-banner-initial-state.test.mts` — 5 cases
covering both initial-state branches + timing constant invariants
(30s > 5s ordering, exact values).
Acceptance criteria from PR-4 spec:
- [x] Grep `window.location.reload` in checkout.ts = 1 hit (the
user-initiated manual Refresh button in the timeout state).
The automatic reload has been removed.
- [x] Banner remains until entitlement transition OR 30s timeout.
- [x] Timeout branch shows retry CTA.
- [x] No double-reload (entitlement watcher is single source).
- [x] `markPostCheckout` flag still consumed post-reload; success
banner still appears on next load.
PR-4 of the 14-PR rollout at
docs/plans/2026-04-21-002-feat-harden-auth-checkout-flow-ux-plan.md.
Stacked on PR-2 (`feat/checkout-attempt-lifecycle`). Uses the
discriminated-union return from checkout-return.ts.
Typecheck + scoped lint + boundaries clean. test:data 6078/6078
passing.
* fix(checkout): auto-dismiss already-entitled success banner
showCheckoutSuccess({waitForEntitlement:true}) branched into the
"active" fast-path when isEntitled() was already true at mount — but
that branch had no auto-dismiss and no timeout. The only exit was the
entitlement-watcher reload, which never fires when we're already in
steady pro state (watcher only triggers on transitions). Result: the
"Premium activated — reloading…" banner sat forever until the user
manually refreshed.
Treat the fast-path like a classic confirmation: show active copy and
dismiss on CLASSIC_AUTO_DISMISS_MS (5s) so the user gets closure instead
of a stuck banner.
|
||
|
Elie Habib
|
221a4dba65 |
fix(sentry): filter iOS WKWebView UnknownError + Convex re-auth race (#3290)
Two new beforeSend filters, each gated on a positive provenance signal
narrow enough to avoid hiding genuine first-party regressions. Both
patterns have a first-party frame in the stack (so the existing
!hasFirstParty ambiguous-error block misses them), but the THROW itself
is guaranteed non-first-party:
• WORLDMONITOR-NM — UnknownError: "Cannot inject key into script value"
Thrown by iOS Safari WKWebView native bridge when passing a value
that can't be structurally cloned (history.pushState, IndexedDB,
etc.). Gate: excType === 'UnknownError' — that exception type is
WebKit-only and cannot originate from our TypeScript.
• WORLDMONITOR-NJ — Convex BaseConvexClient.tryToReauthenticate reads
this.authState.config.fetchToken during WebSocket reconnect when
authState.config is still undefined. Known Convex SDK internal race.
Gate: stack frame function name matches /tryToReauthenticate/ so a
first-party `.fetchToken` regression still surfaces.
+4 regression tests (2 positive, 2 negative) bring the beforesend suite
to 105/105.
|