mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
80797e7cc81dcdfb0d846ab1349a9986bc114811
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Elie Habib
|
80797e7cc8 |
feat(energy-atlas): seed-side countries[] denorm + CountryDeepDive row (§R #5 = B)
Per plan §R/#5 decision B: denormalise countries[] at seed time on each
disruption event so CountryDeepDivePanel can filter events per country
without an asset-registry round trip. Schema join (pipeline/storage
→ event.assetId) happens once in the weekly cron, not on every panel
render. The alternative (client-side join) was rejected because it
couples UI logic to asset-registry internals and duplicates the join
for every surface that wants a per-country filter.
Changes:
- `proto/.../list_energy_disruptions.proto`: add `repeated string
countries = 15` to EnergyDisruptionEntry with doc comment tying it
to the plan decision and the always-non-empty invariant.
- `scripts/_energy-disruption-registry.mjs`:
• Load pipeline-gas + pipeline-oil + storage-facilities registries
once per seed cycle; index by id.
• `deriveCountriesForEvent()` resolves assetId to {fromCountry,
toCountry, transitCountries} (pipeline) or {country} (storage),
deduped + alpha-sorted so byte-diff stability holds.
• `buildPayload()` attaches the computed countries[] to every
event before writing.
• `validateRegistry()` now requires non-empty countries[] of
ISO2 codes. Combined with the seeder's `emptyDataIsFailure:
true`, this surfaces orphaned assetIds loudly — the next cron
tick fails validation and seed-meta stays stale, tripping
health alarms.
- `scripts/data/energy-disruptions.json`: fix two orphaned assetIds
that the new join caught:
• `cpc-force-majeure-2022`: `cpc-pipeline` → `cpc` (matches the
entry in pipelines-oil.json).
• `pdvsa-designation-2019`: `ve-petrol-2026-q1` (non-existent) →
`venezuela-anzoategui-puerto-la-cruz`.
- `server/.../list-energy-disruptions.ts`: project countries[] into
the RPC response via coerceStringArray. Legacy pre-denorm rows
surface as empty array (always present on wire, length 0 => old).
- `src/components/CountryDeepDivePanel.ts`: add 4th Atlas row —
"Energy disruptions in {iso2}" — filtered by `iso2 ∈ countries[]`.
Failure is silent; EnergyDisruptionsPanel (upcoming) is the
primary disruption surface.
- `tests/energy-disruptions-registry.test.mts`: switch to validating
the buildPayload output (post-denorm), add §R #5 B invariant
tests, plus a raw-JSON invariant ensuring curators don't hand-edit
countries[] (it's derived, not declared).
Proto regen note: `make generate` currently fails with a duplicate
openapi plugin collision in buf.gen.yaml (unrelated bug — 3 plugin
entries emit to the same out dir). Worked around by temporarily
trimming buf.gen.yaml to just the TS plugins for this regen. Added
only the `countries: string[]` wire field to both service_client and
service_server; no other generated-file drift in this PR.
|
||
|
Elie Habib
|
df91e99142 |
feat(energy): expand 5 curated registries to 100% of plan target (#3337)
* feat(energy): expand gas pipeline registry 12 → 28 (phase 1a batch 1)
Data validation after v1 launch showed pipelines shipped at ~16% of the
plan target (12 gas + 12 oil vs. the plan's 75 + 75 critical
pipelines). This commit closes ~20% of the gas gap with 16 hand-curated
global additions, every entry carrying a full evidence bundle matching
the schema enforced by scripts/_pipeline-registry.mjs.
New additions by region:
North Sea / NW Europe (6):
europipe-1, europipe-2, franpipe, zeepipe, interconnector-uk-be, bbl
Mediterranean / North Africa (3):
transmed (Enrico Mattei), greenstream (LY→IT, reduced),
meg-maghreb-europe (DZ→ES via MA, offline since Oct 2021)
Middle East (1):
arab-gas-pipeline (EG→LB via JO/SY, offline under Caesar Act)
Former Soviet / Turkey (1):
blue-stream (RU→TR, carries EU sanctions ref)
Asia (3):
west-east-3 (CN internal, 7378 km), myanmar-china-gas (shwe),
igb (interconnector-greece-bulgaria, 2022)
Africa / LatAm (2):
wagp (west african gas pipeline, 4-country transit),
gasbol (bolivia-brazil, 3150 km)
Badge distribution on new entries:
flowing: 12, reduced: 2, offline: 2
First non-Russia-exposure offline entries (MEG — Morocco-Algeria
diplomatic closure, Arab Gas — Syria sanctions) — broadens the
geographic distribution of evidence-bundle-backed non-positive badges.
Registry tests: 17/17 pass (identity, geometry bounds, ISO2 country
codes, evidence contract, capacity-commodity pairing, validateRegistry
negative cases).
Next batches in this phase: oil pipelines +16, then second batches
each commodity to reach plan target (75+75). Tracked in
docs/internal/energy-atlas-registry-expansion.md.
* feat(energy): expand oil pipeline registry 12 → 28 (phase 1a batch 2)
Mirror of the gas batch — 16 hand-curated global additions with full
evidence bundles. Closes ~20% of the oil gap.
New additions by region:
North America (6):
enbridge-mainline (CA→US 3.15 mbd), enbridge-line-3-replacement (2021),
flanagan-south, seaway (Cushing→Gulf), marketlink (TC, Cushing→Gulf),
spearhead
Middle East (3):
sumed (EG crude bypass of Suez, 2.8 mbd),
east-west-saudi (Petroline, 5 mbd — largest single oil pipeline in
the registry by capacity),
ipsa-2 (IQ→SA, offline since Iraq invasion of Kuwait 1990, later
converted to gas on the western stretch)
Central Asia (1):
kazakhstan-china-crude (KZ→CN Alashankou, 2228 km)
Africa (1):
chad-cameroon-cotco (TD→CM Kribi, 1070 km)
South America (2):
ocp-ecuador (heavy crude, 450 kbd),
sote-ecuador (lighter grades, 360 kbd)
Europe (3):
tal-trieste-ingolstadt (IT→DE via AT, 770 kbd),
janaf-adria (HR→RS→HU, 280 kbd),
norpipe-oil (NO→DE North Sea crude, 900 kbd)
Badge distribution on new entries:
flowing: 15, offline: 1 (IPSA-2, regulator-sourced + nationalisation
statement backing the offline badge per the evidence-contract rules).
Registry totals after this batch:
gas: 12 → 28 (37% of plan target 75)
oil: 12 → 28 (37% of plan target 75)
total: 24 → 56
Registry tests: 17/17 registry + 23/23 evidence-derivation = 40/40 pass.
Typecheck-free (JSON only).
Next batches (per docs/internal/energy-atlas-registry-expansion.md):
gas batch 2: +22 → 50 (North Sea remainder, Caspian, Asia)
oil batch 2: +22 → 50 (North Sea remainder, Russia diversified,
Asia long-haul)
* feat(energy): expand gas pipeline registry 28 → 50 (phase 1a batch 3)
Second gas batch, 22 additions, bringing gas to ~67% of the 75-pipeline
plan target. Geographic distribution deliberately skewed this batch
toward under-represented regions (Middle East, Central Asia, South
America, Africa, Southeast Asia) since the first batch filled Europe
and North America.
New additions (22):
North Sea / UK (2):
vesterled (NO→GB, 13 bcm/yr),
cats (UK, 9.6 bcm/yr)
Iran family (3):
iran-turkey-gas (Tabriz→Ankara, 14 bcm/yr, OFAC sanctions ref),
iran-armenia-gas (2.3 bcm/yr),
iran-iraq-basra-gas (reduced state — waiver-dependent flows)
Central Asia (2):
central-asia-center (TM→RU via UZ/KZ, 44 bcm/yr nominal, reduced),
turkmenistan-iran-korpeje (expired contract, reduced)
Caucasus / Turkey (2):
south-caucasus-scp (BTE predecessor to TANAP, 22 bcm/yr),
sakarya-black-sea-tr (2023 Turkish offshore)
China (2):
west-east-1 (4200 km, 17 bcm/yr),
west-east-2 (8700 km, 30 bcm/yr)
South America (2):
bolivia-argentina-yacuiba (reduced),
antonio-ricaurte (CO→VE, offline since 2015, PDVSA sanctions)
Saudi / Middle East (2):
saudi-master-gas-system (SA internal, 95 bcm/yr — largest capacity
in the registry), egypt-jordan-aqaba (AGP south leg, flowing)
Israel-Egypt (1):
israel-egypt-arish-ashkelon (reverse-flow since 2020, IL→EG export)
Planned / FID-stage (5):
galsi-planned (DZ→IT, consortium paused),
eastmed-planned (IL→CY→GR, US political support withdrawn Jan 2022),
trans-saharan-planned (NG→DZ via NE, insurgency + financing unresolved),
morocco-nigeria-offshore-planned (NG→MA 11-country offshore route),
power-of-siberia-2-planned (RU→CN via MN, no binding CNPC contract),
kirkuk-dohuk-turkey-gas-planned (IQ→TR, Baghdad-Erbil dispute)
Badge distribution on new batch:
flowing: 10 (incl. Sakarya 2023 commissioned)
reduced: 3 (CAC, BO-AR, IR-IQ)
offline: 1 (Antonio Ricaurte, CO-VE, with operator statement + sanction)
unknown: 6 (all planned/FID-stage, classifierConfidence 0.6-0.75)
All non-flowing badges have evidence (sanction refs, operator
statements, or press sourcing) per the evidence-contract validator.
Registry totals after this batch:
gas: 28 → 50 (67% of plan target; gas ≥60 gate threshold not yet
hit but approaching)
oil: 28 (unchanged — batch 4 will target oil to 50)
total: 56 → 78
Registry tests: 17/17 pass. Includes 8 new fully-hedged "unknown" /
planned-status entries; validator accepts them.
Next: oil batch 2 (+22 → 50), then gas batch 3 (+10 → 60), oil batch 3
(+10 → 60). After that the gate criteria on pipelines hit and we can
focus on storage / shortages / disruptions.
* feat(energy): expand oil pipeline registry 28 → 50 (phase 1a batch 4)
Second oil batch, 22 additions, bringing oil to 67% of plan target and
matching gas (50 each, 100 total pipelines).
New additions (22):
Russia Baltic export (2):
bps-1 (Primorsk, 1.3 mbd — largest single line in oil registry),
bps-2 (Ust-Luga, 0.75 mbd). Both carry G7+EU price-cap sanctions ref.
North America diversified (3):
enbridge-line-5 (CA→CA via US Straits of Mackinac, ongoing litigation),
keystone-xl-cancelled (CA→US, permit revoked 2021, Biden; TC
terminated Jun 2021; listed for historical + geopolitical
completeness, physicalState=unknown by deriver rule),
trans-panama-pipeline (PA, 0.9 mbd cross-isthmus)
Europe remaining (3):
rotterdam-rhine-rrp (NL→DE, 275 km),
spse (FR→DE Lyon→Karlsruhe, 769 km),
forties-pipeline (UK North Sea, 0.6 mbd),
brent-pipeline (NO→GB Sullom Voe, reduced — Brent field in
decommissioning)
Middle East (2):
khafji-neutral-zone (SA/KW, reduced post-2015 neutral-zone dispute),
ab-1-bahrain (SA→BH, 2018, 0.35 mbd)
Africa (4):
greater-nile-petroleum (SS→SD Port Sudan, 1610 km),
djeno-congo (CG terminal system),
nigeria-forcados-export (reduced — recurring force-majeure),
nigeria-bonny-export (Trans Niger Pipeline, reduced)
Latin America (2):
pemex-nuevo-cactus (MX, 0.44 mbd),
trans-andino (AR→CL, offline since 2006 export restrictions)
Ukraine (1):
odesa-brody (offline, under EU 2022/879 Russian-crude embargo
framework)
Asia (1):
myanmar-china-crude (MM→CN Kunming, 771 km parallel to
myanmar-china-gas)
Caspian (1):
baku-novorossiysk-northern (AZ→RU historical route, reduced, carries
Russian crude price-cap ref)
Historical / planned (2):
kirkuk-haifa-idle (IQ→IL via JO, closed 1948 — listed for
completeness; periodically floated as reopening proposal),
uganda-tanzania-eacop-planned (UG→TZ, under construction, Western
bank-financing pulled but TotalEnergies continues)
Badge distribution on new batch:
flowing: 10
reduced: 6 (Brent decommissioning, Khafji dispute, Greater Nile,
Forcados, Bonny, Baku-Novorossiysk)
offline: 2 (Odesa-Brody, Trans-Andino, Kirkuk-Haifa)
unknown: 2 (Keystone XL cancelled, EACOP under construction)
Wait, Kirkuk-Haifa is offline not among 2. Corrected count:
flowing: 10, reduced: 6, offline: 3 (Odesa-Brody, Trans-Andino,
Kirkuk-Haifa), unknown: 2, plus 1 flowing Myanmar-China-crude = 22.
All non-flowing badges carry supporting evidence (operator statements,
sanction refs, or press citations) per the evidence-contract validator.
Registry totals after this batch:
gas: 50 (67% of plan target)
oil: 28 → 50 (67% of plan target)
total: 78 → 100
Registry tests: 17/17 + 23/23 evidence-derivation = 40/40 pass.
Next batches to hit the 60-each gate criteria from
docs/internal/energy-atlas-registry-expansion.md:
gas batch 3: +10 → 60 (EastMed details, Galsi alternative routes,
minor EU-interconnectors, Nigeria LNG feeder gas lines)
oil batch 3: +10 → 60 (Pluto crude, Chinese Huabei system, Latam
infill: Brazil Campos, Peru Northern Trunk)
After 60/60: hit gate, move to storage expansion.
* feat(energy): gas registry 50 → 75 — plan target hit
Batch 3 adds 25 more gas pipelines, bringing gas to 100% of the
75-pipeline plan target.
New additions by region (25):
- Norwegian transport spine: statpipe, sleipner-karsto, troll-a,
oseberg-gas-transport, asgard-transport (covers the major offshore
export collectors — the rest of the Gassco system)
- Australia: dampier-bunbury (1594 km), moomba-sydney (1299 km)
- Africa: mozambique-rompco (MZ→ZA), escravos-lagos-gas (NG),
tanzania-mtwara-dar, ghana-gas (atuabo)
- Southeast Asia: thailand-malaysia-cakerawala, indonesia-singapore
west-natuna + grissik-sakra
- German hubs for Nord Stream continuation: nel-pipeline, opal-pipeline,
eugal-pipeline (built but dormant after NS2 halt/destruction),
megal-pipeline, gascade-jagal, zeelink-germany
- Russia/Ukraine/EU transit: progress-urengoy-uzhhorod (halted 1 Jan
2025 when Ukraine did not renew transit agreement), trans-austria-gas
- Iran: kish-iran-gas, iran-pakistan-gas-planned (Pakistani segment
stalled since 2014)
- China/HK: china-hong-kong-gas
Badge distribution on new batch: 15 flowing, 4 reduced (NEL, OPAL,
TAG, Escravos-Lagos), 2 offline (EUGAL dormant post-NS2,
Urengoy-Uzhhorod transit halt), 4 sanction-exposed (NS-continuation
pipelines + TAG + Urengoy), 1 unknown (Iran-Pakistan stalled
completion).
Plan progress: gas 50 → 75 (100% of plan target).
Registry tests: 17/17 pass.
* feat(energy): oil registry 50 → 75 — plan target hit
Batch 4 adds 25 more oil pipelines, bringing oil to 100% of the
75-pipeline plan target. Combined with gas at 75, total registry is
150 pipelines — full plan coverage for Phase 1a.
New additions by region (25):
- Latin America: colombia-cano-limon-covenas (ELN-sabotaged, reduced),
colombia-ocensa (main trunk), peru-norperuano (reduced from jungle
spills + protests), ecuador-lago-agrio-orellana,
venezuela-anzoategui-puerto-la-cruz (under OFAC PDVSA sanctions),
mexico-salina-cruz-minatitlan, mexico-madero-cadereyta,
mexico-gulf-coast-pipeline (Tuxpan-Mexico City)
- Africa: angola-cabinda-offshore, south-sudan-kenya-lamu-planned
(LAPSSET)
- Middle East: iran-abadan-isfahan, iran-neka-tehran (reduced,
Caspian swap arrangements), saudi-abqaiq-yanbu-products,
iraq-strategic-pipeline (1000 km north-south), iraq-bai-hassan,
oman-muscat-export (Fahud-Mina al-Fahal), uae-habshan-ruwais
- Asia-Pacific: india-salaya-mathura (1770 km, largest Indian crude
trunk), india-vadinar-kandla, india-mundra-bhatinda,
china-qinhuangdao-tianjin-huabei, china-yangzi-hefei-hangzhou
- Russia East: russia-sakhalin-2-crude, russia-komsomolsk-perevoznaya,
russia-omsk-pavlodar (cross-border to KZ)
Badge distribution on this batch: 18 flowing, 6 reduced, 1 unknown
(LAPSSET planned). Sanctions-exposure diversified: Iran framework (3),
Venezuela/PDVSA (1), Russian price-cap (3). All non-flowing badges
carry supporting evidence per validator rules.
Phase 1a final state (pipelines):
gas: 12 → 75 (100% of plan target, 6 batches)
oil: 12 → 75 (100% of plan target, 6 batches)
total: 24 → 150
Geographic distribution now global:
- Russia-exposure: ~22 of 150 entries (~15%, down from 50% at v1)
- US-only: ~8 (~5%, down from 33% storage-side skew)
- Six continents represented in active infrastructure
- Historical + planned pipelines (Kirkuk-Haifa, Keystone XL cancelled,
EACOP u/c, EastMed planned, GALSI planned, TSGP planned,
Nigeria-Morocco offshore, Power of Siberia 2, Iran-Pakistan Peace,
LAPSSET) listed with honest 'unknown' physicalState per validator
Registry tests: 17/17 pass.
Phase 1a complete. Next phase (per
docs/internal/energy-atlas-registry-expansion.md):
- Phase 2: storage 21 → ~200 (+179) via curation + GIIGNL/GIE/EIA
- Phase 3: shortages 14 → 28 countries
- Phase 4: disruptions 12 → 50 events
* feat(energy): shortages 15 → 29 entries across 28 countries — plan target hit
+14 country additions matching the 28-country plan target. The
validator's 'confirmed severity requires authoritative source' rule
caught two of my drafts (Myanmar + Sudan) where I had labeled them
confirmed with press-only evidence because regulator/operator sources
under a junta + active civil war are not independently verifiable.
Downgraded both to 'watch' with an inline note explaining the
evidence-quality choice — exactly the validator's intended behavior
(better to under-claim than over-claim severity when the authoritative
channel is broken).
New shortages (14):
- BD diesel: BPC LC delays, regulator-confirmed
- ZA diesel: loadshedding demand spike
- AO diesel: Luanda/Benguela depot delays
- MZ diesel: FX-allocation import constraints
- ZM diesel: mining-sector demand + TAZAMA product tightness
- MW diesel: FX shortfalls + MERA rationing
- GH petrol: Tema port congestion
- MM diesel: post-coup chronic (watch, press-only evidence)
- MN diesel: winter logistics
- CO diesel: trucker strike cycles
- UA diesel: war-driven chronic (confirmed — Ministry of Energy source)
- SY diesel: Caesar Act chronic (confirmed — Syrian Ministry statement)
- SD diesel: civil-war disruption (watch, press-only)
- DE heating_oil: Rhine low-water logistics (watch)
Badge distribution on new batch: 3 confirmed (BD, UA, SY — all with
regulator/operator evidence), 11 watch.
Plan progress:
shortages: 15 → 29 entries (28 unique countries = 100% of plan)
gas: 75 (100%)
oil: 75 (100%)
storage: 21 (unchanged, next batch)
disruptions: 12 (unchanged, next batch)
Registry tests: 19/19 pass.
* feat(energy): disruption event log 12 → 52 events — plan target hit
+40 historical and ongoing events covering the asset registry,
bringing disruptions to 104% of the 50-event plan target. Every event
ties to an assetId now in pipelines/storage registries (following the
75-gas + 75-oil + 21-storage registry expansion in the preceding
commits).
New additions by eventType:
Sabotage / war (7):
- abqaiq-khurais-drone-strike-2019 (Saudi, 5.7 mbd removed 11 days)
- russia-refinery-drone-strikes-2024 (Ukrainian drone strike series)
- houthi-red-sea-attacks-2024 (indirect SuMed demand impact)
- russia-ukraine-oil-depot-strikes-2022 (series)
- nigeria-trans-niger-attacks-2024 (Bonny system)
- bai-hassan-attack-2022 (Iraq Bai Hassan)
- sudan-pipeline-attacks-2023 (Greater Nile disruption)
Sanctions (7):
- russia-price-cap-implementation-2022 (G7+EU $60/bbl cap)
- eu-oil-embargo-2022 (6th package)
- pdvsa-designation-2019 (Venezuela)
- btc-kurdistan-shutdown-2023 (ICC ruling, ongoing)
- ipsa-nationalization-2001 (SA nationalised after Iraq invasion of Kuwait)
- arctic-lng-2-foreign-partner-withdrawal-2024
- yamal-lng-arctic-sanctions-ongoing (Novatek)
- ogm-moldova-transit-2022
Mechanical (4):
- druzhba-contamination-2019 (chlorides, 3-month shut)
- keystone-milepost-14-leak-2022 (Kansas, 22-day shut, 14k bbl spill)
- forties-crack-2017 (Red Moss hairline)
- ocensa-ocp-ecuador-suspensions-2022 (Amazon landslide)
Weather (2):
- hurricane-ida-lng-2021 (Gulf coast LNG shutdown)
- rotterdam-hub-low-water-2022 (Rhine 2.5-month disruption)
Commercial (9):
- cpc-blockage-threat-2022 (Russian court 30-day halt threat)
- gme-closure-2021 (Algeria-Morocco MEG)
- ukraine-transit-end-2025 (Progress pipeline halted 1 Jan 2025)
- eugal-dormant-since-2022 (NS2 knock-on)
- keystone-xl-permit-revoked-2021 (Biden day-1)
- antonio-ricaurte-halt-2015 (CO→VE gas export halt)
- langeled-brent-decommissioning-2020
- eacop-financing-2023 (Western bank withdrawal)
- dolphin-qatar-uae-commercial-2024 (contract renegotiation)
- trans-austria-gas-reduction-2022 (Gazprom volume drops)
- cushing-stocks-tank-bottoms-2022
- spr-drawdown-2022-2023 (largest ever 180 mbbl release)
- zhoushan-storage-expansion-2023
- fujairah-stockbuild-2024
- futtsu-lng-demand-decline-2024
- bolivia-diesel-import-cut-2023 (GASBOL)
- myanmar-china-gas-reduced-2023
- yamal-europe-poland-halt-follow-on-2024
Maintenance (1): gladstone-lng-maintenance-2023
Ongoing events (endAt=null): 31 of 52 (~60%). Reflects the structural
reality that many 2022-era sanctions + war events remain live in 2026.
Plan progress:
gas: 75 (100%)
oil: 75 (100%)
storage: 21 (unchanged, next batch)
shortages: 29 (100% — 28 countries)
disruptions: 12 → 52 events (104% of plan)
Registry tests: 16/16 pass.
* feat(energy): storage registry 21 → 66 (storage batch 1)
+45 facilities, 33% of plan. Focus: European UGS + second LNG wave.
European UGS additions (35 — mostly filling the gap against GIE AGSI+
coverage which has ~140 EU sites; we now register the majority of
operationally significant ones with non-trivial working capacity):
Germany (9): bierwang, etzel-salt-cavern, jemgum, krummhoern,
peckensen, reckrod, uelsen, xanten, epe-salt-cavern
Netherlands (3): alkmaar, norg (largest NL, 59.2 TWh), zuidwending
Austria (3): 7fields-schonkirchen (24.6 TWh), baumgarten-uhs,
puchkirchen
France (7): chemery (38.5 TWh), cerville-velaine, etrez, manosque,
lussagnet (35 TWh), izaute
Italy (4): minerbio (45 TWh, largest IT), ripalta, sergnano,
brugherio
UK (2): rough (reduced, post-2017 partial reopening 2022), hornsea
Central/Eastern Europe (8): damborice (CZ), lobodice (CZ),
lab-slovakia (36 TWh), hajduszoboszlo (HU), mogilno (PL),
lille-torup (DK), incukalns (LV), gaviota (ES)
Russia (1): kasimovskoe (124 TWh — Gazprom UGS flagship; EU sanctions
ref carried as evidence)
LNG terminals (9 additions to round out global coverage):
- US: freeport-lng, cameron-lng, cove-point-lng, elba-island-lng
- Middle East: qalhat-lng (Oman), adgas-das-island (UAE)
- Russia: sakhalin-2-lng (sanctions-exposed)
- Indonesia: tangguh-lng, bontang-lng (reduced — declining upstream)
Badge distribution on this batch: 43 operational, 2 reduced (Rough,
Bontang). Most entries from GIE AGSI+ fill-disclosed data; Russian
site + LNG terminals fill-not-disclosed (operator choice + sanctions).
Plan progress:
gas pipelines: 75 (100%)
oil pipelines: 75 (100%)
fuel shortages: 29 / 28 countries (100%)
disruptions: 52 (104%)
storage: 21 → 66 (33% of ~200 target)
Registry tests: 21/21 pass.
Next storage batches remaining:
batch 2 (+45): more European UGS tail + Asian national reserves
(CN SPR, IN SPR, JP national reserves, KR KNOC)
batch 3 (+45): LNG import terminals + additional US tank farms +
European tank farms (Rotterdam detail, ARA sub-sites)
batch 4 (+45): remainder to ~200
* feat(energy): storage registry 66 → 110 (storage batch 2)
+44 facilities. Focus: Asian national reserves + global LNG coverage
+ Singapore/ARA tank-farm detail.
Asian national reserves (11):
- IN ISPRL: vizag (9.8 Mb), mangalore (11 Mb), padur (17.4 Mb)
- CN: zhanjiang (45 Mb), huangdao (20 Mb) — fill opaque, press-only
- JP JOGMEC: shibushi (31.2 Mb), kiire (22 Mb), mutsu-ogawara (28 Mb)
- KR KNOC: yeosu (42 Mb), ulsan (33 Mb), geoje (47 Mb)
LNG export additions (11):
- Australia: pluto-lng, prelude-flng (reduced), darwin-lng (reduced
upstream)
- Southeast Asia: mlng-bintulu (29.3 Mtpa — largest in registry),
brunei-lng, donggi-senoro-lng
- Africa: angola-lng (reduced), equatorial-guinea-lng, hilli-episeyo-flng
- Pacific: png-lng
- Caribbean: trinidad-atlantic-lng (reduced)
- Mexico: costa-azul-lng (2025 reverse-to-export commissioning)
LNG import (12):
- UK: south-hook-lng (21 Mtpa), dragon-lng
- EU: zeebrugge-lng, dunkerque-lng, fos-cavaou-lng,
montoir-de-bretagne-lng, gate-terminal (Rotterdam),
revithoussa-lng
- Turkey: aliaga-ege-gaz-lng
- Chile: mejillones-lng, quintero-bay-lng
Tank farms (10):
- Africa: saldanha-bay (ZA 45 Mb)
- Norway: mongstad-crude
- ARA: antwerp-petroleum-hub (BE 55 Mb), amsterdam-petroleum-hub
- Asia hubs: singapore-jurong (120 Mb — largest in registry),
singapore-pulau-ayer-chawan, thailand-sriracha, korea-gwangyang-crude
- Russia Baltic: ust-luga-crude-terminal, primorsk-crude-terminal
(both carry Russian price-cap sanction refs)
Badge distribution on this batch: 39 operational, 5 reduced (Prelude,
Darwin, Angola, Bontang — no wait Bontang already in. Correct: Prelude,
Darwin, Angola, Trinidad).
Plan progress:
gas pipelines: 75 (100%)
oil pipelines: 75 (100%)
fuel shortages: 29 / 28 countries (100%)
disruptions: 52 (104%)
storage: 66 → 110 (55% of ~200 target)
Registry tests: 21/21 pass.
Next batches remaining: ~90 more storage to hit ~200
batch 3 (+45): Middle East tank farms, Chinese coastal commercial
storage, EU UGS tail, African LNG import
batch 4 (+45): remainder to 200
* feat(energy): storage registry 110 → 155 (storage batch 3)
Adds 45 facilities toward 200 plan target:
- 7 Middle East export terminals (Kharg, Sidi Kerir, Mina al-Ahmadi,
Mesaieed, Jebel Dhanna, Mina al-Fahal, Bandar Imam Khomeini)
- 10 EU UGS tail (Reitbrook, Empelde, Kirchheilingen, Stockstadt,
Nüttermoor, Grijpskerk, Târgu Mureș, Třanovice, Uhřice, Háje)
- 4 Chinese coastal crude (Yangshan, Qingdao, Rizhao, Maoming)
- 6 EU LNG import tail (La Spezia, Adriatic, OLT Livorno, Klaipeda,
Mugardos, Cartagena)
- 5 Indian LNG import (Hazira, Kochi reduced, Ennore, Mundra, Dabhol)
- 6 Japan/Korea LNG import (Chita, Negishi, Sodegaura, Himeji,
Pyeongtaek, Incheon)
- 5 NA tank farms (Lake Charles, Corpus Christi, Patoka, Edmonton,
Hardisty)
- 2 Asia-Pacific (Kaohsiung, Nghi Son)
Registry validator: 21/21 tests pass.
* feat(energy): storage registry 155 → 200 (storage batch 4 — plan target hit)
Final batch brings storage to the 200-facility plan target with broad
geographic + facility-type coverage.
New entries (45):
- 6 LNG export: NLNG Bonny (NG, reduced), Arzew (DZ), Skikda (DZ),
Perú LNG, Calcasieu Pass (US), North West Shelf Karratha (AU)
- 7 LNG import: Świnoujście (PL), Krk FSRU (HR), Wilhelmshaven FSRU (DE),
Brunsbüttel (DE), Map Ta Phut (TH), Port Qasim (PK), Batangas (PH)
- 6 UGS: Bilche-Volytsko-Uherske (UA, 154 TWh — largest Europe), Banatski
Dvor (RS), Okoli (HR), Yela (ES), Loenhout (BE), Kushchevskoe (RU)
- 26 crude tank farms: José Terminal (VE, sanctioned), Santos (BR),
TEBAR São Sebastião (BR), Dos Bocas (MX), Bonny (NG, reduced), Es
Sider (LY, reduced), Ras Lanuf (LY, reduced), Ceyhan (TR), Puerto
Rosales (AR), Novorossiysk Sheskharis (RU, sanctioned), Kozmino (RU,
sanctioned), Tema (GH, reduced), Mombasa (KE), Abidjan SIR (CI),
Juaymah (SA), Ras Tanura (SA), Yanbu (SA), Kirkuk (IQ, reduced),
Basra Gulf (IQ), Djibouti Horizon (DJ), Yokkaichi (JP), Mailiao
(TW), Ventspils (LV, reduced), Gdańsk Naftoport (PL), Constanța
(RO), Wood River IL (US).
Geographic balance improved: Africa coverage (NG, DZ, LY, GH, KE, CI,
DJ) from 5 to 12 countries; first Iraq + Saudi entries; Balkans +
Ukraine + Romania now covered. Type mix: UGS 56, SPR 15, LNG export 33,
LNG import 38, crude tank farm 58.
Non-operational entries all carry authoritative evidence (press
operator statements + sanctionRefs for Russia/Venezuela).
Registry validator: 21/21 tests pass. Total: 200 facilities across 55
countries. Plan target hit.
* fix(energy): address Greptile review findings on registries
P1 — abqaiq-khurais-drone-strike-2019 (energy-disruptions.json):
capacityOfflineMbd was 5.7 (plant-level Saudi production loss headline)
against assetId east-west-saudi (5.0 mbd pipeline). Capped offline
figure at the linked pipeline's 5.0 mbd ceiling; moved the 5.7 mbd
historical headline into shortDescription with an explanatory note.
Preserves capacity-offline ≤ asset-capacity invariant for downstream
consumers.
P1 — russia-price-cap-implementation-2022 (energy-disruptions.json):
was linked to assetId espo (land pipeline to China — explicitly out of
scope for G7/EU price cap). Relinked to primorsk-crude-terminal
(largest Baltic seaborne crude export terminal, directly affected);
assetType pipeline → storage. Updated shortDescription to clarify
tanker-shipment scope + out-of-scope note for ESPO.
P2 — 13 reduced-state pipelines missing press citation text
(pipelines-gas.json × 8 + pipelines-oil.json × 5):
Added operatorStatement sentences naming the press/regulator sources
backing each reduction claim (Reuters, NNPC/Chevron releases, NIGC,
Pemex annual reports, S&P Platts, IEA Gas Market Report, BBC, etc.).
Clears the evidence-source-type gap flagged by Greptile for entries
that declared physicalStateSource: "press" with a null statement.
All 6583 data tests + 94 registry tests still pass.
* style(energy): restore compact registry formatting (preserve Greptile-fix evidence)
Prior commit
|
||
|
Elie Habib
|
84ee2beb3e |
feat(energy): Energy Atlas end-to-end — pipelines + storage + shortages + disruptions + country drill-down (#3294)
* feat(energy): pipeline registries (gas + oil) — evidence-based schema
Day 6 of the Energy Atlas Release 1 plan (Week 2). First curated asset
registry for the atlas — the real gap vs GEF.
## Curated data (critical assets only, not global completeness)
scripts/data/pipelines-gas.json — 12 critical gas lines:
Nord Stream 1/2 (offline; Swedish EEZ sabotage 2022; EU sanctions refs),
TurkStream, Yamal–Europe (offline; Polish counter-sanctions),
Brotherhood/Soyuz (offline; Ukraine transit expired 2024-12-31),
Power of Siberia, Dolphin, Medgaz, TAP, TANAP,
Central Asia–China, Langeled.
scripts/data/pipelines-oil.json — 12 critical oil lines:
Druzhba North/South (N offline per EU 2022/879; S under landlocked
derogation), CPC, ESPO (+ price-cap sanction ref), BTC, TAPS,
Habshan–Fujairah (Hormuz bypass), Keystone, Kirkuk–Ceyhan (offline
since 2023 ICC ruling), Baku–Supsa, Trans-Mountain (TMX expansion
May 2024), ESPO spur to Daqing.
Scope note: 75+ each is Week 2b work via GEM bulk import. Today's cut
is curated from first-hand operator disclosures + regulator filings so
I can stand behind every evidence field.
## Evidence-based schema (not conclusion labels)
Per docs/methodology/pipelines.mdx: no bare `sanctions_blocked` field.
Every pipeline carries an evidence bundle with `physicalState`,
`physicalStateSource`, `operatorStatement`, `commercialState`,
`sanctionRefs[]`, `lastEvidenceUpdate`, `classifierVersion`,
`classifierConfidence`. The public badge (`flowing|reduced|offline|
disputed`) is derived server-side from this bundle at read time.
## Seeder
scripts/seed-pipelines.mjs — single process publishes BOTH keys
(energy:pipelines:{gas,oil}:v1) via two runSeed() calls. Tiny datasets
(<20KB each) so co-location is cheap and guarantees classifierVersion
consistency.
Conventions followed (worldmonitor-bootstrap-registration skill):
- TTL 21d = 3× weekly cadence (gold-standard per
feedback_seeder_gold_standard.md)
- maxStaleMin 20_160 = 2× cadence (health-maxstalemin-write-cadence skill)
- sourceVersion + schemaVersion + recordCount + declareRecords wired
(seed-contract-foundation)
- Zero-case explicitly NOT allowed — MIN_PIPELINES_PER_REGISTRY=8 floor
## Health registration (dual, per feedback_two_health_endpoints_must_match)
- api/health.js: BOOTSTRAP_KEYS adds pipelinesGas + pipelinesOil;
SEED_META adds both with maxStaleMin=20_160.
- api/seed-health.js: mirror entries with intervalMin=10_080 (maxStaleMin/2).
## Bundle registration
scripts/seed-bundle-energy-sources.mjs adds a single Pipelines entry
(not two) because seed-pipelines.mjs publishes both keys in one run —
listing oil separately would double-execute. Monitoring of the oil key
staleness happens in api/health.js instead.
## Tests (tests/pipelines-registry.test.mts)
17 passing node:test assertions covering:
- Schema validation (both registries pass validateRegistry)
- Identity resolution (no id collisions, id matches object key)
- Country ISO2 normalization (from/to/transit all match /^[A-Z]{2}$/)
- Endpoint geometry within Earth bounds
- Evidence rigor: non-flowing badges require at least one supporting
evidence source (operator statement / sanctionRefs / ais-relay /
satellite / press)
- ClassifierConfidence in 0..1
- Commodity/capacity pairing (gas uses capacityBcmYr, oil uses
capacityMbd — mixing = test fail)
- validateRegistry rejects: empty object, null, no-evidence fixtures,
below-floor counts
Typecheck clean (both tsconfig.json and tsconfig.api.json).
Next: Day 7 will add list-pipelines / get-pipeline-detail RPCs in
supply-chain/v1. Day 8 ships PipelineStatusPanel with DeckGL PathLayer
consuming the registry.
* fix(energy): split seed-pipelines.mjs into two entry points — runSeed hard-exits
High finding from PR review. scripts/seed-pipelines.mjs called runSeed()
twice in one process and awaited Promise.all. But runSeed() in
scripts/_seed-utils.mjs hard-exits via process.exit on ~9 terminal paths
(lines 816, 820, 839, 888, 917, 989, plus fetch-retry 946, fatal 859,
skipped-lock 81). The first runSeed to reach any terminal path exits the
entire node process, so the second runSeed's resolve never fires — only
one of energy:pipelines:{gas,oil}:v1 would ever be written.
Since the bundle scheduled seed-pipelines.mjs exactly once, and both
api/health.js and api/seed-health.js expect both keys populated, the
other registry would stay permanently EMPTY/STALE after deploy.
Fix: split into two entry-point scripts around a shared utility.
- scripts/_pipeline-registry.mjs (NEW, was seed-pipelines.mjs) — shared
helpers ONLY. Exports GAS_CANONICAL_KEY, OIL_CANONICAL_KEY,
PIPELINES_TTL_SECONDS, MAX_STALE_MIN, buildGasPayload, buildOilPayload,
validateRegistry, recordCount, declareRecords. Underscore prefix marks
it as non-entry-point (matches _seed-utils.mjs / _seed-envelope-source.mjs
convention).
- scripts/seed-pipelines-gas.mjs (NEW) — imports from the shared module,
single runSeed('energy','pipelines-gas',…) call.
- scripts/seed-pipelines-oil.mjs (NEW) — same shape, oil.
- scripts/seed-bundle-energy-sources.mjs — register BOTH seeders (not one).
- scripts/seed-pipelines.mjs — deleted.
- tests/pipelines-registry.test.mts — update import path to the shared
module. All 17 tests still pass.
Typecheck clean (both configs). Tests pass. No other consumers import
from the deleted script.
* fix(energy): complete pipeline bootstrap registration per 4-file checklist
High finding from PR review. My earlier PR description claimed
worldmonitor-bootstrap-registration was complete, but I only touched two
of the four registries (api/health.js + api/seed-health.js). The bootstrap
hydration payload itself (api/bootstrap.js) and the shared cache-keys
registry (server/_shared/cache-keys.ts) still had no entry for either
pipeline key, so any consumer that reads bootstrap data would see
pipelinesGas/pipelinesOil as missing on first load.
Files updated this commit:
- api/bootstrap.js — KEYS map + SLOW_KEYS set both gain pipelinesGas +
pipelinesOil. Placed next to sprPolicies (same curated-registry cadence
and tier). Slow tier is correct: weekly cron, not needed on first paint.
- server/_shared/cache-keys.ts — PIPELINES_GAS_KEY + PIPELINES_OIL_KEY
exported constants (matches SPR_POLICIES_KEY pattern), BOOTSTRAP_KEYS map
entries, and BOOTSTRAP_TIERS entries (both 'slow').
Not touched (intentional):
- server/gateway.ts — pipeline data is free-tier per the Energy Atlas
plan; no PREMIUM_RPC_PATHS entry required. Energy Atlas monetization
hooks (scenario runner, MCP tools, subscriptions) are Release 2.
Full 4-file checklist now complete:
✅ server/_shared/cache-keys.ts (this commit)
✅ api/bootstrap.js (this commit)
✅ api/health.js (earlier in PR)
✅ api/seed-health.js (earlier in PR — dual-registry rule)
Typecheck clean (both configs).
* feat(energy): ListPipelines + GetPipelineDetail RPCs with evidence-derived badges
Day 7 of the Energy Atlas Release 1 plan (Week 2). Exposes the pipeline
registries (shipped in Day 6) via two supply-chain RPCs and ships the
evidence-to-badge derivation server-side.
## Proto
proto/worldmonitor/supply_chain/v1/list_pipelines.proto — new:
- ListPipelinesRequest { commodity_type?: 'gas' | 'oil' }
- ListPipelinesResponse { pipelines[], fetched_at, classifier_version, upstream_unavailable }
- GetPipelineDetailRequest { pipeline_id (required, query-param) }
- GetPipelineDetailResponse { pipeline?, revisions[], fetched_at, unavailable }
- PipelineEntry — wire shape mirroring scripts/data/pipelines-{gas,oil}.json
+ a server-derived public_badge field
- PipelineEvidence, OperatorStatement, SanctionRef, LatLon, PipelineRevisionEntry
service.proto adds both rpc methods with HTTP_METHOD_GET + path bindings:
/api/supply-chain/v1/list-pipelines
/api/supply-chain/v1/get-pipeline-detail
`make generate` regenerated src/generated/{client,server}/… + docs/api/
OpenAPI json/yaml.
## Evidence-derivation
server/worldmonitor/supply-chain/v1/_pipeline-evidence.ts — new.
derivePublicBadge(evidence) → 'flowing' | 'reduced' | 'offline' | 'disputed'
is deterministic + versioned (DERIVER_VERSION='badge-deriver-v1').
Rules (first match wins):
1. offline + sanctionRef OR expired/suspended commercial → offline
2. offline + operator statement → offline
3. offline + only press/ais/satellite → disputed (single-source negative claim)
4. reduced → reduced
5. flowing → flowing
6. unknown / malformed → disputed
Staleness guard: non-flowing badges on >14d-old evidence demote to
disputed. Flowing is the optimistic default — stale "still flowing" is
safer than stale "offline". Matches seed-pipelines-{gas,oil}.mjs maxStaleMin.
Tests (tests/pipeline-evidence-derivation.test.mts) — 15 passing cases
covering happy paths, disputed fallbacks, staleness guard, versioning.
## Handlers
server/worldmonitor/supply-chain/v1/list-pipelines.ts
- Reads energy:pipelines:{gas,oil}:v1 via getCachedJson.
- projectPipeline() narrows the Upstash `unknown` into PipelineEntry
shape + calls derivePublicBadge.
- Honors commodity_type filter (skip the opposite registry's Redis read
when the client pre-filters).
- Returns upstream_unavailable=true when BOTH registries miss.
server/worldmonitor/supply-chain/v1/get-pipeline-detail.ts
- Scans both registries by id (ids are globally unique per
tests/pipelines-registry.test.mts).
- Empty revisions[] for now; auto-revision log wires up in Week 3.
handler.ts registers both into supplyChainHandler.
## Gateway
server/gateway.ts adds 'static' cache-tier for both new RPC paths
(registry is slow-moving; 'static' matches the other read-mostly
supply-chain endpoints).
## Consumer wiring
Not in this commit — PipelineStatusPanel (Day 8) is what will call
listPipelines/getPipelineDetail via the generated client. pipelinesGas
+ pipelinesOil stay in PENDING_CONSUMERS until Day 8.
Typecheck clean (both configs). 15 new tests + 17 registry tests all pass.
* feat(energy): PipelineStatusPanel — evidence-backed status table + drawer
Day 8 of the Energy Atlas Release 1 plan. First consumer of the Day 6–7
registries + RPCs.
## What this PR adds
- src/components/PipelineStatusPanel.ts — new panel (id=pipeline-status).
* Bootstrap-hydrates from pipelinesGas + pipelinesOil for instant first
paint; falls through to listPipelines() RPC if bootstrap misses.
Background re-fetch runs on every render so a classifier-version bump
between bootstrap stamp and first view produces a visible update.
* Table rows sorted non-flowing-first (offline / reduced / disputed
before flowing) — what an atlas reader cares about.
* Click-to-expand drawer calls getPipelineDetail() lazily — operator
statements, sanction refs (with clickable source URLs), commercial
state, classifier version + confidence %, capacity + route metadata.
* publicBadge color-chip palette matches the methodology doc.
* Attribution footer with GEM (CC-BY 4.0) credit + classifier version.
- src/components/index.ts — barrel export.
- src/app/panel-layout.ts — import + createPanel('pipeline-status', …).
- src/config/panels.ts — ENERGY_PANELS adds 'pipeline-status' at priority 1.
## PENDING_CONSUMERS cleanup
tests/bootstrap.test.mjs — removes 'pipelinesGas' + 'pipelinesOil' from
the allowlist. The invariant "every bootstrap key has a getHydratedData
consumer" now enforces real wiring for these keys: the panel literally
calls getHydratedData('pipelinesGas') and getHydratedData('pipelinesOil').
Future regressions that remove the consumer will fail pre-push.
## Consumer contract verified
- 67 tests pass including bootstrap.test.mjs consumer coverage check.
- Typecheck clean.
- No DeckGL PathLayer in this commit — existing 'pipelines-layer' has a
separate data source, so modifying DeckGLMap.ts to overlay evidence-
derived badges on the map is a follow-up commit to avoid clobbering.
## Out of scope for Day 8 (next steps on same PR)
- DeckGL PathLayer integration (color pipelines on the main map by
publicBadge, click-to-open this drawer) — Day 8b commit.
- Storage facility registry + StorageFacilityMapPanel — Days 9-10.
* fix(energy): PipelineStatusPanel bootstrap path — client-side badge derivation
High finding from PR review. The Day-8 panel crashed on first paint
whenever bootstrap hydration succeeded, because:
- Bootstrap hydrates raw scripts/data/pipelines-{gas,oil}.json verbatim.
- That JSON does NOT include publicBadge — that field is only added by
the server handler's projectPipeline() in list-pipelines.ts.
- PipelineStatusPanel passed raw entries into badgeChip(), which called
badgeLabel(undefined).charAt(0) → TypeError.
The background RPC refresh that would have repaired the data never ran
because the panel threw before reaching it. So the exact bootstrap path
newly wired in commit
|