mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
044598346e
* feat(seed-contract): PR 2a — runSeed envelope dual-write + 91 seeders migrated
Opt-in contract path in runSeed: when opts.declareRecords is provided, write
{_seed, data} envelope to the canonical key alongside legacy seed-meta:*
(dual-write). State machine: OK / OK_ZERO / RETRY with zeroIsValid opt.
declareRecords throws or returns non-integer → hard fail (contract violation).
extraKeys[*] support per-key declareRecords; each extra key writes its own
envelope. Legacy seeders (no declareRecords) entirely unchanged.
Migrated all 91 scripts/seed-*.mjs to contract mode. Each exports
declareRecords returning the canonical record count, and passes
schemaVersion: 1 + maxStaleMin (matched to api/health.js SEED_META, or 2.5x
interval where no registry entry exists). Contract conformance reports 84/86
seeders with full descriptor (2 pre-existing warnings).
Legacy seed-meta keys still written so unmigrated readers keep working;
follow-up slices flip health.js + readers to envelope-first.
Tests: 61/61 PR 1 tests still pass.
Next slices for PR 2:
- api/health.js registry collapse + 15 seed-bundle-*.mjs canonicalKey wiring
- reader migration (mcp, resilience, aviation, displacement, regional-snapshot)
- direct writers — ais-relay.cjs, consumer-prices-core publish.ts
- public-boundary stripSeedEnvelope + test migration
Plan: docs/plans/2026-04-14-002-fix-runseed-zero-record-lockout-plan.md
* fix(seed-contract): unwrap envelopes in internal cross-seed readers
After PR 2a enveloped 91 canonical keys as {_seed, data}, every script-side
reader that returned the raw parsed JSON started silently handing callers the
envelope instead of the bare payload. WoW baselines (bigmac, grocery-basket,
fear-greed) saw undefined .countries / .composite; seed-climate-anomalies saw
undefined .normals from climate:zone-normals:v1; seed-thermal-escalation saw
undefined .fireDetections from wildfire:fires:v1; seed-forecasts' ~40-key
pipeline batch returned envelopes for every input.
Fix: route every script-side reader through unwrapEnvelope(...).data. Legacy
bare-shape values pass through unchanged (unwrapEnvelope returns
{_seed: null, data: raw} for any non-envelope shape).
Changed:
- scripts/_seed-utils.mjs: import unwrapEnvelope; redisGet, readSeedSnapshot,
verifySeedKey all unwrap. Exported new readCanonicalValue() helper for
cross-seed consumers.
- 18 seed-*.mjs scripts with local redisGet-style helpers or inline fetch
patched to unwrap via the envelope source module (subagent sweep).
- scripts/seed-forecasts.mjs pipeline batch: parse() unwraps each result.
- scripts/seed-energy-spine.mjs redisMget: unwraps each result.
Tests:
- tests/seed-utils-envelope-reads.test.mjs: 7 new cases covering envelope
+ legacy + null paths for readSeedSnapshot and verifySeedKey.
- Full seed suite: 67/67 pass (was 61, +6 new).
Addresses both of user's P1 findings on PR #3097.
* feat(seed-contract): envelope-aware reads in server + api helpers
Every RPC and public-boundary reader now automatically strips _seed from
contract-mode canonical keys. Legacy bare-shape values pass through unchanged
(unwrapEnvelope no-ops on non-envelope shapes).
Changed helpers (one-place fix — unblocks ~60 call sites):
- server/_shared/redis.ts: getRawJson, getCachedJson, getCachedJsonBatch
unwrap by default. cachedFetchJson inherits via getCachedJson.
- api/_upstash-json.js: readJsonFromUpstash unwraps (covers api/mcp.ts
tool responses + all its canonical-key reads).
- api/bootstrap.js: getCachedJsonBatch unwraps (public-boundary —
clients never see envelope metadata).
Left intentionally unchanged:
- api/health.js / api/seed-health.js: read only seed-meta:* keys which
remain bare-shape during dual-write. unwrapEnvelope already imported at
the meta-read boundary (PR 1) as a defensive no-op.
Tests: 67/67 seed tests pass. typecheck + typecheck:api clean.
This is the blast-radius fix the PR #3097 review called out — external
readers that would otherwise see {_seed, data} after the writer side
migrated.
* fix(test): strip export keyword in vm.runInContext'd seed source
cross-source-signals-regulatory.test.mjs loads scripts/seed-cross-source-signals.mjs
via vm.runInContext, which cannot parse ESM `export` syntax. PR 2a added
`export function declareRecords` to every seeder, which broke this test's
static-analysis approach.
Fix: strip the `export` keyword from the declareRecords line in the
preprocessed source string so the function body still evaluates as a plain
declaration.
Full test:data suite: 5307/5307 pass. typecheck + typecheck:api clean.
* feat(seed-contract): consumer-prices publish.ts writes envelopes
Wrap the 5 canonical keys written by consumer-prices-core/src/jobs/publish.ts
(overview, movers:7d/30d, freshness, categories:7d/30d/90d, retailer-spread,
basket-series) in {_seed, data} envelopes. Legacy seed-meta:<key> writes
preserved for dual-write.
Inlined a buildEnvelope helper (10 lines) rather than taking a cross-package
dependency — consumer-prices-core is a standalone npm package. Documented the
four-file parity contract (mjs source, ts mirror, js edge mirror, this copy).
Contract fields: sourceVersion='consumer-prices-core-publish-v1', schemaVersion=1,
state='OK' (recordCount>0) or 'OK_ZERO' (legitimate zero).
Typecheck: no new errors in publish.ts.
* fix(seed-contract): 3 more server-side readers unwrap envelopes
Found during final audit:
- server/worldmonitor/resilience/v1/_shared.ts: resilience score reader
parsed cached GetResilienceScoreResponse raw. Contract-mode seed-resilience-scores
now envelopes those keys.
- server/worldmonitor/resilience/v1/get-resilience-ranking.ts: p05/p95
interval lookup parsed raw from seed-resilience-scores' extra-key path.
- server/worldmonitor/infrastructure/v1/_shared.ts: mgetJson() used for
count-source keys (wildfire:fires:v1, news:insights:v1) which are both
contract-mode now.
All three now unwrap via server/_shared/seed-envelope. Legacy shapes pass
through unchanged.
Typecheck clean.
* feat(seed-contract): ais-relay.cjs direct writes produce envelopes
32 canonical-key write sites in scripts/ais-relay.cjs now produce {_seed, data}
envelopes. Inlined buildEnvelope() (CJS module can't require ESM source) +
envelopeWrite(key, data, ttlSeconds, meta) wrapper. Enveloped keys span market
bootstrap, aviation, cyber-threats, theater-posture, weather-alerts, economic
spending/fred/worldbank, tech-events, corridor-risk, usni-fleet, shipping-stress,
social:reddit, wsb-tickers, pizzint, product-catalog, chokepoint transits,
ucdp-events, satellites, oref.
Left bare (not seeded data keys): seed-meta:* (dual-write legacy),
classifyCacheKey LLM cache, notam:prev-closed-state internal state,
wm:notif:scan-dedup flags.
Updated tests/ucdp-seed-resilience.test.mjs regex to accept both upstashSet
(pre-contract) and envelopeWrite (post-contract) call patterns.
* feat(seed-contract): 15 bundle files add canonicalKey for envelope gate
54 bundle sections across 12 files now declare canonicalKey alongside the
existing seedMetaKey. _bundle-runner.mjs (from PR 1) prefers canonicalKey
when both are present — gates section runs on envelope._seed.fetchedAt
read directly from the data key, eliminating the meta-outlives-data class
of bugs.
Files touched:
- climate (5), derived-signals (2), ecb-eu (3), energy-sources (6),
health (2), imf-extended (4), macro (10), market-backup (9),
portwatch (4), relay-backup (2), resilience-recovery (5), static-ref (2)
Skipped (14 sections, 3 whole bundles): multi-key writers, dynamic
templated keys (displacement year-scoped), or non-runSeed orchestrators
(regional brief cron, resilience-scores' 222-country publish, validation/
benchmark scripts). These continue to use seedMetaKey or their own gate.
seedMetaKey preserved everywhere — dual-write. _bundle-runner.mjs falls
back to legacy when canonicalKey is absent.
All 15 bundles pass node --check. test:data: 5307/5307. typecheck:all: clean.
* fix(seed-contract): 4 PR #3097 review P1s — transform/declareRecords mismatches + envelope leaks
Addresses both P1 findings and the extra-key seed-meta leak surfaced in review:
1. runSeed helper-level invariant: seed-meta:* keys NEVER envelope.
scripts/_seed-utils.mjs exports shouldEnvelopeKey(key) — returns false for
any key starting with 'seed-meta:'. Both atomicPublish (canonical) and
writeExtraKey (extras) gate the envelope wrap through this helper. Fixes
seed-iea-oil-stocks' ANALYSIS_META_EXTRA_KEY silently getting enveloped,
which broke health.js parsing the value as bare {fetchedAt, recordCount}.
Also defends against any future manual writeExtraKey(..., envelopeMeta)
call that happens to target a seed-meta:* key.
2. seed-token-panels canonical + extras fixed.
publishTransform returns data.defi (the defi panel itself, shape {tokens}).
Old declareRecords counted data.defi.tokens + data.ai.tokens + data.other.tokens
on the transformed payload → 0 → RETRY path → canonical market:defi-tokens:v1
never wrote, and because runSeed returned before the extraKeys loop,
market:ai-tokens:v1 + market:other-tokens:v1 stayed stale too.
New: declareRecords counts data.tokens on the transformed shape. AI_KEY +
OTHER_KEY extras reuse the same function (transforms return structurally
identical panels). Added isMain guard so test imports don't fire runSeed.
3. api/product-catalog.js cached reader unwraps envelope.
ais-relay.cjs now envelopes product-catalog:v2 via envelopeWrite(). The
edge reader did raw JSON.parse(result) and returned {_seed, data} to
clients, breaking the cached path. Fix: import unwrapEnvelope from
./_seed-envelope.js, apply after JSON.parse. One site — :238-241 is
downstream of getFromCache(), so the single reader fix covers both.
4. Regression lock tests/seed-contract-transform-regressions.test.mjs (11 cases):
- shouldEnvelopeKey invariant: seed-meta:* false, canonical true
- Token-panels declareRecords works on transformed shape (canonical + both extras)
- Explicit repro of pre-fix buggy signature returning 0 — guards against revert
- resolveRecordCount accepts 0, rejects non-integer
- Product-catalog envelope unwrap returns bare shape; legacy passes through
Verification:
- npm run test:data → 5318/5318 pass (was 5307 — 11 new regressions)
- npm run typecheck:all → clean
- node --check on every modified script
iea-oil-stocks canonical declareRecords was NOT broken (user confirmed during
review — buildIndex preserves .members); only its ANALYSIS_META_EXTRA_KEY
was affected, now covered generically by commit 1's helper invariant.
* fix(seed-contract): seed-token-panels validateFn also runs on post-transform shape
Review finding: fixing declareRecords wasn't sufficient — atomicPublish() runs
validateFn(publishData) on the transformed payload too. seed-token-panels'
validate() checked data.defi/.ai/.other on the transformed {tokens} shape,
returned false, and runSeed took the early skipped-write branch (before even
reaching the declareRecords RETRY logic). Net effect: same as before the
declareRecords fix — canonical + both extras stayed stale.
Fix: validate() now checks the canonical defi panel directly (Array.isArray
(data?.tokens) && has at least one t.price > 0). AI/OTHER panels are validated
implicitly by their own extraKey declareRecords on write.
Audited the other 9 seeders with publishTransform (bls-series, bis-extended,
bis-data, gdelt-intel, trade-flows, iea-oil-stocks, jodi-gas, sanctions-pressure,
forecasts): all validateFn's correctly target the post-transform shape. Only
token-panels regressed.
Added 4 regression tests (tests/seed-contract-transform-regressions.test.mjs):
- validate accepts transformed panel with priced tokens
- validate rejects all-zero-price tokens
- validate rejects empty/missing tokens
- Explicit pre-fix repro (buggy old signature fails on transformed shape)
Verification:
- npm run test:data → 5322/5322 pass (was 5318; +4 new)
- npm run typecheck:all → clean
- node --check clean
* feat(seed-contract): add /api/seed-contract-probe validation endpoint
Single machine-readable gate for 'is PR #3097 working in production'.
Replaces the curl/jq ritual with one authenticated edge call that returns
HTTP 200 ok:true or 503 + failing check list.
What it validates:
- 8 canonical keys have {_seed, data} envelopes with required data fields
and minRecords floors (fsi-eu, zone-normals, 3 token panels + minRecords
guard against token-panels RETRY regression, product-catalog, wildfire,
earthquakes).
- 2 seed-meta:* keys remain BARE (shouldEnvelopeKey invariant; guards
against iea-oil-stocks ANALYSIS_META_EXTRA_KEY-class regressions).
- /api/product-catalog + /api/bootstrap responses contain no '_seed' leak.
Auth: x-probe-secret header must match RELAY_SHARED_SECRET (reuses existing
Vercel↔Railway internal trust boundary).
Probe logic is exported (checkProbe, checkPublicBoundary, DEFAULT_PROBES) for
hermetic testing. tests/seed-contract-probe.test.mjs covers every branch:
envelope pass/fail on field/records/shape, bare pass/fail on shape/field,
missing/malformed JSON, Redis non-2xx, boundary seed-leak detection,
DEFAULT_PROBES sanity (seed-meta invariant present, token-panels minRecords
guard present).
Usage:
curl -H "x-probe-secret: $RELAY_SHARED_SECRET" \
https://api.worldmonitor.app/api/seed-contract-probe
PR 3 will extend the probe with a stricter mode that asserts seed-meta:*
keys are GONE (not just bare) once legacy dual-write is removed.
Verification:
- tests/seed-contract-probe.test.mjs → 15/15 pass
- npm run test:data → 5338/5338 (was 5322; +16 new incl. conformance)
- npm run typecheck:all → clean
* fix(seed-contract): tighten probe — minRecords on AI/OTHER + cache-path source header
Review P2 findings: the probe's stated guards were weaker than advertised.
1. market:ai-tokens:v1 + market:other-tokens:v1 probes claimed to guard the
token-panels extra-key RETRY regression but only checked shape='envelope'
+ dataHas:['tokens']. If an extra-key declareRecords regressed to 0, both
probes would still pass because checkProbe() only inspects _seed.recordCount
when minRecords is set. Now both enforce minRecords: 1.
2. /api/product-catalog boundary check only asserted no '_seed' leak — which
is also true for the static fallback path. A broken cached reader
(getFromCache returning null or throwing) could serve fallback silently
and still pass this probe. Now:
- api/product-catalog.js emits X-Product-Catalog-Source: cache|dodo|fallback
on the response (the json() helper gained an optional source param wired
to each of the three branches).
- checkPublicBoundary declaratively requires that header's value match
'cache' for /api/product-catalog, so a fallback-serve fails the probe
with reason 'source:fallback!=cache' or 'source:missing!=cache'.
Test updates (tests/seed-contract-probe.test.mjs):
- Boundary check reworked to use a BOUNDARY_CHECKS config with optional
requireSourceHeader per endpoint.
- New cases: served-from-cache passes, served-from-fallback fails with source
mismatch, missing header fails, seed-leak still takes precedence, bad
status fails.
- Token-panels sanity test now asserts minRecords≥1 on all 3 panels.
Verification:
- tests/seed-contract-probe.test.mjs → 17/17 pass (was 15, +2 net)
- npm run test:data → 5340/5340
- npm run typecheck:all → clean
259 lines
11 KiB
JavaScript
259 lines
11 KiB
JavaScript
#!/usr/bin/env node
|
|
|
|
import { loadEnvFile, CHROME_UA, getRedisCredentials, runSeed } from './_seed-utils.mjs';
|
|
import { readFileSync } from 'node:fs';
|
|
import { dirname, join } from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
|
|
loadEnvFile(import.meta.url);
|
|
|
|
const __dirname = dirname(fileURLToPath(import.meta.url));
|
|
const CANONICAL_KEY = 'conflict:iran-events:v1';
|
|
|
|
const LOCATION_COORDS = {
|
|
'tehran': { lat: 35.6892, lon: 51.3890 },
|
|
'isfahan': { lat: 32.6546, lon: 51.6680 },
|
|
'shiraz': { lat: 29.5918, lon: 52.5837 },
|
|
'mashhad': { lat: 36.2605, lon: 59.6168 },
|
|
'tabriz': { lat: 38.0800, lon: 46.2919 },
|
|
'ahvaz': { lat: 31.3183, lon: 48.6706 },
|
|
'kermanshah': { lat: 34.3142, lon: 47.0650 },
|
|
'urmia': { lat: 37.5527, lon: 45.0761 },
|
|
'bushehr': { lat: 28.9234, lon: 50.8203 },
|
|
'bandar abbas': { lat: 27.1865, lon: 56.2808 },
|
|
'erbil': { lat: 36.1912, lon: 44.0119 },
|
|
'baghdad': { lat: 33.3152, lon: 44.3661 },
|
|
'basra': { lat: 30.5085, lon: 47.7804 },
|
|
'mosul': { lat: 36.3350, lon: 43.1189 },
|
|
'tel aviv': { lat: 32.0853, lon: 34.7818 },
|
|
'israel': { lat: 31.7683, lon: 35.2137 },
|
|
'negev': { lat: 30.8, lon: 34.8 },
|
|
'manama': { lat: 26.2285, lon: 50.5860 },
|
|
'bahrain': { lat: 26.0667, lon: 50.5577 },
|
|
'kuwait': { lat: 29.3759, lon: 47.9774 },
|
|
'dubai': { lat: 25.2048, lon: 55.2708 },
|
|
'abu dhabi': { lat: 24.4539, lon: 54.3773 },
|
|
'fujairah': { lat: 25.1288, lon: 56.3265 },
|
|
'qatar': { lat: 25.2854, lon: 51.5310 },
|
|
'doha': { lat: 25.2854, lon: 51.5310 },
|
|
'jordan': { lat: 31.9454, lon: 35.9284 },
|
|
'irbid': { lat: 32.5560, lon: 35.8500 },
|
|
'syria': { lat: 34.8021, lon: 38.9968 },
|
|
'daraa': { lat: 32.6189, lon: 36.1021 },
|
|
'cyprus': { lat: 34.7071, lon: 33.0226 },
|
|
'akrotiri': { lat: 34.5839, lon: 32.9879 },
|
|
'hormuz': { lat: 27.0, lon: 56.5 },
|
|
'strait of hormuz': { lat: 26.5, lon: 56.3 },
|
|
'parchin': { lat: 35.5167, lon: 51.7667 },
|
|
'mehrabad': { lat: 35.6892, lon: 51.3134 },
|
|
'paveh': { lat: 35.0442, lon: 46.3558 },
|
|
'poldokhtar': { lat: 33.1517, lon: 47.7133 },
|
|
'azadi': { lat: 35.6997, lon: 51.3380 },
|
|
'kohak': { lat: 35.6000, lon: 51.5000 },
|
|
'zibashir': { lat: 29.55, lon: 52.55 },
|
|
'jam': { lat: 27.82, lon: 52.35 },
|
|
'london': { lat: 51.5074, lon: -0.1278 },
|
|
'azerbaijan': { lat: 40.4093, lon: 49.8671 },
|
|
'baku': { lat: 40.4093, lon: 49.8671 },
|
|
'gibraltar': { lat: 36.1408, lon: -5.3536 },
|
|
'iran': { lat: 32.4279, lon: 53.6880 },
|
|
'iraq': { lat: 33.2232, lon: 43.6793 },
|
|
'saudi': { lat: 24.7136, lon: 46.6753 },
|
|
'uae': { lat: 24.4539, lon: 54.3773 },
|
|
'al udeid': { lat: 25.1173, lon: 51.3150 },
|
|
'jomhouri': { lat: 35.6850, lon: 51.4050 },
|
|
'jurf al-sakhar': { lat: 32.9500, lon: 44.1000 },
|
|
'haji omeran': { lat: 36.6500, lon: 45.0500 },
|
|
'nineveh': { lat: 36.3500, lon: 43.1500 },
|
|
'rashidiya': { lat: 36.4000, lon: 43.1000 },
|
|
'gaza': { lat: 31.3547, lon: 34.3088 },
|
|
'riyadh': { lat: 24.7136, lon: 46.6753 },
|
|
'sulaimaniyah': { lat: 35.5613, lon: 45.4306 },
|
|
'sulaimani': { lat: 35.5613, lon: 45.4306 },
|
|
'haifa': { lat: 32.7940, lon: 34.9896 },
|
|
'karaj': { lat: 35.8400, lon: 50.9391 },
|
|
'shahran': { lat: 35.7900, lon: 51.2900 },
|
|
'kouhak': { lat: 35.6200, lon: 51.4800 },
|
|
'hamadan': { lat: 34.7988, lon: 48.5146 },
|
|
'hamedan': { lat: 34.7988, lon: 48.5146 },
|
|
'yazd': { lat: 31.8974, lon: 54.3569 },
|
|
'kish': { lat: 26.5400, lon: 53.9800 },
|
|
'qazvin': { lat: 36.2688, lon: 50.0041 },
|
|
'najafabad': { lat: 32.6340, lon: 51.3670 },
|
|
'malayer': { lat: 34.2968, lon: 48.8234 },
|
|
'mehran': { lat: 33.1222, lon: 46.1646 },
|
|
'aqaba': { lat: 29.5267, lon: 35.0078 },
|
|
'eilat': { lat: 29.5577, lon: 34.9519 },
|
|
'choman': { lat: 36.6269, lon: 44.8856 },
|
|
'baqer shahr': { lat: 35.5400, lon: 51.3900 },
|
|
'jubail': { lat: 27.0046, lon: 49.6225 },
|
|
'shaybah': { lat: 22.5200, lon: 54.0000 },
|
|
'al dhafra': { lat: 24.2500, lon: 54.5500 },
|
|
'juffair': { lat: 26.2167, lon: 50.6000 },
|
|
'qeshm': { lat: 26.9500, lon: 56.2700 },
|
|
'pakdasht': { lat: 35.4747, lon: 51.6856 },
|
|
'tasluja': { lat: 35.5100, lon: 45.3700 },
|
|
'al-kharj': { lat: 24.1500, lon: 47.3100 },
|
|
'petah tikva': { lat: 32.0841, lon: 34.8878 },
|
|
'beersheba': { lat: 31.2518, lon: 34.7913 },
|
|
'oman': { lat: 23.5880, lon: 58.3829 },
|
|
'oslo': { lat: 59.9139, lon: 10.7522 },
|
|
'norway': { lat: 59.9139, lon: 10.7522 },
|
|
'aghdasiyeh': { lat: 35.7900, lon: 51.4500 },
|
|
'rey': { lat: 35.5959, lon: 51.4350 },
|
|
'beirut': { lat: 33.8938, lon: 35.5018 },
|
|
'azraq': { lat: 31.8300, lon: 36.8300 },
|
|
'yehud': { lat: 32.0333, lon: 34.8833 },
|
|
'sitra': { lat: 26.1547, lon: 50.6028 },
|
|
'sanandaj': { lat: 35.3219, lon: 46.9862 },
|
|
'ma\'ameer': { lat: 26.0500, lon: 50.5200 },
|
|
'northern cyprus': { lat: 35.1856, lon: 33.3823 },
|
|
'borujerd': { lat: 33.8973, lon: 48.7516 },
|
|
'lamerd': { lat: 27.3373, lon: 53.1831 },
|
|
'chabahar': { lat: 25.2919, lon: 60.6430 },
|
|
'shahrekord': { lat: 32.3256, lon: 50.8644 },
|
|
'parand': { lat: 35.4870, lon: 51.0050 },
|
|
'rabat karim': { lat: 35.4700, lon: 51.0700 },
|
|
'shahriar': { lat: 35.6569, lon: 51.0592 },
|
|
'punak': { lat: 35.7600, lon: 51.3600 },
|
|
'bonab': { lat: 37.3404, lon: 46.0561 },
|
|
'ghaniabad': { lat: 35.4500, lon: 51.6500 },
|
|
'beit shemesh': { lat: 31.7469, lon: 34.9876 },
|
|
'bnei brak': { lat: 32.0833, lon: 34.8333 },
|
|
'quneitra': { lat: 33.1260, lon: 35.8240 },
|
|
'khan arnabeh': { lat: 33.1450, lon: 35.8600 },
|
|
'ruwais': { lat: 24.1100, lon: 52.7300 },
|
|
'mehrshahr': { lat: 35.8300, lon: 50.9700 },
|
|
'qaim': { lat: 34.3800, lon: 41.0400 },
|
|
'prince sultan': { lat: 24.0625, lon: 47.5808 },
|
|
'ramat david': { lat: 32.6650, lon: 35.1792 },
|
|
'vietnam': { lat: 14.0583, lon: 108.2772 },
|
|
'south korea': { lat: 35.9078, lon: 127.7669 },
|
|
'ilam': { lat: 33.6374, lon: 46.4227 },
|
|
'kerman': { lat: 30.2839, lon: 57.0834 },
|
|
'lorestan': { lat: 33.4941, lon: 48.3530 },
|
|
'jerusalem': { lat: 31.7683, lon: 35.2137 },
|
|
'fardis': { lat: 35.7230, lon: 50.9875 },
|
|
'marivan': { lat: 35.5269, lon: 46.1761 },
|
|
'salalah': { lat: 17.0151, lon: 54.0924 },
|
|
'palmachim': { lat: 31.8970, lon: 34.7000 },
|
|
'umm qasr': { lat: 30.0362, lon: 47.9298 },
|
|
'al-siba': { lat: 29.8700, lon: 48.6100 },
|
|
'taleghan': { lat: 36.1700, lon: 50.7600 },
|
|
'persian gulf': { lat: 27.0000, lon: 51.5000 },
|
|
'eastern province': { lat: 26.4207, lon: 50.0888 },
|
|
'empty quarter': { lat: 22.5200, lon: 54.0000 },
|
|
'ovadia': { lat: 31.4700, lon: 34.5300 },
|
|
'shin bet': { lat: 31.7683, lon: 35.2137 },
|
|
'kharg': { lat: 29.2635, lon: 50.3273 },
|
|
'qom': { lat: 34.6401, lon: 50.8764 },
|
|
'andisheh': { lat: 35.7050, lon: 51.0000 },
|
|
'ankara': { lat: 39.9334, lon: 32.8597 },
|
|
'minab': { lat: 27.1452, lon: 57.0812 },
|
|
'asaluyeh': { lat: 27.4834, lon: 52.6121 },
|
|
'south pars': { lat: 27.5000, lon: 52.5000 },
|
|
'al-ahmadi': { lat: 29.0769, lon: 48.0839 },
|
|
'mina al-ahmadi': { lat: 29.0769, lon: 48.0839 },
|
|
'mina abdullah': { lat: 28.9600, lon: 48.1600 },
|
|
'kafr qasim': { lat: 32.1161, lon: 34.9750 },
|
|
'taybad': { lat: 35.1884, lon: 60.7814 },
|
|
'sweihan': { lat: 24.4041, lon: 55.3325 },
|
|
'mahshahr': { lat: 30.5630, lon: 49.1980 },
|
|
'bandar khamir': { lat: 26.9592, lon: 55.8981 },
|
|
'shalamcheh': { lat: 30.5244, lon: 47.9297 },
|
|
'ramat gan': { lat: 32.0704, lon: 34.8237 },
|
|
'shuwaikh': { lat: 29.3636, lon: 47.9417 },
|
|
};
|
|
|
|
const CATEGORY_MAP = {
|
|
cat1: 'military',
|
|
cat2: 'international',
|
|
cat6: 'political',
|
|
cat7: 'civil',
|
|
cat9: 'intelligence',
|
|
cat10: 'airstrike',
|
|
cat11: 'defense',
|
|
};
|
|
|
|
function geolocate(title) {
|
|
const lower = title.toLowerCase();
|
|
for (const [name, coords] of Object.entries(LOCATION_COORDS)) {
|
|
if (lower.includes(name)) return { ...coords, locationName: name };
|
|
}
|
|
return { lat: 32.4279, lon: 53.6880, locationName: 'Iran' };
|
|
}
|
|
|
|
function categorizeSeverity(title) {
|
|
const lower = title.toLowerCase();
|
|
if (/killed|dead|casualties|death toll|wounded/.test(lower)) return 'critical';
|
|
if (/airstrike|bombing|missile|explosion|struck|destroyed/.test(lower)) return 'high';
|
|
if (/intercept|defense|sirens|alert/.test(lower)) return 'elevated';
|
|
return 'moderate';
|
|
}
|
|
|
|
function parseRelativeTime(timeStr) {
|
|
const now = Date.now();
|
|
const match = timeStr.match(/(\d+)\s+hours?\s+ago/);
|
|
if (match) return now - parseInt(match[1], 10) * 3600_000;
|
|
const minMatch = timeStr.match(/(\d+)\s+min/);
|
|
if (minMatch) return now - parseInt(minMatch[1], 10) * 60_000;
|
|
if (/a day ago/.test(timeStr)) return now - 86400_000;
|
|
const dayMatch = timeStr.match(/(\d+)\s+days?\s+ago/);
|
|
if (dayMatch) return now - parseInt(dayMatch[1], 10) * 86400_000;
|
|
return now;
|
|
}
|
|
|
|
async function fetchIranEvents() {
|
|
const dataPath = process.argv[2] || join(__dirname, 'data', 'iran-events-latest.json');
|
|
console.log(` Reading from: ${dataPath}`);
|
|
|
|
const raw = JSON.parse(readFileSync(dataPath, 'utf8'));
|
|
const events = raw.filter(e => e.id && e.title);
|
|
|
|
console.log(` Raw events: ${events.length}`);
|
|
|
|
const mapped = events.map(e => {
|
|
const geo = geolocate(e.title);
|
|
const cat = CATEGORY_MAP[e.category] || 'general';
|
|
return {
|
|
id: e.id,
|
|
title: e.title.slice(0, 500),
|
|
category: cat,
|
|
sourceUrl: e.link || '',
|
|
latitude: geo.lat,
|
|
longitude: geo.lon,
|
|
locationName: geo.locationName,
|
|
timestamp: parseRelativeTime(e.time || ''),
|
|
severity: categorizeSeverity(e.title),
|
|
};
|
|
});
|
|
|
|
mapped.sort((a, b) => b.timestamp - a.timestamp);
|
|
|
|
return {
|
|
events: mapped,
|
|
scrapedAt: Date.now(),
|
|
};
|
|
}
|
|
|
|
function validate(data) {
|
|
return Array.isArray(data?.events) && data.events.length >= 1;
|
|
}
|
|
|
|
export function declareRecords(data) {
|
|
return Array.isArray(data?.events) ? data.events.length : 0;
|
|
}
|
|
|
|
runSeed('conflict', 'iran-events', CANONICAL_KEY, fetchIranEvents, {
|
|
validateFn: validate,
|
|
ttlSeconds: 172800,
|
|
sourceVersion: 'liveuamap-manual-v1',
|
|
|
|
declareRecords,
|
|
schemaVersion: 1,
|
|
maxStaleMin: 20160,
|
|
}).catch((err) => {
|
|
const _cause = err.cause ? ` (cause: ${err.cause.message || err.cause.code || err.cause})` : ''; console.error('FATAL:', (err.message || err) + _cause);
|
|
process.exit(0);
|
|
});
|