Commit Graph

2189 Commits

Author SHA1 Message Date
Elie Habib
799178e4ba fix(seeds): increase forecast TTL from 3600s to 4200s (#1625)
TTL (1h) equaled cron interval (1h), leaving zero buffer for container
cold starts. Health saw EMPTY records=0 during the gap between key
expiry and next seed run. 70min TTL covers the cold start window.
2026-03-15 08:23:59 +04:00
Elie Habib
0d330dd8ee fix(forecast): add LLM provider diagnostics for Railway debugging (#1622)
- Log which providers are available before calling
- Log HTTP error response body (shows Groq 429 reason)
- Log when response is empty/short
- Log success with response length
- Log explicit "All providers failed" at end of loop
2026-03-15 03:33:16 +04:00
Elie Habib
45bb87188d fix(health): bump unrestEvents maxStaleMin from 30 to 45 (#1621)
Seed runs every ~15min but maxStaleMin of 30 was too tight,
causing STALE_SEED alerts during slow runs or brief gaps.
2026-03-15 03:30:47 +04:00
Elie Habib
ba3f94133c fix(seeds): add Origin header to warm-ping scripts for CORS bypass (#1619)
Warm-ping requests from Railway IPs were getting Cloudflare bot
challenge HTML instead of JSON. The relay's service-statuses warm-ping
works because it sends Origin: 'https://worldmonitor.app'. Added the
same header to seed-infra.mjs and seed-military-maritime-news.mjs.

Fixes: usniFleet STALE_SEED, cableHealth EMPTY_ON_DEMAND
2026-03-15 03:15:30 +04:00
Elie Habib
94d1ac572d fix(forecast): improve prediction quality for chokepoints, GPS jamming, LLM logging (#1618)
- Supply chain: add severity floor (critical=0.55, high=0.35) and raise
  multiplier from 0.7 to 0.9. Hormuz at 80/100 RED now produces ~60-75%
  instead of 47%.
- GPS jamming: raise normalization range from 30 to 60 hexes, multiplier
  from 0.5 to 0.7, add 10% bonus for 20+ hexes. 53 hexes now produces
  ~72% instead of 49%.
- LLM: log when no API keys configured (was silently skipping all providers)
- Logging: add per-domain breakdown and top-5 predictions to pipeline output
  for Railway log diagnosis
2026-03-15 03:13:37 +04:00
Elie Habib
e2686da713 fix(railway): bump scripts engine to Node 20 (Railpack --expose-gc fix) (#1617)
Railway's Railpack builder sets NODE_OPTIONS=--expose-gc which Node 18
rejects during mise install. Node 20+ accepts this flag. All Railway
seed services using scripts/package.json will now build with Node 20.
2026-03-15 02:50:13 +04:00
Elie Habib
31b9643583 feat(supply-chain): surface CorridorRisk intelligence in chokepoint panel (#1616)
The CorridorRisk API provides rich intelligence that we were storing
but not displaying. Now surfaced in the panel:

- risk_summary: live intelligence narrative shown in the description
  area (e.g. "Armed confrontations are active across the Persian Gulf
  with 52% of events classified as armed clashes")
- risk_report.action: routing recommendation shown when card is
  expanded (e.g. "Recommend REROUTING via Cape of Good Hope for all
  non-essential Gulf cargo")

Changes:
- Proto: add risk_summary and risk_report_action to TransitSummary
- Relay: extract risk_report.action in seedCorridorRisk, pass both
  fields through seedTransitSummaries
- Handler: pass through to API response + include in description
- UI: riskSummary in risk row, riskReportAction in expanded view
2026-03-15 02:40:33 +04:00
Elie Habib
c0f8407ca7 fix(economic): WALCL change displayed in millions instead of billions (#1615)
Fed Total Assets (WALCL) display value was correctly divided by 1000
to show billions, but the change delta was left in raw millions.
A $17.45B weekly change showed as +17450$B. Now divides change by
1000 to match the display unit.
2026-03-15 02:34:12 +04:00
Elie Habib
7e7ce79ad2 fix(ui): hide zero warnings/disruptions when no signals present (#1614)
"0 warning(s), 0 AIS disruption(s)" adds no information and
contradicts the actual intelligence (e.g. 95% traffic drop at
Hormuz). Now only shows the warnings/disruptions line when at
least one is non-zero. Directions still display on their own.
2026-03-15 02:25:59 +04:00
Elie Habib
ebf46ec7e7 fix(chart): use app design system colors instead of arbitrary values (#1613)
Tanker line: --semantic-info (#3b82f6 blue) instead of #4fc3f7
Cargo line: --semantic-high (#ff8800 orange) instead of #ff9800
Grid: --border (#2a2a2a) instead of --border-subtle
Endpoint dot stroke: --panel-bg (#141414) instead of --bg-primary
2026-03-15 02:23:27 +04:00
Elie Habib
b7079fb145 refactor: dedupe rss proxy domain and header checks (#1612) 2026-03-15 02:22:28 +04:00
Elie Habib
33f15a26a9 refactor(chart): replace lightweight-charts with Canvas 2D transit chart (#1609)
Remove TradingView's lightweight-charts dependency (~60KB) and replace
with a self-contained Canvas 2D implementation. Fixes:

- TradingView logo/branding and redirect links removed
- Chart data no longer hidden by legend labels (Cargo/Tanker overlapping March data)
- Shows last 60 days instead of full 6-month range
- Uses app theme CSS variables (--text-dim, --border-subtle, --accent-primary)
- Hover tooltips showing date + both values
- Bottom legend with colored dots and current values
- Highlighted endpoint dots for latest data
- Responds to theme-changed events
- ResizeObserver for responsive width
2026-03-15 02:09:33 +04:00
Elie Habib
3cbf1169a8 fix(supply-chain): add CorridorRisk diagnostic logging and HTML detection (#1611)
corridorrisk: EMPTY in health despite relay running new code. The seed
produces zero log output, making it impossible to diagnose. Added:
- Log fetch start ("Fetching...") and in-flight skip
- Log HTTP error with response body and content-type
- Detect HTML responses (Cloudflare challenge) before JSON.parse
- Increase timeout from 10s to 15s for slow Railway regions
2026-03-15 02:09:17 +04:00
Elie Habib
ca451c732d fix: add forecast RPC_CACHE_TIER entry, fix transitSummary test regex (#1610)
- Add /api/forecast/v1/get-forecasts to RPC_CACHE_TIER as 'medium'
  (route-cache-tier test requires every GET route has explicit entry)
- Fix transitSummary test regex to accept optional field syntax (?:)
  from proto codegen v0.7.0
2026-03-15 02:02:37 +04:00
Elie Habib
45f5e5a457 feat(forecast): AI Forecasts prediction module (#1579)
* feat(forecast): add AI Forecasts prediction module (Pro-tier)

MiroFish-inspired prediction engine that generates structured forecasts
across 6 domains (conflict, market, supply chain, political, military,
infrastructure) using existing WorldMonitor data streams.

- Proto definitions for ForecastService with GetForecasts RPC
- Dedicated seed script (seed-forecasts.mjs) with 6 domain detectors,
  cross-domain cascade resolver, prediction market calibration, and
  trend detection via prior snapshot comparison
- Premium-gated RPC handler (PREMIUM_RPC_PATHS enforcement)
- Lazy-loaded ForecastPanel with domain filters, probability bars,
  trend arrows, signal evidence, and cascade links
- Health monitoring integration (seed-meta freshness tracking)
- Refresh scheduler with API key guard

* test(forecast): add 47 unit tests for forecast detectors and utilities

Covers forecastId, normalize, resolveCascades, calibrateWithMarkets,
computeTrends, and smoke tests for all 6 domain detectors. Exports
testable functions from seed script with direct-run guard.

* fix(forecast): domain mismatch 'infra' vs 'infrastructure', add panel category

- Seed script used 'infra' but ForecastPanel filtered on 'infrastructure',
  causing Infra tab to show zero results
- Added 'forecast' to intelligence category in PANEL_CATEGORY_MAP

* fix(forecast): move CSS to one-time injection, improve type safety

- P2: Move style block from setContent to one-time document.head injection
  to prevent CSS accumulation on repeated renders
- P3: Replace +toFixed(3) with Math.round for readability in seed script
- P3: Use Forecast type instead of any[] in RPC handler filter

* fix(forecast): handle sebuf proto data shapes from Redis

Detectors now normalize CII scores from server-side proto format
(combinedScore, TREND_DIRECTION_RISING, region) to uniform shape.
Outage severity handles proto enum format (SEVERITY_LEVEL_HIGH).
Added confidence floor of 0.3 for single-source predictions.

Verified against live Redis: 2 predictions generated (Iran infra
shutdown, IL political instability).

* feat(forecast): unlock AI Forecasts on web, lock desktop only (trial)

- Remove forecast RPC from PREMIUM_RPC_PATHS (web access is free)
- Panel locked on desktop only (same as oref-sirens/telegram-intel)
- Remove API key guards from data-loader and refresh scheduler
- Web users get full access during trial period

* chore: regenerate proto types with make generate

Re-ran make generate after rebasing on main. Plugin v0.7.0 dropped
@ts-nocheck from output, added it back to all 50 generated files.
Fixed 4 type errors from proto codegen changes:
- MarketSource enum -> string union type
- TemporalAnomalyProto -> TemporalAnomaly rename
- webcam lastUpdated number -> string

* fix(forecast): use chokepoints v4 key, include ciiContribution in unrest

- P1: Switch chokepoints input from stale v2 to active v4 Redis key,
  matching bootstrap.js and cache-keys.ts
- P2: Add ciiContribution to unrest component fallback chain in
  normalizeCiiEntry so political detector reads the correct sebuf field

* feat(forecast): Phase 2 LLM scenario enrichment + confidence model

MiroFish-inspired enhancements:
- LLM scenario narratives via Groq/OpenRouter (narrative-only, no numeric
  adjustment). Evidence-grounded prompts with mandatory signal citation
  and few-shot examples from MiroFish's SECTION_SYSTEM_PROMPT_TEMPLATE.
- Top-4 predictions batched into single LLM call for cost efficiency.
- News context from newsInsights attached to all predictions for LLM
  prompt grounding (NOT in signals, cannot affect confidence).
- Deterministic confidence model: source diversity via SIGNAL_TO_SOURCE
  mapping (deduplicates cii+cii_delta, theater+indicators) + calibration
  agreement from prediction market drift. Floor 0.2, ceiling 1.0.
- Output validation: rejects scenarios without signal references.
- Truncated JSON repair for small model output.
- Structured JSON logging for LLM calls.
- Redis cache for LLM scenarios (1h TTL).
- 23 new tests (70 total), all passing.
- Live-tested: OpenRouter gemini-2.5-flash produces evidence-grounded
  scenario narratives from real WorldMonitor data.

* feat(forecast): Phase 3 multi-perspective scenarios, projections, data-driven cascades

MiroFish-inspired enhancements:
- Multi-perspective LLM analysis: top-2 predictions get strategic,
  regional, and contrarian viewpoints via combined LLM call
- Probability projections: domain-specific decay curves (h24/d7/d30)
  anchored to timeHorizon so probability equals projections[timeHorizon]
- Data-driven cascade rules: moved from hardcoded array to JSON config
  (scripts/data/cascade-rules.json) with schema validation, named
  predicate evaluators, unknown key rejection, and fallback to defaults
- 4 new cascade paths: infrastructure->supply_chain, infrastructure->market
  (both requiresSeverity:total), conflict->political, political->market
- Proto: added Perspectives and Projections messages to Forecast
- ForecastPanel: renders projections row and conditional perspectives toggle
- 89 tests (19 new), all passing
- Live-tested: OpenRouter produces perspectives from real data

* feat(forecast): Phase 4 data utilization + entity graph

Fixes data gaps that prevented 4 of 6 detectors from firing:
- Input normalizers: chokepoint v4 shape + GPS hexes-to-zones mapping
- Chokepoint warm-ping (production-only, requires WM_API_BASE_URL)
- Lowered CII conflict threshold from 70 to 60, gated on level=high|critical

4 new standalone detectors:
- UCDP conflict zones (10+ events per country)
- Cyber threat concentration (5+ threats per country)
- GPS jamming in maritime shipping zones (5 regions)
- Prediction markets as signals (60-90% probability markets)

Entity-relationship graph (file-based, 38 nodes):
- Countries, theaters, commodities, chokepoints, alliances
- Alias table resolves both ISO codes and display names
- Graph cascade discovery links predictions across entities

Result: 51 predictions (up from 1-2), spanning conflict, infrastructure,
and supply chain domains. 112 tests, all passing.

* fix(forecast): redis cache format, signal source mapping, type safety

Fresh-eyes audit fixes:
- BUG: redisSet used wrong Upstash API format (POST body with {value,ex}
  instead of command array ['SET',key,value,'EX',ttl]). LLM cache writes
  were silently failing, causing fresh LLM calls every run.
- BUG: prediction_market signal type missing from SIGNAL_TO_SOURCE,
  inflating confidence for market-derived predictions.
- CLEANUP: Remove unnecessary (f as any) casts in ForecastPanel since
  generated Forecast type already has projections/perspectives fields.
- CLEANUP: Bump health maxStaleMin from 60 to 90 to avoid false STALE
  alerts when LLM calls add latency to seed runs.

* feat(forecast): headline-entity matching with news corroboration signals

Uses entity graph aliases to match headlines to predictions by
country/theater (excludes commodity/infrastructure nodes to prevent
false positives). Predictions with matching headlines get a
news_corroboration signal visible in the panel.

Also fixes buildUserPrompt to merge unique headlines from ALL
predictions in the LLM batch (was only reading preds[0].newsContext).

Live-tested: 13 of 51 predictions now have corroborating headlines
(Iran, Israel, Syria, Ukraine, etc). 116 tests, all passing.

* feat(forecast): add country-codes.json for headline-entity matching

56 countries with ISO codes, full names, and scoring keywords (extracted
from src/config/countries.ts + UCDP-relevant additions). Used by
attachNewsContext for richer headline matching via getSearchTermsForRegion
which combines country-codes + entity graph + keyword aliases.

14/57 predictions now have news corroboration (limited by headline
coverage, not matching quality: only 8 headlines currently available).

* feat(forecast): read 300 headlines from news digest instead of 8

Read news:digest:v1:full:en (300 headlines across 16 categories) instead
of just news:insights:v1 topStories (8 headlines). Fallback to topStories
if digest is unavailable.

Result: news corroboration jumped from 25% to 64% (38/59 predictions).

* fix(forecast): handle parenthetical country names in headline matching

Strip suffixes like '(Zaire)', '(Burma)', '(Soviet Union)' from UCDP
region names before matching against country-codes.json. Also use
includes() for reverse name lookup to catch partial matches.

Corroboration: 64% -> 69% (41/59). Remaining 18 unmatched are countries
with no current English-language news coverage.

* fix(forecast): cache validated LLM output, add digest test, log cache errors

Fresh-eyes audit fixes:
- Combined LLM cache now stores only validated items (was caching raw
  unvalidated output, serving potentially invalid scenarios on cache hit)
- redisSet logs warnings on failure (was silently swallowing all errors)
- Added digest-based test for attachNewsContext (primary path was untested)
- Fixed test arity: attachNewsContext(preds, news, digest) with 3 params

* fix(forecast): remove dead confidenceFromSources, reduce warm-ping timeout

- P2: Remove confidenceFromSources (dead code, computeConfidence overwrites
  all initial confidence values). Inline the formula in original detectors.
- P3: Reduce warm-ping timeout from 30s to 15s (non-critical step)
- P3: Add trial status comment on forecast panel config

* fix(forecast): resolve ISO codes to country names, fix market detector, safe pre-push

P1 fixes from code review:
- CII ISO codes (IL, IR) now resolved to full country names (Israel, Iran)
  via country-codes.json. Prevents substring false positives (IL matching
  Chile) in event correlation. Uses word-boundary regex for matching.
- Market detector CII-to-theater mapping now uses entity graph traversal
  instead of broken theater-name substring matching. Iran correctly maps
  to Middle East theater via graph links.
- Pre-push hook no longer runs destructive git checkout on proto freshness
  failure. Reports mismatch and exits without modifying worktree.
2026-03-15 01:42:04 +04:00
Elie Habib
3f24169a05 fix(health): align shipping maxStaleMin with 6h cron interval (#1608)
Shipping seed cron runs every 6h (360min) but maxStaleMin was 240min
(4h), causing STALE alerts for 2h out of every 6h cycle.
Increased to 420min (7h) to provide buffer for slow runs.
2026-03-15 01:31:30 +04:00
Elie Habib
f209c11713 fix(seeds): rethrow non-fetch failures, separate publish errors (#1606)
* fix(seeds): rethrow non-fetch failures in runSeed()

Split runSeed() into two phases so only upstream fetch errors get
the graceful TTL-extension path. Redis publish, seed-meta, and
verification failures now rethrow (exit 1) so monitoring catches them.

* fix(seeds): separate fetch from publish errors in standalone scripts

Split seed-airport-delays, seed-military-flights, and
seed-service-statuses into two phases matching runSeed() pattern:
- Phase 1: upstream fetch errors are graceful (extend TTL, exit 0)
- Phase 2: Redis publish/verify errors propagate (exit 1)

* fix(seeds): make Redis SET throw on failure so publish errors propagate

Local redisSet() returned false instead of throwing, silently masking
Redis write failures. writeExtraKey() also warned instead of throwing.
Both now throw on non-OK responses, ensuring Phase 2 catch fires.

* fix(seed): treat empty Redis key after successful RPC as publish failure

When cachedFetchJson() silently swallows a Redis write failure, the
warm-ping script now throws instead of warning, reaching the outer
catch handler (exit 1) so monitoring detects the issue.
2026-03-15 01:30:54 +04:00
Elie Habib
39cf56dd4d perf: reduce ~14M uncached API calls/day via client caches + workbox fix + USNI Railway migration (#1605)
* perf: reduce uncached API calls via client-side circuit breaker caches

Add client-side circuit breaker caches with IndexedDB persistence to the
top 3 uncached API endpoints (CF analytics: 10.5M uncached requests/day):

- classify-events (5.37M/day): 6hr cache per normalized title, shouldCache
  guards against caching null/transient failures
- get-population-exposure (3.45M/day): 6hr cache per coordinate key
  (toFixed(4) for ~11m precision), 64-entry LRU
- summarize-article (1.68M/day): 2hr cache per headline-set hash via
  buildSummaryCacheKey, eliminates both cache-check and summarize RPCs

Fix workbox-*.js getting no-cache headers (3.62M/day): exclude from SPA
catch-all regex in vercel.json, add explicit immutable cache rule for
content-hashed workbox files.

Migrate USNI fleet fetch from Vercel edge to Railway relay (gold standard):
- Add seedUSNIFleet() loop to ais-relay.cjs (6hr interval, gzip support)
- Make server handler Redis-read-only (435 lines reduced to 38)
- Move usniFleet from ON_DEMAND to BOOTSTRAP_KEYS in health.js
- Add persistCache + shouldCache to client breaker

Estimated reduction: ~14.3M uncached requests/day.

* fix: address code review findings (P1 + P2)

P1: Include SummarizeOptions in summary cache key to prevent cross-option
cache pollution (e.g. cloud summary replayed after user disables cloud LLMs).

P2: Document that forceRefresh is intentionally ignored now that USNI
fetching moved to Railway relay (Vercel is Redis-read-only).

* fix: reject forceRefresh explicitly instead of silently ignoring it

Return an error response with explanation when forceRefresh=true is sent,
rather than silently returning cached data. Makes the behavior regression
visible to any caller instead of masking it.

* fix(build): set worker.format to 'es' for Vite 6 compatibility

Vite 6 defaults worker.format to 'iife' which fails with code-splitting
workers (analysis.worker.ts uses dynamic imports). Setting 'es' fixes
the Vercel production build.

* fix(test): update deploy-config test for workbox regex exclusion

The SPA catch-all regex test hard-coded the old pattern without the
workbox exclusion. Update to match the new vercel.json source pattern.
2026-03-15 00:52:10 +04:00
Elie Habib
485d416065 feat(seeds): Railway seed scripts for all unseeded Vercel RPC endpoints (#1599)
* feat(seeds): add Railway seed scripts for economic and trade endpoints

Two new seed scripts to eliminate Vercel edge external API calls:

seed-economy.mjs:
- EIA energy prices (WTI, Brent) -> economic:energy:v1:all
- EIA energy capacity (Solar, Wind, Coal) -> economic:capacity:v1:COL,SUN,WND:20
- FRED series (10 series) -> economic:fred:v1:<id>:120
- Macro signals (Yahoo, Alternative.me, Mempool) -> economic:macro-signals:v1

seed-supply-chain-trade.mjs:
- Shipping rates (FRED) -> supply_chain:shipping:v2
- Trade barriers (WTO tariff gap) -> trade:barriers:v1:tariff-gap:50
- Trade restrictions (WTO MFN overview) -> trade:restrictions:v1:tariff-overview:50
- Trade flows (WTO, 15 major reporters) -> trade:flows:v1:<reporter>:000:10
- Tariff trends (WTO, 15 major reporters) -> trade:tariffs:v1:<reporter>:all:10

Cache keys match handler patterns exactly so cachedFetchJson finds
pre-seeded data and avoids live external API calls from Vercel edge.

* feat(seeds): add seed-aviation.mjs for airport ops and aviation news

Seeds 2 aviation endpoints with predictable default params:
- getAirportOpsSummary (AviationStack + NOTAM) -> aviation:ops-summary:v1:CDG,ESB,FRA,IST,LHR,SAW
- listAviationNews (9 RSS feeds, 24h window) -> aviation:news::24:v1

NOT seeded (inherently on-demand, user-specific inputs):
- getFlightStatus: specific flight number lookup
- trackAircraft: bounding-box or icao24 queries
- listAirportFlights: arbitrary airport+direction+limit combos
- getCarrierOps: depends on listAirportFlights with variable params

* feat(seeds): add seed-conflict-intel.mjs for ACLED, HAPI, and PizzINT

Seeds 3 conflict/intelligence endpoints with predictable default params:
- listAcledEvents (all countries, last 30 days) -> conflict:acled:v1:all:0:0
- getHumanitarianSummary (20 top conflict countries) -> conflict:humanitarian:v1:<CC>
- getPizzintStatus (base + GDELT variants) -> intel:pizzint:v1:base, intel:pizzint:v1:gdelt

NOT seeded (inherently on-demand, LLM or user-specific inputs):
- classifyEvent: per-headline LLM classification
- deductSituation: per-query LLM deduction
- getCountryIntelBrief: per-country LLM brief with context hash
- getCountryFacts: per-country REST Countries + Wikidata + Wikipedia
- searchGdeltDocuments: per-query GDELT search

Requires: ACLED_EMAIL, ACLED_KEY, UPSTASH_REDIS_REST_URL/TOKEN

* feat(seeds): add seed-research.mjs for arXiv, HN, tech events, trending repos

Seeds 4 research endpoints:
- listArxivPapers (cs.AI, cs.CL, cs.CR) -> research:arxiv:v1:<cat>::50
- listHackernewsItems (top, best feeds) -> research:hackernews:v1:<feed>:30
- listTechEvents (Techmeme ICS + dev.events RSS) -> research:tech-events:v1
- listTrendingRepos (python, javascript, typescript) -> research:trending:v1:<lang>:daily:50

Tech events key is also seeded by the relay, this script provides backup
hydration and ensures the key is warm even if relay hasn't run yet.

Requires: UPSTASH_REDIS_REST_URL/TOKEN

* feat(seeds): add seed-military-maritime-news.mjs for USNI and nav warnings

Seeds 2 endpoints with predictable default params:
- USNI Fleet Report (WordPress JSON API) -> usni-fleet:sebuf:v1 + stale backup
- Navigational Warnings (NGA broadcast, all areas) -> maritime:navwarnings:v1:all

NOT seeded (inherently on-demand):
- getAircraftDetails/batch: per-icao24 Wingbits lookup
- listMilitaryFlights: bounding-box query (quantized 1-degree grid)
- getVesselSnapshot: in-memory cache, reads from relay /ais-snapshot
- listFeedDigest: per-feed-URL RSS caching (hundreds of feeds, relay proxied)
- summarizeArticle: per-article LLM summarization

Requires: UPSTASH_REDIS_REST_URL/TOKEN

* feat(seeds): add seed-infra.mjs warm-ping for service statuses and cable health

Uses warm-ping pattern (calls Vercel RPC from Railway) because:
- list-service-statuses: 30 status page parsers with 8 custom formats
- get-cable-health: NGA text analysis with cable name matching + proximity
Replicating this logic in a standalone script is fragile and duplicative.

NOT seeded (on-demand):
- search-imagery: per-bbox/datetime STAC query
- get-giving-summary: hardcoded baselines, no external fetches
- get-webcam-image: per-webcamId Windy API lookup

* fix(seeds): move secondary key writes before process.exit, fix data shapes

Critical bugs found in code review:

1. runSeed() calls process.exit(0) after primary key write, so .then()
   callbacks were dead code. All secondary keys (FRED, macro signals,
   trade data, HAPI summaries, pizzint, HN, trending, etc.) were NEVER
   written. Fix: move writeExtraKey calls inside fetchAll() before return.

2. FRED cache key used :120 suffix but handler default is :0 (req.limit||0).
   Fixed to :0 so seed matches handler cache key for default requests.

3. USNI and nav warnings seed parsers produced wrong data shapes vs handler
   (different field names, missing fields). Converted to warm-ping pattern
   (like seed-infra.mjs) to avoid shape divergence.

* fix(seeds): reduce GDELT 429 rate limiting in seed-gdelt-intel

Problems from logs: every topic fetch hits 429, runs take 3-5min,
4th run failed fatally after 12min of cascading retries.

Fixes:
- Increase inter-topic delay: 12s -> 20s (GDELT needs longer cooldown)
- Increase initial backoff: 10s -> 20s, with 15s increments per retry
- Graceful degradation: exhausted retries return empty topic instead of
  throwing (prevents withRetry from restarting ALL topics from scratch)
- Align TTL with health.js: 3600s -> 7200s (matches maxStaleMin:120)
- Validation allows partial success (3/6 topics minimum)

Cron interval should also be increased from 30min to 2h on Railway
to match the new 2h TTL.

* fix(seeds): 4 bugs from review - ACLED auth, NOTAM key, infra precedence, curated events

P1: ACLED auth used wrong endpoint (api/acled/token) and env vars (ACLED_KEY).
Fixed to match server/acled-auth.ts: ACLED_EMAIL+ACLED_PASSWORD via /oauth/token,
with ACLED_ACCESS_TOKEN static fallback.

P1: Aviation NOTAM key was aviation:notam-closures:v1, handler reads
aviation:notam:closures:v2. Fixed key to match _shared.ts.

P2: Infra warm-ping had operator precedence bug in nullish coalescing:
(a ?? b) ? c : d instead of a ?? (b ? c : d). Added parens.

P2: Research seed missed curated conferences that the handler appends
(CURATED_EVENTS in list-tech-events.ts). Added same curated events so
seeded data matches what the handler would produce.

* fix(seeds): add seed-meta freshness metadata for all secondary keys

Added writeExtraKeyWithMeta() to _seed-utils.mjs that writes both the
data key and a seed-meta:<key> freshness metadata entry. All secondary
key writes in seed scripts now use this helper so health.js can track
freshness for: energy capacity, FRED series, macro signals, trade
barriers/restrictions/flows/tariffs, aviation news, HAPI summaries,
PizzINT, arXiv categories, HN feeds, tech events, trending repos.

Previously only the primary key per script got seed-meta (via runSeed),
leaving secondary keys operationally invisible to health monitoring.

* fix(seeds): align seed-meta keys with health.js conventions

P1: writeExtraKeyWithMeta wrote seed-meta:<full-cache-key> (e.g.,
seed-meta:economic:macro-signals:v1), but health.js expects normalized
names without version suffixes (seed-meta:economic:macro-signals).
Fixed by stripping trailing :v\d+ from key. Added metaKeyOverride
param for cases needing explicit control.

P1: shipping seed used runSeed('supply-chain', 'shipping-trade', ...)
producing seed-meta:supply-chain:shipping-trade, but health.js expects
seed-meta:supply_chain:shipping. Fixed domain/resource to match.

* fix(seeds): only write seed-meta after successful data key write

writeExtraKey() now returns false on failure. writeExtraKeyWithMeta()
skips seed-meta write when the data write fails, preventing false-positive
health reports for keys like macro-signals and tech-events.
2026-03-15 00:37:31 +04:00
Elie Habib
19ee1f38e4 fix(seeds): extend TTL on stale data instead of crashing on fetch errors (#1600)
* fix(seeds): extend TTL on stale data instead of crashing on fetch errors

Seed scripts crashed with process.exit(1) when upstream APIs returned
errors (e.g., Wingbits 401), causing Redis keys to expire and panels
to lose data. Now all seeds gracefully extend TTL on existing keys and
exit 0, keeping stale data alive until the API recovers.

- Add shared extendExistingTtl() helper to _seed-utils.mjs
- Update runSeed() catch block (fixes 24 scripts using it)
- Fix fetch-gpsjam.mjs, seed-airport-delays.mjs,
  seed-military-flights.mjs, seed-service-statuses.mjs

* fix(seeds): preserve per-key TTLs when extending stale military data

THEATER_POSTURE_BACKUP_KEY has a 7-day TTL (604800s) but was being
extended with STALE_TTL (86400s), shortening it from 7 days to 1 day
during upstream outages. Now each key group gets its original TTL.
2026-03-14 23:42:30 +04:00
Elie Habib
80c51f149a refactor: dedupe company enrichment headers and iso date formatting (#1603) 2026-03-14 23:41:54 +04:00
Elie Habib
48c86842ac fix(test): update CorridorRisk tests for open beta API (no auth) (#1604)
PR #1598 switched CorridorRisk to the open beta API at
corridorrisk.io/api/corridors (no API key required). The old tests
expected CORRIDOR_RISK_API_KEY and Bearer token auth, causing 4 CI
failures. Updated to match the new endpoint and score-based risk
level derivation.
2026-03-14 23:41:28 +04:00
Elie Habib
13bb3ef080 fix(supply-chain): increase Redis timeout for PortWatch and remove content height cap (#1598)
* fix(supply-chain): increase Redis timeout for PortWatch and remove content height cap

Root cause: getCachedJson has a 1500ms timeout, but the PortWatch
payload (~149KB for 13 chokepoints x 175 days) exceeds this on
high-latency Edge regions. The fetch silently times out and returns
null, so the handler builds responses with empty transit summaries.

Fix: add optional timeoutMs param to getCachedJson, use 5000ms for
the PortWatch fetch. Also remove the 300px max-height on
.economic-content so the Supply Chain panel fills available height.

* refactor(supply-chain): move transit summary assembly to Railway relay

Vercel Edge was reading 3 large Redis keys (PortWatch 149KB, transit
counts, CorridorRisk) and assembling transit summaries on every request.
The 1500ms Redis timeout caused the 149KB PortWatch fetch to silently
fail on high-latency Edge regions (Mumbai bom1), leaving all transit
data empty.

Now Railway builds the pre-assembled transit summaries (including
anomaly detection) and writes them to a single key. Vercel reads
ONE small pre-built key instead of 3 raw keys.

Flow: Railway seeds PortWatch + transit counts -> builds summaries ->
writes supply_chain:transit-summaries:v1 -> Vercel reads it.

This follows the gold standard: "Vercel reads Redis ONLY; Railway
makes ALL external API calls and data assembly."

* test(supply-chain): add sync tests for relay threat levels and name mappings

detectTrafficAnomalyRelay and CHOKEPOINT_THREAT_LEVELS in the relay are
duplicated from _scoring.mjs and get-chokepoint-status.ts because
ais-relay.cjs is CJS. Added sync tests that validate:
- Every canonical chokepoint has a relay threat level
- Relay threat levels match handler CHOKEPOINTS config
- RELAY_NAME_TO_ID covers all canonical chokepoints

This catches drift between the two source-of-truth files.

* fix(ui): restore bounded scroll on economic-content with flex layout

The previous fix replaced max-height: 300px with flex: 1 1 auto, but
.panel-content was not a flex container so the flex rule was ignored.
This caused tabs to scroll away with the content.

Fix: use :has(.economic-content) to make .panel-content a flex column
only for panels with tabbed economic content. Tabs stay pinned, content
area scrolls independently.

* feat(supply-chain): fix CorridorRisk API integration (open beta, no key needed)

The CorridorRisk API is in open beta at corridorrisk.io/api/corridors
(not api.corridorrisk.io/v1/corridors). No API key required during beta.

Changes:
- Fix URL to corridorrisk.io/api/corridors
- Remove API key requirement (open beta)
- Update name matching for actual API names (e.g. "Persian Gulf &
  Strait of Hormuz" -> hormuz_strait)
- Derive riskLevel from score (>=70 critical, >=50 high, etc.)
- Store riskScore, vesselCount, eventCount7d, riskSummary
- Feed CorridorRisk data into transit summaries

* test(supply-chain): comprehensive transit summary integration tests

75 tests across 10 suites covering:
- Relay seedTransitSummaries assembly (Redis key, fields, triggers)
- CorridorRisk name mapping and risk level derivation from score
- Handler reads pre-built summaries (not raw upstream keys)
- Handler isolation: no PortWatchData/CorridorRiskData/CANONICAL_CHOKEPOINTS imports
- detectTrafficAnomalyRelay sync with _scoring.mjs (side-by-side execution)
- detectTrafficAnomaly edge cases (boundaries, threat levels, unsorted history)
- CHOKEPOINT_THREAT_LEVELS relay-handler sync validation

* fix(supply-chain): hydrate transit summaries from Redis on relay restart

After relay restart, latestPortwatchData and latestCorridorRiskData are
null. The initial seedTransitSummaries call (35s after boot) would
return early with no data, leaving the transit-summaries:v1 key stale
until the next PortWatch seed completes (6+ seconds later).

Fix: seedTransitSummaries now reads persisted PortWatch and CorridorRisk
data from Redis when in-memory state is empty. This covers the cold-start
gap so Vercel always has fresh transit summaries.

Also adds 5 tests validating the hydration path order and assignment.

* fix(supply-chain): add fallback to raw Redis keys when pre-built summaries are empty

P1: If supply_chain:transit-summaries:v1 is absent (relay not deployed,
restart in progress, or transient PortWatch failure), the handler now
falls back to reading the raw portwatch, corridorrisk, and transit count
keys directly and assembling summaries on the fly.

This ensures corridor risk data (riskLevel, incidentCount7d, disruptionPct)
is never silently zeroed out, and users keep history/counts even during
the 6-hour PortWatch re-seed window.

Strategy: pre-built summaries (fast path) -> raw keys fallback (slow path)
-> all-zero defaults (last resort).
2026-03-14 23:27:27 +04:00
Elie Habib
519ae55980 feat(supply-chain): detect AIS dark-transit anomalies in war zones (#1595)
* feat(supply-chain): detect AIS dark-transit anomalies in war zones

When PortWatch history shows >50% traffic drop in war_zone or critical
chokepoints, surface it as intelligence: "Traffic down X% vs 30-day
baseline — vessels may be transiting dark (AIS off)".

The absence of AIS signals in conflict zones like Hormuz is itself a
signal (vessels disabling transponders to avoid targeting).

Changes:
- Add detectTrafficAnomaly() comparing 7-day vs 30-day baseline
- Boost disruption score by 10 when traffic anomaly detected
- Show WoW% from PortWatch even when real-time AIS counts are 0
- 6 new tests for anomaly detection edge cases

* fix(supply-chain): clamp disruptionScore to 100 and dedupe anomaly function

P1: disruptionScore could exceed 100 when anomalyBonus was added on top
of a max-score base, rendering "110/100" in the UI. Now clamped before
assignment, not just for status.

P2: detectTrafficAnomaly was duplicated in test file, so regressions in
the real code path would go undetected. Moved function into _scoring.mjs
(pure, no server deps). Both handler and tests import the same function.

* fix(supply-chain): require 37 days for traffic anomaly detection

detectTrafficAnomaly needs 7 recent + 30 baseline days. The threshold
was 30, which would use a partial baseline (23 days). Now correctly
requires 37 rows before signaling.
2026-03-14 21:29:48 +04:00
Elie Habib
fe67111dc9 feat: harness engineering P0 - linting, testing, architecture docs (#1587)
* feat: harness engineering P0 - linting, testing, architecture docs

Add foundational infrastructure for agent-first development:

- AGENTS.md: agent entry point with progressive disclosure to deeper docs
- ARCHITECTURE.md: 12-section system reference with source-file refs and ownership rule
- Biome 2.4.7 linter with project-tuned rules, CI workflow (lint-code.yml)
- Architectural boundary lint enforcing forward-only dependency direction (lint-boundaries.mjs)
- Unit test CI workflow (test.yml), all 1083 tests passing
- Fixed 9 pre-existing test failures (bootstrap sync, deploy-config headers, globe parity, redis mocks, geometry URL, import.meta.env null safety)
- Fixed 12 architectural boundary violations (types moved to proper layers)
- Added 3 missing cache tier entries in gateway.ts
- Synced cache-keys.ts with bootstrap.js
- Renamed docs/architecture.mdx to "Design Philosophy" with cross-references
- Deprecated legacy docs/Docs_To_Review/ARCHITECTURE.md
- Harness engineering roadmap tracking doc

* fix: address PR review feedback on harness-engineering-p0

- countries-geojson.test.mjs: skip gracefully when CDN unreachable
  instead of failing CI on network issues
- country-geometry-overrides.test.mts: relax timing assertion
  (250ms -> 2000ms) for constrained CI environments
- lint-boundaries.mjs: implement the documented api/ boundary check
  (was documented but missing, causing false green)

* fix(lint): scan api/ .ts files in boundary check

The api/ boundary check only scanned .js/.mjs files, missing the 25
sebuf RPC .ts edge functions. Now scans .ts files with correct rules:
- Legacy .js: fully self-contained (no server/ or src/ imports)
- RPC .ts: may import server/ and src/generated/ (bundled at deploy),
  but blocks imports from src/ application code

* fix(lint): detect import() type expressions in boundary lint

- Move AppContext back to app/app-context.ts (aggregate type that
  references components/services/utils belongs at the top, not types/)
- Move HappyContentCategory and TechHQ to types/ (simple enums/interfaces)
- Boundary lint now catches import('@/layer') expressions, not just
  from '@/layer' imports
- correlation-engine imports of AppContext marked boundary-ignore
  (type-only imports of top-level aggregate)
2026-03-14 21:29:21 +04:00
Elie Habib
03bd3d0743 refactor: move shared types from services/config to types layer (#1597)
types/index.ts had backward imports via inline import() types
referencing @/services/ and @/config/ (higher layers). This
partially violated the architectural boundary rule from PR #1587.

Moved ThreatClassification, ThreatLevel, EventCategory,
HappyContentCategory, TechHQ, Port, and PortType definitions
into src/types/index.ts (lowest layer). Original files re-export
from @/types for backward compatibility.

Zero inline import() and zero backward from '@/...' imports remain
in src/types/index.ts.
2026-03-14 21:18:27 +04:00
Elie Habib
c82ffe5e75 ci: upgrade GitHub Actions to Node.js 24 (#1591)
* ci: upgrade GitHub Actions to Node.js 24-compatible versions

Node.js 20 actions are deprecated and will be forced to Node.js 24
starting June 2, 2026. Bump all affected actions:

- actions/checkout v4 -> v6
- actions/setup-node v4 -> v6
- actions/cache v4 -> v5
- actions/upload-artifact v4 -> v6
- docker/setup-qemu-action v3 -> v4
- docker/setup-buildx-action v3 -> v4
- docker/login-action v3 -> v4
- docker/metadata-action v5 -> v6
- docker/build-push-action v6 -> v7

* ci: pin all GitHub Actions to full commit SHAs

Replace floating @vN tags with immutable SHA refs for supply-chain
security. Tags can be moved; SHAs cannot. Each ref includes a # vN
comment for readability.

Also pins actions/cache@v5, actions/setup-go@v5, and all docker/*
actions that were previously using floating tags.
2026-03-14 21:15:39 +04:00
Elie Habib
6e8f3037e9 fix(feeds): remove dead RSS feeds, fix broken URLs, drop oversized sat group (#1596)
- Remove 403-blocked feeds: Breaking Defense, My Modern Met, AEI
- Fix Infobae URL: /feeds/rss/ -> /arc/outboundfeeds/rss/ (both files)
- Fix CSIS URL: /feed -> /rss.xml (server _feeds.ts)
- Drop 'active' from CelesTrak SAT_GROUPS (>2MB, always rejected)
2026-03-14 21:14:46 +04:00
Elie Habib
1eb8fab7cf refactor: dedupe query string array parsing (#1592) 2026-03-14 20:48:34 +04:00
Elie Habib
047ab0dfa1 ci: add proto freshness check to pre-push hook (#1594)
Mirror the CI proto-check workflow locally so stale generated code
is caught before push, not after. Only triggers when proto-related
files changed. Gracefully skips if buf/plugins are not installed.
2026-03-14 20:42:00 +04:00
Elie Habib
259cbd17d4 fix(supply-chain): use ArcGIS timestamp syntax for PortWatch date filter (#1593)
* fix(supply-chain): use ArcGIS timestamp syntax for PortWatch date filter

The PortWatch seed loop was silently failing on every cycle because
`date >= <epoch_ms>` is not valid ArcGIS Feature Service syntax.
ArcGIS requires `date >= timestamp 'YYYY-MM-DD HH:MM:SS'`.

This caused: no chart history, no transit counts, no WoW% in the
Supply Chain panel chokepoints tab (all added in PRs #1560/#1572/#1577).

Verified: 13/13 chokepoints return 175 days of history with the fix.

Also adds chokepointTransits to health.js STANDALONE_KEYS and SEED_META
so the transit counter seed freshness is monitored.

* fix(supply-chain): preserve full UTC time in PortWatch timestamp filter

pwEpochToTimestamp() was hardcoding 00:00:00, expanding the 180-day
window to the start of that UTC date. Now preserves HH:MM:SS from the
original epoch to match the intended query boundary exactly.
2026-03-14 20:36:05 +04:00
Elie Habib
66403ce13b refactor: dedupe theme meta color assignment (#1581) 2026-03-14 19:37:44 +04:00
Elie Habib
b6a026f330 refactor: dedupe economic fetch timeout wiring (#1588) 2026-03-14 19:37:30 +04:00
Elie Habib
79ae930eba feat(ui): glowing border pulse for search highlights (#1586)
* feat(ui): add glowing border pulse effect for search highlights

Replace subtle background fade with a 3-pulse blue glow animation
(3s duration) on panels and map countries when navigating via search
or breaking news alerts. Helps users spot the target result.

* fix(map): reset country highlight opacity after pulse animation ends

Without this, fill-opacity decays to near-zero at animation end,
leaving the country nearly invisible until the brief panel closes.

* fix(ui): cancel stale highlight timers and clean up pulse RAF on destroy

- Cancel previous setTimeout before scheduling new highlight, preventing
  premature class removal on rapid re-search (Codex P2)
- Extract shared applyHighlight() in SearchManager with WeakMap tracking
- Cancel countryPulseRaf in DeckGLMap.destroy() to prevent orphaned RAF

* fix(ui): theme-aware highlight opacity and per-element timer tracking

- Use getHighlightRestOpacity() to read current map theme instead of
  hardcoded dark-theme values (0.12), fixing light-theme regression
- Pulse animation base values now scale from theme-correct resting opacity
- BreakingNewsBanner: switch from single timer slot to WeakMap per-element
  tracking, preventing permanent highlight on rapid alert clicks
2026-03-14 19:36:42 +04:00
Elie Habib
b255672fee perf(relay): reduce proxy egress with upstream compression, keep-alive, and brotli (#1585)
* perf(relay): add upstream compression, keep-alive, brotli, and OREF pre-serialization

1. OpenSky upstream: request gzip/br/deflate via Accept-Encoding header
   and decompress responses, reducing inbound transfer on cache misses
2. HTTPS keep-alive agent: reuse TCP+TLS connections to OpenSky,
   eliminating ~2-5KB handshake overhead per request
3. Brotli support: sendCompressed and sendPreGzipped now prefer br
   encoding (~15-20% smaller than gzip on JSON). Pre-computed brotli
   buffers stored alongside gzip in all caches (OpenSky, AIS snapshot)
4. OREF pre-serialization: pre-compute alerts and history JSON + gzip +
   brotli buffers on each poll, switching handlers from on-the-fly
   sendCompressed to zero-CPU sendPreGzipped

All changes reduce proxy egress volume (charged by volume).

* fix(relay): re-serialize OREF cache on poll failure

Ensures _alertsCache reflects lastError when a poll fails,
so /oref/alerts clients see the error instead of stale
success data. Addresses Codex P1 review comment.

* fix(relay): reject on decompression failure and honor Accept-Encoding q-values

P1: _collectDecompressed now rejects on stream error instead of
resolving with partial data. Callers in _openskyRawFetch catch the
rejection and return { status: 0, error }, preventing malformed
responses from being cached as valid 200s.

P2: Replace naive includes('br')/includes('gzip') with
_acceptsEncoding() that parses q-values. A client sending
br;q=0 will no longer receive Brotli-encoded responses.
2026-03-14 19:36:17 +04:00
Frank
9d88aff739 fix: re-sync globe map on fullscreen transitions (#1510)
* fix: re-sync globe map on fullscreen transitions

* fix: call resize() instead of render() and anchor test regex

resize() propagates to MapLibre canvas resize in DeckGL mode,
fixing stale canvas dimensions after fullscreen transitions.
Anchor test regex to setupMapFullscreen to avoid matching the
wrong toggle block.

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
2026-03-14 18:11:11 +04:00
Elie Habib
945de7f0d0 feat(ui): add info tooltips to 10 panels and remove Supply Chain source footer (#1580)
Add "?" info tooltips explaining methodology to: Escalation Monitor,
Economic Warfare, Economic Indicators, Trade Policy, Supply Chain,
Markets, Sector Heatmap, Commodities, Market Radar, BTC ETF Tracker.

Remove the source attribution footer from the Supply Chain panel that
was listing all upstream data providers.
2026-03-14 17:30:54 +04:00
Elie Habib
39d11d2f0e fix(ui): add span-2 to content-heavy panels truncated at 380px (#1578)
* fix(ui): add span-2 to content-heavy panels truncated at 380px

Panels with tabs + dense list content were capped at 380px by
grid-auto-rows: minmax(200px, 380px). Adding span-2 gives them
2 grid rows (~760px) so content is visible without scrolling.

Panels: SupplyChain, TelegramIntel, Economic, TradePolicy,
GdeltIntel, SecurityAdvisories, CII, UcdpEvents, Displacement,
StrategicPosture.

* fix(ui): use defaultRowSpan option to preserve user resize preferences

Move span-2 from post-constructor classList.add to Panel's
defaultRowSpan option, applied before saved span restore. This
ensures user resize-to-1-row preferences persist across reloads
instead of being overridden by subclass constructors.

Also fix Panel restore logic to handle savedSpan=1 (was skipping
when savedSpan <= 1).
2026-03-14 17:24:51 +04:00
Elie Habib
9fe850586c fix(supply-chain): correct PortWatch ArcGIS API integration (#1577)
* fix(supply-chain): correct PortWatch ArcGIS service URL, field names, and chokepoint mappings

The PortWatch seed was returning no data because the ArcGIS service name,
WHERE clause fields, date field, and chokepoint names were all wrong.
Verified all 12 chokepoints return 175 days of data against the live API.
Added error logging to pwFetchAllPages for future debugging.

* fix(supply-chain): sync geofence names with relayName renames

CHOKEPOINT_GEOFENCES in ais-relay.cjs still used old names
('Strait of Malacca', 'Bab el-Mandeb', 'Strait of Gibraltar')
while _chokepoint-ids.ts relayName was updated. buildRelayLookup
does exact string match, so these 3 chokepoints had zero transit
counts despite relay data being present.

Rename geofence entries to match the new relayName values and
update corresponding test assertions.
2026-03-14 17:14:46 +04:00
Elie Habib
f32e57fc62 fix(insights): clear stale state on server fallback and AI flow toggle (#1576)
Two follow-up fixes for the server insights fallback (PR #1574):

1. onAiFlowChanged() checked lastClusters.length before re-rendering,
   dropping back to "waiting" state instead of trying server insights.
   Now delegates to updateInsights() which checks server path first.

2. renderServerInsights() rendered lastMissedStories from a previous
   clustering run. When clusters are empty (fallback path), these
   stale ML-detected stories are unrelated to the server stories.
   Clear lastMissedStories when entering server path with no clusters.
2026-03-14 16:12:48 +04:00
Elie Habib
d4088fede5 fix(feeds): update dead RSS feed URLs (#1575)
- a16z: a16z.com/feed/ -> www.a16z.news/feed
- First Round Review: /feed.xml -> /articles/rss
- RAND: Google News proxy -> rand.org/pubs/articles.xml (direct)
- Add www.a16z.news to allowed domains
2026-03-14 16:00:35 +04:00
Elie Habib
52c09b5beb fix(relay): add error logging to Wingbits flight fetcher (#1573)
The Wingbits function silently returned null on missing API key,
HTTP errors, and caught exceptions. Now logs the specific failure
reason so issues are visible in Railway logs.
2026-03-14 16:00:24 +04:00
Elie Habib
14bc59e543 fix(supply-chain): correct PortWatch ArcGIS URL and field mappings (#1572)
* fix(supply-chain): correct PortWatch ArcGIS URL, field names, and chokepoint mappings

The PortWatch seed was failing silently because:
1. Wrong service name: portal_chokepoint_daily -> Daily_Chokepoints_Data
2. Wrong query fields: chokepoint/observation_date -> portname/date (epoch)
3. Wrong data model: expected one row per vessel type, actual schema has
   all counts as columns (n_tanker, n_cargo, n_total) per row
4. Wrong chokepoint names: e.g. "Strait of Malacca" -> "Malacca Strait",
   "Bab el-Mandeb" -> "Bab el-Mandeb Strait", "Bosphorus" -> "Bosporus Strait"
5. Removed Dardanelles (not in PortWatch dataset)

Discovered via IMF PortWatch ArcGIS service directory and returnDistinctValues
query on the portname field.

* feat(supply-chain): add Korea, Dover, Kerch, Lombok chokepoints

Extend from 10 to 14 monitored chokepoints using PortWatch data
availability. All 4 new straits have IMF PortWatch coverage.

- Korea Strait: Japan-Korea trade, busiest East Asia corridor
- Dover Strait: world's busiest shipping lane
- Kerch Strait: war_zone (Russia controls, Ukraine grain restricted)
- Lombok Strait: Malacca bypass for VLCCs

Added to: handler config, canonical ID map, PortWatch seed names,
AIS relay transit counter, tests.

* docs: update maritime docs and changelog for 14 chokepoints + transit intelligence

- maritime-intelligence.mdx: 9 -> 14 chokepoints, add data source descriptions,
  add chart rendering note
- changelog.mdx + CHANGELOG.md: add [Unreleased] section for #1560 and #1572

* fix(tests): update portwatch test for pre-aggregated column model

pwClassifyVesselType was removed when switching to pre-aggregated
n_tanker/n_cargo/n_total columns. Update test to verify the new
field names instead.

* fix(supply-chain): sync canonical PortWatch names with actual ArcGIS feed

P1: Dardanelles has no PortWatch data (0 rows). Set portwatchName to empty
    string so it won't attempt fetch or show phantom zero history.
P2: portwatchNameToId() returned undefined for Malacca Strait, Bab el-Mandeb
    Strait, Gibraltar Strait, Bosporus Strait because canonical map used
    old names instead of actual ArcGIS portname values.

Fixed mappings:
  Strait of Malacca -> Malacca Strait
  Bab el-Mandeb -> Bab el-Mandeb Strait
  Strait of Gibraltar -> Gibraltar Strait
  Bosphorus -> Bosporus Strait
  Dardanelles -> '' (not in PortWatch)

* refactor(supply-chain): merge Dardanelles into Turkish Straits

IMF PortWatch tracks Bosphorus+Dardanelles as a single corridor
(Bosporus Strait). Keeping them separate caused double-counting in
AIS transit data and left Dardanelles with permanently empty history.

- Merge into single "Turkish Straits" entry (id stays 'bosphorus')
- Absorb all Dardanelles keywords (canakkale, gallipoli, aegean)
- Single wider AIS geofence (lat 40.70, lon 28.0, radius 1.5)
- 14 -> 13 chokepoints
- Update docs, changelog, tests

* fix: rename Turkish Straits to Bosporus Strait (match PortWatch naming)
2026-03-14 16:00:07 +04:00
Elie Habib
215f44215e fix(insights): check server insights before empty cluster guard (#1574)
The empty cluster guard (clusters.length === 0) returned early with
"UNAVAILABLE" before checking getServerInsights(). This caused the
AI Insights panel to show "Waiting for news data..." when clustering
failed or was slow, even though pre-computed server insights were
available via bootstrap.

Move the server insights check before the cluster guard so the panel
can render from Railway-seeded data regardless of client-side
clustering state.
2026-03-14 15:50:02 +04:00
Elie Habib
210fe3b915 feat(telegram): add media support, LIVE indicator, and error state (#1566)
* feat(telegram): add media support, LIVE indicator, and error state

- Display images and videos from Telegram messages with media grid
- Add LIVE indicator badge for messages < 10 minutes old
- Show error state when relay returns errors
- Add "View Source" action button per message
- Update TelegramItem type with optional mediaUrls field

Co-authored-by: lspassos1 <lspassos@icloud.com>

* fix(telegram): escape HTML entities in message text, add noopener to media links

- Escape &, <, >, " in item.text before safeHtml() to prevent
  untrusted Telegram messages from injecting links or styling
- Add noopener,noreferrer to window.open() on image click to
  prevent tabnabbing via window.opener reference

* fix(telegram): guard null text, add missing i18n keys

- Guard item.text with fallback to empty string for media-only messages
- Add missing viewSource and live i18n keys to all 21 locale files
- Replace hardcoded 'LIVE' string with t() call

* fix(telegram): surface relay errors to panel error state

data-loader catch block was swallowing fetch failures silently.
Now forwards error to TelegramIntelPanel.setData() so users see
the error state UI instead of a stale/empty panel.

* fix(telegram): remove hardcoded English error fallback

Use enabled:false without error field to trigger the existing
translated disabled message in TelegramIntelPanel.

---------

Co-authored-by: lspassos1 <lspassos@icloud.com>
2026-03-14 15:25:08 +04:00
Elie Habib
db6a4a2763 feat(correlation): server-side correlation engine seed + bootstrap hydration (#1571)
* feat(correlation): server-side correlation engine seed + bootstrap hydration

Move correlation card computation from client-side (per-browser, 10-30s delay)
to server-side (Railway cron, instant via bootstrap). Seed script reads 8 Redis
keys, runs 4 adapter signal collectors (military, escalation, economic, disaster),
clusters/scores/generates cards, writes to Redis with 10min TTL.

- New: scripts/seed-correlation.mjs (pure JS port of correlation engine)
- bootstrap.js: add correlationCards to FAST_KEYS tier
- health.js + seed-health.js: register for monitoring (maxStaleMin: 15)
- CorrelationPanel: consume bootstrap on construction, show "Analyzing..." only
  after live engine has run (not for bootstrap-only cards)
- _seed-utils.mjs: support opts.recordCount override (function or number)

* fix(correlation): stale timestamp fallback + coordinate-based country resolution

P1: news stories lacked per-story pubDate, causing Date.now() fallback on
every seed run. Now _clustering.mjs propagates pubDate through to
enrichedStories, and seed-correlation reads s.pubDate then generatedAt.

P2: normalizeToCode dropped signals with unparseable country names.
Added centroid-based coordinate fallback (haversine nearest-match within
800km) matching the live engine's getCountryAtCoordinates behavior.

* fix(correlation): add 11 missing country centroids to coordinate fallback

CI, CR, CV, CY, GA, IS, LA, SZ, TL, TT, XK were in the normalization
maps but missing from COUNTRY_CENTROIDS, causing coordinate-only signals
in those countries to be misclassified or dropped during bootstrap.

* fix(correlation): align protest/outage field names with actual Redis schema

Codex review P1 findings: seed-correlation read wrong field names from
Redis data.

Protests (unrest:events:v1): p.time -> p.occurredAt, p.lat/lon ->
p.location.latitude/longitude, severity enum SEVERITY_LEVEL_* mapping.

Outages (infra:outages:v1): o.pubDate -> o.detectedAt, o.lat/lon ->
o.location.latitude/longitude, severity enum OUTAGE_SEVERITY_* mapping.

Both escalation and disaster adapters updated. Old field names kept as
fallbacks for data shape compatibility.
2026-03-14 15:07:30 +04:00
Elie Habib
0383253a59 feat(supply-chain): chokepoint transit intelligence with 3 data sources (#1560)
* feat(supply-chain): replace S&P Global with 3 free maritime data sources

Replace expensive S&P Global Maritime API with IMF PortWatch (vessel transit
counts), CorridorRisk (risk intelligence), and AISStream chokepoint crossing
counter. All external API calls run on Railway relay, Vercel reads Redis only.

- Add 4 new chokepoints (10 total): Cape of Good Hope, Gibraltar, Bosphorus, Dardanelles
- Add TransitSummary proto (field 14) with today counts, WoW%, 180d history, risk context
- Add D3 multi-line chart (tanker vs cargo) with expandable chokepoint cards
- Add crossing detection with enter+dwell+exit semantics, 30min cooldown, 5min min dwell
- Add PortWatch seed loop (6h), CorridorRisk seed loop (1h), transit seed loop (10min)
- Add canonical chokepoint ID map for cross-source name resolution
- 177 tests passing across 6 test files

* fix(supply-chain): address P2 review findings

- Discard partial PortWatch pagination results on mid-page failure (prevents
  truncated history with wrong WoW numbers cached for 6h)
- Rename "Transit today" to "24h" label (rolling 24h window, not calendar day)
- Fix chart label from "30d" to "180d" (matches actual PortWatch query range)
- Add 30s initial seed for chokepoint transits on relay cold start (prevents
  10min gap of zero transit data)

* feat(supply-chain): swap D3 chart for TradingView lightweight-charts

Replace hand-rolled D3 SVG transit chart with lightweight-charts v5 canvas
rendering for Bloomberg-quality time-series visualization.

- Add TransitChart helper class with mount/destroy lifecycle, theme listener,
  and autoSize support
- Use MutationObserver (not rAF) to mount chart after setContent debounce
- Clean up chart on tab switch, collapse, and re-render (no orphaned canvases)
- Respond to theme-changed events via chart.applyOptions()
- D3 stays for other 5 components (ProgressCharts, RenewableEnergy, etc.)

* feat(supply-chain): add geo coords and trade routes for 4 new chokepoints

Cherry-pick from PR #1511: Cape of Good Hope, Gibraltar, Bosphorus, and
Dardanelles map-layer coordinates and trade route definitions.

* fix(supply-chain): health.js v2->v4 key + double cache TTLs for missed seeds

- health.js chokepoints key was still v2, now v4 (matches handler + bootstrap)
- PortWatch TTL: 21600s (6h) -> 43200s (12h), seed interval stays 6h
- CorridorRisk TTL: 3600s (1h) -> 7200s (2h), seed interval stays 1h
- Ensures one missed seed run doesn't expire the key and cause empty data
2026-03-14 14:20:49 +04:00
Elie Habib
158e9693d8 feat(popups): enhance vessel details with flags, USNI intel, and tracking (#1567)
* feat(popups): enhance vessel details with flags, USNI intel, and tracking history

- Add flag emojis for operator countries in vessel and cluster popups
- Add collapsible USNI intel section (strike group, region, description)
- Add collapsible tracking history trail from vessel.track
- Display nearChokepoint, nearBase, lastSeen with aisGapMinutes
- Restructure header with hull badge and badge row
- Fix hemisphere bug: use N/S and E/W based on coordinate sign
- Fix lastAisUpdate null check to prevent "Invalid Date"
- Deduplicate cluster vessel rendering into shared helper
- Add i18n keys (recentTracking, lastReport, nearChokepoint,
  nearBase, lastSeen) across all 21 locale files

Co-authored-by: lspassos1 <lspassos@icloud.com>

* fix(popups): correct track coord order, fix CSS variables, remove dead styles

- Fix coordinate swap: track stores [lat, lon], was passing (lon, lat)
  to formatCoord, now correctly passes (lat, lon)
- Replace undefined --semantic-high-rgb CSS variable with literal
  rgba(255, 136, 0, ...) values matching --semantic-high: #ff8800
- Remove dead CSS classes (.popup-badge.usni-deployed/underway/in-port)
  that were never applied in HTML
- Add missing CSS for .popup-stat.warning (orange highlight for
  nearChokepoint) and .popup-stat.full-width (span full grid row)

---------

Co-authored-by: lspassos1 <lspassos@icloud.com>
2026-03-14 14:07:22 +04:00
Elie Habib
68f8e69f3e refactor: dedupe ACLED OAuth token request flow (#1569) 2026-03-14 13:59:39 +04:00
Elie Habib
24b502d0b1 fix(health): bump theaterPosture maxStaleMin from 30 to 60 (#1568)
The 10-min seed interval combined with occasional slow OpenSky API
responses causes the 30-min threshold to trigger false STALE_SEED
warnings on UptimeRobot. 60 min gives adequate headroom.
2026-03-14 13:45:49 +04:00